> Sanjay.Summary

Data Engineer focused on building and maintaining efficient data pipelines for both batch and streaming tasks. Skilled in automating processes with Python. Proficient in optimizing Spark scripts and orchestrating complex workflows using Apache Airflow and deploying applications effectively. Excels at organizing and analyzing diverse data sources to drive informed decision-making. Committed to ongoing learning and staying ahead of technological advancements.

> Sanjay.Skills

● Languages: Expert in Python, proficient in Java, Javascript, SQL, PL/SQL, Shell Scripting, R.

● Web Technologies: Proficient in HTML, CSS, JavaScript, PHP, Bootstrap, Jquery.

● Technologies: Big Data, Apache Spark, Hadoop, Oracle DB, Airflow, PySpark, Docker, Redis.

● Tools: Experienced with PyCharm,  Anaconda, Jyputer, Visual Studio Code,  Git, BitBucket, Putty, FileZilla, Jira

> Sanjay.Education

Master of Science, Computer Science, GPA: 3.9/4 
University of North Carolina at Charlotte, Charlotte, NC 

Course Work: Algorithm & Data Structures, Intelligent Systems, Visual Analytics, Information Visualization, Big Data, Databases, Computer Networks, Software System Design & Implementation, Knowledge Based Systems, Data Science, and Machine Learning

> Sanjay.WorkExperience

Operations Assistant - Data Engineering, University of North Carolina at Charlotte, USA
Feb 2023 - May 2024
● Designed and implemented efficient Data Pipelines to manage and automate event setup processes, integrating SQL databases to store and retrieve event scheduling, setup instructions, and resource allocation data.
● Leveraged Apache Airflow to manage workflows, allocate resources by event schedule, generate tokens for devices, manage room access, and send timely event emails, ensuring effective resource and time management.
● Developed Python scripts to collect, analyze, perform ETL operations, and store feedback for future analysis.
● Used Data Processing techniques with PySpark to analyze space utilization data, identifying high-demand areas and underused spaces, helping the university optimize space allocation and improve event planning efficiency.
● Analyzed and organized Raw Data sourced from various channels, Ingesting data from different file formats, including older files, to ensure comprehensive data integration and consistency.
● Created Tableau dashboards to track event statistics, space usage, and customer satisfaction
● Utilized Data Visualizations and Heat maps to identify high-demand areas (hotspots) for event spaces across the campus.
● Improved transportation efficiency and resource use by dynamically adjusting routes based on passenger load, reducing waiting times and storing data for future scheduling with Airflow and Spark.

        

Systems Engineer - Data Engineering, Tata Consultancy Services Limited, Bangalore, India
Jan 2021 - Dec 2022
● Designed, enhanced, and managed Data Ingestion Pipelines including ETL/ELT processes.
● Authored Python Scripts to automate data tasks, increasing efficiency and reducing manual intervention.
● Converted Scala files to Python for better Version Control and environment compatibility.
● Orchestrated multifaceted workflows using Apache Airflow, improving task scheduling and dependencies.
● Deployed multi-environment apps via YARN and conducted advanced tuning of PySpark, enhancing system performance.
● Automated and optimized Spark Scripts to resolve small file issues in HDFS, enhancing Storage Efficiency.
● Optimized SQL Scripts for large data sets, improving data processing efficiency by 40%.
● Developed pruning procedures for Docker resources, reducing system overhead and improving container management.
● Developed personalized Promotions for products using Location Data to boost customer engagement, and implemented automated Email notifications and price tracking systems to enhance customer communication.
● Worked with ETL operations, data storage, and analysis using AWS Glue, Amazon Redshift, EMR, and QuickSight.
● Worked with Data Visualization tools like Power BI and Tableau, creating insightful interactive visualizations.


Campus Student Assistant - Sree Vidyanikethan Engineering College (JNTUA)
Jun 2019 - Dec 2020 
● Led Python and Big Data instructional sessions for 40 students.
● Provided personalized debugging and problem-solving support.

> Sanjay.Projects

● Web Scraping:
  ● Implemented a comprehensive Web Scraping solution using BeautifulSoup and URL handling module to extract pricing and data from an e-commerce website.
  ● Data was then organized for easy comprehension, enabling future price tracking and forecasting.
  ● Demonstrated proficiency in data extraction and manipulation.

● NextGen overseas Edu-Social Networking:
  ● Led the development of a web-based social networking platform focused on overseas education.
  ● Enabled like-minded students to connect, share university insights, and exchange information.
  ● Leveraged HTML, CSS, JavaScript, PHP, and MySQL to create an interactive platform fostering educational collaboration.

● Attendance Management System:
  ● Utilized OpenCV-Python Facial recognition to develop an Attendance Management System captures facial features for attendance marking, facilitating streamlined tracking and management oversight.
  ● Significantly improved attendance accuracy and efficiency.

● Driver Drowsiness Detection System:
  ● Created an application utilizing OpenCV for face and eye detection.
  ● Employed a CNN model for predicting driver alertness.
  ● Actively alerts drivers with sound when drowsiness is detected, enhancing road safety.

● Alumni gate pass using QR code:
  ● Developed an efficient system generating unique QR codes for alumni gate pass.
  ● Ensured hassle-free access to university premises.
  ● Enhanced security and streamlined access control.

> Sanjay.Certificates

● Infosys Certified Software Programmer - View Certificate
● Apache Spark Developer using Python - View Certificate
● Apache Airflow: The Hands-On Guide - View Certificate