B Lavanya

Azure Data Engineer

Balaji Colony, 583101, Ballari, IN.

About

Results-driven Azure Data Engineer with 1.5 years of experience in designing, developing, and deploying scalable data solutions. Proven expertise in building end-to-end data pipelines using Azure Data Factory, Azure Databricks, Spark (PySpark, SQL), and Python for large-scale data ingestion, processing, and migration. Adept at ensuring data quality, optimizing performance, and collaborating within Agile frameworks to deliver robust data solutions that drive business insights and operational efficiency.

Work

Kion Group

|

Data Engineer

Chennai, Tamil Nadu, India

→

Summary

As a Data Engineer at Kion Group, contributed to the Jarvis_ML (Telecom) project by designing and implementing end-to-end data solutions, focusing on scalable data ingestion and processing within the Azure ecosystem.

Highlights

Designed and deployed end-to-end data solutions on Azure for the Jarvis_ML telecom project, encompassing storage, integration, and processing.

Developed and optimized data pipelines in Azure Data Factory, integrating diverse sources (Azure SQL, ADLS, Datawarehouses) for efficient ETL operations.

Engineered PySpark scripts for large-scale data transformation, pushing processed data to Azure Data Lake Storage Gen2, and implementing daily incremental load strategies.

Constructed complex distributed systems for high-volume data handling, collecting key metrics, and building robust data pipelines for analytics, while collaborating with Agile teams.

PWC

|

Associate

Chennai, Tamil Nadu, India

→

Summary

As an Associate at PWC, served as a Data Engineer on the CDW project, focusing on designing and optimizing data processing workflows and database solutions leveraging Azure services.

Highlights

Developed and deployed data processing workflows using Azure Databricks and Spark, facilitating large-scale data transformations for the CDW project.

Engineered and implemented efficient database solutions using Azure Data Lake for optimized data storage and retrieval.

Optimized data pipelines and Spark jobs in Azure Databricks, enhancing performance through configuration tuning, caching, and data partitioning techniques.

Established robust data lineage and metadata management solutions, improving data governance and traceability while collaborating with cross-functional teams.

Education

Leading Indian University

Chennai, Tamil Nadu, India
→
B. Tech Electronics and Communication Engineering Grade:

Skills

Big Data Technologies

Spark, PySpark, Spark SQL, Data Partitioning, Performance Tuning, Distributed Systems.

Cloud Services

Azure Databricks, Azure Data Factory (ADF), Azure Data Lake, Azure Blob Storage, Azure Logic Apps, Azure Key Vault, Azure SQL Database, ADLS Gen2, On-Prem to Azure Migration.

Programming Languages

Python, SQL.

Databases

MySQL, SQL Server, Oracle, Data Warehousing.

Methodologies & Tools

Scrum, Agile, Jira, Git, Azure DevOps (ADO).

Core Data Engineering

Data Ingestion, Data Transformation, ETL, Data Quality, Data Migration, Data Storage Solutions, Data Pipelines, Data Lineage, Metadata Management, Troubleshooting, Data Analytics, Reporting, Documentation.