Senior Azure Data Engineer

About USEREADY

USEReady helps enterprises apply AI and agentic intelligence to improve decisions, automate operations, and build smarter, more autonomous business systems.

For more than a decade, we have built the foundations that make this possible by modernizing BI environments, migrating legacy platforms, improving data quality, and enabling governed, cloud-first architectures. These foundations now support the next step: AI-driven insights, automated intelligence, and agent-powered decision support that reduce complexity and accelerate outcomes.

We work closely with technology leaders such as AWS, Elementum, Snowflake, Tableau, Databricks, and others to help organizations modernize analytics, strengthen governance, and deploy agentic automation with confidence. We founded in 2011 and Headquartered in New York City with 450+ experts across the United States, Canada, India, and Singapore, we serve industries including financial services, healthcare, manufacturing, government, education, and retail. Our deep expertise, player-coach delivery model, and focus on fast, measurable results make us a trusted partner for building an AI-ready enterprise.

About the Role

We are looking for an experienced Senior Azure Data Engineer with strong hands-on expertise in Databricks, Azure Data Factory (ADF), PySpark, Python, and SQL. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data engineering solutions on Azure.

You will work with modern Azure data technologies to build high-performance data pipelines, implement data governance, optimize Databricks workloads, and deliver reliable data platforms that support analytics and business intelligence initiatives.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Azure Databricks and Azure Data Factory (ADF).

• Develop data processing solutions using Python, PySpark, SQL, and Apache Spark.

• Build and maintain Delta Lake tables and scalable data models for analytics and BI workloads.

• Implement and manage Databricks Unity Catalog for data governance, security, access control, and data discovery.

• Optimize Databricks jobs, Spark workloads, SQL queries, and data pipelines for performance, scalability, and cost efficiency.

• Apply Apache Spark internals and tuning techniques to improve distributed data processing performance.

• Develop complex and optimized SQL queries, including efficient joins and processing of large datasets.

• Design and implement ETL/ELT workflows using Azure Data Factory.

• Work with Azure data services including Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL Database.

• Implement data quality checks, validation, reconciliation, monitoring, and error-handling mechanisms.

• Implement data governance, security, access control, and data lineage across data platforms.

• Troubleshoot pipeline failures and performance issues and ensure high availability and reliability of data solutions.

• Collaborate with business stakeholders, data architects, analysts, and engineering teams to understand requirements and deliver effective data solutions.

• Automate manual processes and continuously improve data engineering workflows and operational efficiency.

• Mentor junior data engineers and promote best practices across Azure and Databricks data engineering.

Required Skills & Expertise

• 6+ years of experience in Data Engineering, with strong experience in Azure-based data platforms.

• Strong hands-on experience with Azure Databricks.

• Expertise in Databricks Notebooks using Python and SQL.

• Strong experience with Databricks Unity Catalog, governance, security, and access control.

• Hands-on experience with Databricks performance optimization and cost/compute optimization.

• Strong experience with Azure Data Factory (ADF) and ETL/ELT orchestration.

• Advanced Python and PySpark programming skills.

• Strong understanding of Apache Spark architecture, internals, and performance tuning.

• Strong SQL skills, including complex queries, query optimization, joins, aggregations, and large-volume data processing.

• Strong understanding of data warehousing concepts and data modeling, including Star and Snowflake schemas.

• Experience with Delta Lake and modern lakehouse architecture.

• Good knowledge of Azure Data Lake Storage (ADLS).

• Experience with Azure Synapse Analytics and Azure SQL Database.

• Understanding of data governance, data security, access management, and data lineage.

• Experience working with large-scale distributed data processing environments.

• Strong problem-solving, communication, and collaboration skills.

Preferred Qualifications

• Databricks certifications are a plus.

• Microsoft Azure Data Engineering certifications are a plus.

• Experience working with enterprise-scale Azure data platforms.

• Experience implementing data quality, governance, and security frameworks.

• Experience mentoring or technically guiding junior data engineers.

Education

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field is preferred.