Senior Data Engineer – AWS & Databricks
TALPRO INDIA PRIVATE LIMITED · Bangalore South, Karnataka, India · ₹2M/yr
IT Services and IT Consulting · 51-200 employees
About the role
Design, develop, and optimize scalable ETL pipelines using Databricks and Apache Spark. Collaborate with cross-functional teams to ensure data quality, governance, and effective deployment of data solutions.
What they look for
Requirements
Requires 6–10 years of experience in data engineering with strong expertise in AWS, Databricks, and programming in Python or Scala. Candidates must possess excellent SQL skills and a solid understanding of distributed computing and data warehousing concepts.
Full description
Senior Data Engineer – AWS & Databricks
Role Details
- Experience: 6–10 Years
- Location: Bengaluru
- Work Mode: Hybrid (2–3 days/week from office mandatory)
- Mandatory Skills: AWS + Databricks
- Budget: 17 LPA
- Joining: Immediate / Early joiners preferred
- Contract Duration: 12 months
Role Overview
We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in AWS and Databricks to design, implement, and optimize scalable data pipelines and analytics solutions. The ideal candidate should have deep expertise in Apache Spark, ETL design, cloud-native data engineering, and modern data governance practices.
You will collaborate closely with data scientists, analysts, and business stakeholders to deliver robust, high-performance data solutions that drive strategic insights.
Key Responsibilities
- Design, develop, and optimize ETL pipelines using Databricks and Apache Spark.
- Build scalable and efficient data models and schemas.
- Collaborate with data scientists to support ML model deployment.
- Ensure data quality, consistency, reliability, and governance.
- Optimize SQL queries and troubleshoot performance bottlenecks.
- Implement data security, access controls, and governance policies.
- Work with Databricks Unity Catalog for centralized governance.
- Develop and maintain technical documentation.
- Participate in code reviews and mentor junior engineers.
- Translate business requirements into scalable technical solutions.
- Support real-time and streaming data processing use cases.
Required Skills & Qualifications
Technical Skills
- 6–10 years of experience in Data Engineering.
- Strong expertise in Databricks Platform and Apache Spark.
- Hands-on experience with AWS cloud services.
- Excellent SQL skills.
- Strong programming skills in Python or Scala.
- Experience building robust ETL processes and data pipelines.
- Experience with Databricks Unity Catalog.
- Understanding of distributed computing and big data architecture.
- Familiarity with Git and version control practices.
- Experience with real-time/streaming data processing.
- Strong understanding of data warehousing and analytics concepts.
Nice to Have
- Databricks certifications.
- Experience with data governance and compliance standards.
- Exposure to Azure or GCP (secondary cloud platforms).
- Experience in fast-paced enterprise environments.
Soft Skills
- Strong analytical and problem-solving ability.
- Excellent communication and stakeholder collaboration skills.
- Ability to manage multiple priorities in a dynamic environment.
- Mentoring and team collaboration mindset.
