Data Engineer
techcarrot FZ LLC Noida, Uttar Pradesh, India
IT Services and IT Consulting · 201-500 employees
Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.
About the role
Assess and integrate complex data sources into the Azure Data platform, building robust, reusable, and scalable pipelines with the Microsoft and Databricks stack. Collaborate across engineering teams, contribute to design and feature definition, communicate insights to stakeholders, and manage project updates, reports, plans, milestones, and dependencies.
What they look for
Requirements
Requires 5–7 years of practical experience, a bachelor’s degree in a relevant field, and production experience with the Azure cloud stack. Candidates should have hands-on Azure Databricks and Spark ETL experience, database and data analysis expertise, proficiency in Python, Scala, or SQL, and Azure DevOps CI/CD experience; project management experience is an advantage.
Full description
As a Data Engineer in our team, you will be responsible for assessing complex new data sources and quickly turning these into business insights. You also will support the implementation and integration of these new data sources into our Azure Data platform.
Responsibilities:
- Collect progress updates from Squad on a regular basis.
- Consolidate updates into weekly project status reports and share with the Squad Leads for their review.
- You participate in early state design and feature definition activities.
- Responsible for implementing robust data pipeline using Microsoft, Databricks Stack
- Responsible for creating reusable and scalable data pipelines.
- You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.
- You have strong communication skills to effectively convey complex data insights to non-technical stakeholders.
- Develop and manage L3-level project plans with sufficient details for tasks, milestones, and dependencies
Requirements
Skills:
- Knowledge and experience with relational and non-relational databases.
- Strong analytic skills related to working with structured and unstructured datasets.
- Hands-on experience in Azure Databricks utilizing Spark to develop ETL pipelines.
- Strong proficiency in data analysis, manipulation, and statistical modeling using tools like Spark, Python, Scala, SQL, or similar languages.
- Strong proficiency in building JIRA dashboards based on tasks.
- Strong experience in Project Management will be considered as an additional advantage.
- Azure DevOps experience to deploy the data pipelines through CI/CD.
Qualifications and Experience:
- Minimum 5-7 years of practical experience.
- Bachelor’s degree in computer science, software engineering, information technology, or a related field.
- Azure cloud stack in-production experience.
Similar roles
-
Senior Data Engineer II
Oscar Health New York, New York, United States · $181K–$237K/yr
-
Data Engineer
AcuityMD $113K–$159K/yr
-
Staff Data Engineer, Analytics
Afresh San Francisco, California, United States · $168K–$250K/yr
-
Data Engineer - Agentic AI
Defense Unicorns Springfield, Virginia, United States · $149K–$201K/yr
-
Senior Data Engineer
Southend Pharmacy Houston, Texas, United States · €45K–€65K/yr
-
Senior Data Engineer (Talent Pool)
ioet San José de Alluriquín, Santo Domingo de los Tsáchilas, Ecuador