Data Engineer, Sr
Insight Global Dunwoody, Georgia, United States
Business Consulting and Services · 10,001+ employees
Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.
About the role
The Senior Data Engineer will design, develop, and maintain scalable data pipelines while ensuring the reliability and stability of enterprise data platforms. This role involves driving DataOps best practices, collaborating with cross-functional teams, and implementing automated solutions to support analytics and AI initiatives.
What they look for
Requirements
Candidates must have 5-7 years of experience in data engineering with strong proficiency in SQL, Python, and PySpark. Hands-on expertise with Azure data services, Databricks, and CI/CD practices is required to succeed in this production-focused role.
Full description
Overview
The Data & Insights organization at Insight Global is responsible for delivering Trusted Data & Verified Insights to drive business decisions across the company. As a Senior Data Engineer, you will be part of the Data Engineering & Operations team responsible for enhancing and maintaining scalable data pipelines, supporting and improving platform operations processes, and enabling analytics, data science, and AI use cases.
This is a hands-on role reporting to the Data Operations Manager with direct impact on Insight Global’s ability to deliver high-quality, actionable data to the business.
Responsibilities
- Design, develop, and enhance scalable data pipelines supporting analytics, reporting, AI/ML, and operational business processes.
- Own the reliability and stability of enterprise data platforms by proactively monitoring pipelines, troubleshooting production issues, and ensuring adherence to SLAs and operational standards.
- Lead root cause analysis and incident resolution efforts, driving permanent fixes, automation opportunities, and preventative controls to reduce recurring issues.
- Develop and maintain data solutions using Databricks, Azure Data Factory, SQL, and PySpark, following enterprise architecture and engineering standards.
- Drive DataOps best practices, including CI/CD implementation, automated testing, code reviews, deployment automation, release management, and environment governance.
- Improve data quality, performance, observability, and operational efficiency through monitoring, alerting, lineage tracking, logging, and automation initiatives.
- Work with structured, semi-structured, and streaming data, building scalable ingestion and transformation frameworks.
- Collaborate with business stakeholders, analytics teams, architects, product owners, and application teams to deliver reliable and business-aligned data solutions.
- Identify opportunities for continuous improvement in processes, architecture, governance, security, cost optimization, and platform scalability.
Qualifications
Required Skills
- 5-7 years of experience in Data Engineering, DataOps, Analytics Engineering, or Enterprise Data Platforms.
- Strong expertise in advanced SQL development, query optimization, data analysis, and troubleshooting.
- Proficiency in Python and PySpark for large-scale data transformation, pipeline development, and automation.
- Hands-on experience with Databricks, Delta Lake, Azure Data Factory, and Azure data services.
- Experience designing and supporting batch, near real-time, and streaming data solutions.
- Strong understanding of CI/CD pipelines, Git-based source control, pipeline testing, and DevOps/DataOps practices.
- Experience with monitoring, observability, logging, alerting, and production support processes.
- Knowledge of data modeling, data warehousing concepts, and dimensional design principles
Preferred Skills
- Experience with Databricks Unity Catalog.
- Knowledge of data governance, data quality frameworks, security, and compliance requirements.
- Proven experience managing production incidents, performing root cause analysis, and driving long-term remediation plans.
- Ability to prioritize operational stability while balancing enhancement and project delivery commitments.
- Experience working in Agile, Scrum, or Scrumban environments.
This Role Is Ideal If You:
- Are excited to be part of a data team supporting data pipelines in Databricks.
- Thrive in Agile environments and enjoy working across disciplines.
- Take ownership of operational responsibilities and ability to fix data issues.
- Are passionate about delivering reliable, high-quality data solutions.
- Embody Insight Global’s shared values and contribute to a collaborative, high-performance culture.
Similar roles
-
Experienced - Data Engineer DATABRICKS - NextHub Bari
Deloitte Italia Bari, Apulia, Italy · €28K–€44K/yr
-
Senior Data Engineer - Databricks Specialist
University of Otago Otago, New Zealand
-
Senior Data Engineer
UBDS Group Manchester, England, United Kingdom
-
Data Engineer
UBDS Group London, England, United Kingdom · £40K–£48K/yr
-
Data Engineer II, AWS Analytics Engineering
Amazon Seattle, Washington, United States · $132K–$179K/yr
-
Data Engineer Intern
Munich Re Singapore, Singapore, Singapore