Senior Data Engineer
XPT Software Australia Pty Ltd Sydney, New South Wales, Australia
IT Services and IT Consulting · 51-200 employees
About the role
The Senior Data Engineer will design, build, and maintain scalable, secure, and high-performance data platforms to support analytics and machine learning. They will also mentor junior engineers and collaborate with cross-functional teams to drive engineering best practices.
What they look for
Requirements
Candidates must have strong hands-on experience with Azure data tools, PySpark, and complex SQL. Proficiency in mentoring team members and communicating technical trade-offs to stakeholders is also required.
Full description
Job Description:
Must Have Skills:
Strong hands‑on experience with Azure Data Lake Gen2, Synapse, Databricks
Solid experience in PySpark / Spark SQL, Databricks performance tuning & job optimization
Proficient in Complex SQL (window functions, CTEs, execution plans)
Strong understanding in Mentoring junior and mid‑level engineers, Communicating trade‑offs to architects and product teams
Experience working with Azure DevOps or GitHub Actions
Nice to Have Skills:
Experience with Streaming & Real‑Time Data Processing on Designing low‑latency pipelines.
Exposure to Data Governance & Metadata Management in Data classification & sensitivity labels
Familiarity with Advanced Python Engineering Practices on Performance‑aware Python design
Experience working in Analytics & BI Awareness
Knowledge of Cloud Cost Optimization & FinOps
Detailed Job Description
The Senior Data Engineer will be responsible for designing, building, and operating scalable, secure, and high‑performance data platforms. The role involves critical data pipelines and architectures that power analytics, reporting, and advanced data use cases such as machine learning and real‑time insights.
The candidate will collaborate closely with analytics, data science, product, and business teams while mentoring engineers and driving engineering best practices across the data organization.
The role combines technical leadership, architecture ownership, and business impact.
Similar roles
-
ML Data Engineer (m/f/d) - Sensor Data & Pipelines
Autonomous Teaming Solutions ATS GmbH Munich, Bavaria, Germany
-
Data Engineer
Ford Motor Company Chennai, Tamil Nadu, India
-
(Senior) Data Engineer with AI - Freelance
Netguru Poland · €62K–€77K/yr
-
Senior Data Engineer
Great Eastern Cuenca, Azuay, Ecuador
-
Data Engineer
Booking Experts Enschede, Overijssel, Netherlands · €49K–€83K/yr
-
Data Engineer
Quicklizard Petah Tikva, Center District, Israel