Senior Data Engineer

Senior Data Engineer

Role

As a Senior Data Engineer, you will:

• Design, develop, test, and maintain scalable data pipelines and data products supporting Enterprise Credit Risk initiatives.

• Build and optimize batch and real-time data processing solutions using modern cloud and lakehouse technologies.

• Partner with Risk, Product, Data Science, and Engineering teams to understand business requirements and translate them into technical solutions.

• Develop reusable frameworks, components, and engineering patterns that improve development efficiency and platform consistency.

• Implement data quality controls, monitoring, alerting, and observability capabilities to ensure data reliability and trustworthiness.

• Support the migration and modernization of existing data assets and workloads into Databricks and cloud-native platforms.

• Build and maintain curated datasets that support lending decisioning, portfolio risk management, and advanced analytics use cases.

• Participate in architecture discussions and contribute technical recommendations for platform enhancements.

• Troubleshoot production issues and drive root cause analysis to improve platform reliability and performance.

• Follow engineering best practices for code quality, testing, deployment automation, security, and operational excellence.

• Mentor junior engineers through code reviews, technical guidance, and knowledge sharing.

• Continuously evaluate opportunities to improve scalability, performance, cost efficiency, and maintainability of data platforms.

All About You

Essential Skills & Experience

• Strong experience developing and maintaining enterprise-scale data engineering solutions.

• Hands-on experience with Databricks, Apache Spark, PySpark, Hadoop, SQL, and Python.

• Experience building ETL/ELT pipelines and large-scale data processing applications.

• Experience working with cloud-based data platforms and storage technologies.

• Strong understanding of data modeling concepts and analytical data structures.

• Experience implementing automated testing, monitoring, and data quality practices.

• Experience with Data formats ( Parquet, Avro, ORC )

• Knowledge of CI/CD pipelines, Git-based development workflows, and DevOps principles.

• Experience with Workflow orchestration Tools like Airflow

• Strong analytical and problem-solving skills with the ability to work independently on complex technical challenges.

• Effective communication skills with the ability to collaborate across technical and business teams.

• Experience mentoring or supporting less experienced engineers.

• Knowledge of Java Based application development is a huge Plus.

•

Preferred Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related STEM field or alternative minimum of 5 years of experience in a related field.

Technical Skills

Experience with several of the following technologies:

• Databricks

• Apache Spark / PySpark

• AirFlow

• SQL

• Python

• Delta Lake

• Azure Data Services

• Data Factory or equivalent orchestration tools

• Kafka or streaming platforms

• GitHub

• CI/CD automation

• Data Quality and Observability tools

ADVANCED ENGLISH

REMOTE