Require a Data Engineer in Pune

Key Responsibilities:

• Design, develop, and maintain new data capabilities and infrastructure for Mastercard's Sustainable Technology Internal Data Lake.

• Create new data pipelines, data transfers, and compliance-oriented infrastructure to facilitate seamless data utilization within on-premise/cloud environments.

• Identify existing data capability and infrastructure gaps or opportunities within and across initiatives and provide subject matter expertise in support of remediation.

• Collaborate with technical teams and business stakeholders to understand data requirements and translate them into technical solutions.

• Work with large datasets, ensuring data quality, accuracy, and performance.

• Implement data transformation, integration, and validation processes to support analytics/BI and reporting needs.

• Optimize and fine-tune data pipelines for improved speed, reliability, and efficiency.

• Implement best practices for data storage, retrieval, and archival to ensure data accessibility and security.

• Troubleshoot and resolve data-related issues, collaborating with the team to identify root causes.

• Document data processes, data lineage, and technical specifications for future reference.

• Participate in code reviews, ensuring adherence to coding standards and best practices.

• Collaborate with DevOps teams to automate deployment and monitoring of data pipelines.

• Additional tasks as required.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field.

• Proven experience as a Data Engineer / Scientist or similar role.

• Deep understanding & expertise in data engineering, ETL/ELT processes, data warehousing, and data modeling.

• Strong command of data integration techniques and data quality management.

• Hands-on experience with data technologies such as Databricks, Spark, Python, SQL, Hadoop, Airflow.

• Familiarity with cloud platforms and services, such as AWS or Azure.

• Excellent analytical, problem-solving skills and ability to provide innovative data solutions.

• Exceptional interpersonal skills with proven experience in relationship building and partnering, must work well in both team/individual settings and must be able to work with a geographically dispersed team.

• Strong written and oral communication skills. Attention to detail is a must.

• Motivated self-starter with ability to excel at multi-tasking in a fast-paced environment and able to function under pressure with a high degree of initiative to drive results.

• Ability to quickly learn and implement new technologies and perform POC to explore best solutions for problem statements.

• Flexibility to work as a member of a matrix-based diverse and geographically distributed project team.

• 0.6 -1.5 years of experience in Data Engineering or Warehouse-related projects.

• Expertise in Data Engineering and Data Analysis: implementing multiple end-to-end Data Engineering or Warehouse projects in Big Data environment.

• Experience building data pipelines through Spark with Scala/Python/Java in Databricks or Hadoop environment

• Experience working with databases like MS SQL Server or Oracle, and strong SQL knowledge.

• Experience automating data flow processes in a Big Data environment using Airflow or similar tools.

• Experience working in Agile teams.