Data Engineer ID94415

We are looking for a Data Engineer to execute large-scale data platform migrations to Snowflake and support reliable Airflow workflows.

The mandatory requirements are 3+ years of Python, advanced SQL, Snowflake data modeling, and hands-on experience with Airflow, Spark/PySpark, and AWS S3.

RECRUITMENT PROCESS

1. Application — Share a few details about your experience and background.

2. Coding Challenge — If applicable, complete it in the coding language you are most comfortable with.

3. Video Interview — Record a short video introduction in English.

4. Technical Interview or Hiring Manager Interview — Discuss your experience and fit for the role.

5. Offer — If it’s a match, you’ll receive an offer.

WHAT YOU’LL GAIN

• Remote work — Work from where you feel most productive.

• Local presence in India — Work in a structured and compliant environment aligned with Indian regulations.

• Competitive compensation in INR — Receive compensation in INR, plus support for learning, education, and wellness.

• Exciting projects — Work with modern technologies for global clients and fast-growing companies.

MUST HAVES

• Strong software engineering fundamentals.

• 3+ years of hands-on Python development experience.

• Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.

• Snowflake data modeling.

• Apache Airflow — developing and testing pipelines in Airflow and Python.

• Apache Kafka.

• Apache Spark, including hands-on PySpark experience.

• Iceberg and Parquet data formats.

• AWS experience, particularly S3 (data lakes, partitioning, and lifecycle policies).

• Testing strategy experience, with the ability to hands-on debug and test code.

• At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.

• Overlap until 12 PM Pacific Time.

• Upper-intermediate English level.

NICE TO HAVES

• Hands-on experience with Snowflake migrations (data warehouse modernization, performance and cost optimization).

• Experience with Apache Airflow migrations (version upgrades, executor changes, or platform migrations).

• Experience using AI tools for development.

WHAT YOU WILL DO

• Work embedded within the data engineering function, partnering closely with the data platform team to support a large-scale data platform modernization effort.

• Execute the migration of approximately 1,000 datasets to Snowflake.

• Adapt existing pipelines to Snowflake, following the core architecture and technical approach defined by the internal team.

• Ensure data quality and reliability throughout the migration process.

• Support workflow scheduling and monitoring in Apache Airflow 3.

• Execute time-sensitive migrations, including Airflow 2 to 3, Confluent region migration, and the move to Iceberg.

• Help accelerate the migration initiatives targeted for completion by the end of March 2027.