Data Engineer (AWS+Pyspark)
Zensar · Hyderabad, Telangana, India
IT Services and IT Consulting · 10,001+ employees
2 d ago
Mid (2-5 yrs)
Full-time
India
Log in
to apply, save this posting, or score it against your profile with AI.
About the role
Develop and optimize PySpark applications using Spark Dataframes to process large volumes of data. Utilize AWS analytics and compute services to build scalable data pipelines.
What they look for
Python
PySpark
AWS EMR
Amazon Athena
AWS Glue
Amazon Lambda
Amazon EC2
Amazon S3
Amazon SNS
Git
Data Warehousing
Parquet
Snappy
Gzip
AWS Databases
Requirements
Requires 4-5 years of experience in Big Data technologies with mandatory proficiency in Python and PySpark. Experience with AWS services and version control tools like Git is essential.
Full description
Data Engineer (AWS + pySpark)
- Having 4-5 yrs years of relevant experience, which includes hands on experience in Big Data technologies.
- Mandatory - Hands on experience in Python and PySpark.
- Build pySpark applications using Spark Dataframes in Python.
- Worked on optimizing spark jobs that processes huge volumes of data.
- Hands on experience in version control tools like Git.
- Worked on Amazon’s Analytics services like Amazon EMR, Amazon Athena, AWS Glue.
- Worked on Amazon’s Compute services like Amazon Lambda, Amazon EC2 and Amazon’s Storage service like S3 and few other services like SNS.
- Good to have knowledge of datawarehousing concepts – dimensions, facts, schemas- snowflake, star etc.
- Have worked with columnar storage formats - Parquet etc. Well versed with compression techniques – Snappy, Gzip.
- Good to have knowledge of AWS databases (atleast one) Aurora, RDS, Redshift, ElastiCache, DynamoDB.am
