Amaze - Cloud Data Architect
HEXAWARE India
IT Services and IT Consulting · 10,001+ employees
Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.
About the role
Design and implement scalable batch and real-time ETL pipelines using PySpark, Structured Streaming, and Delta Lake. Develop autonomous multi-agent workflows and end-to-end RAG pipelines while managing data governance and security.
What they look for
Requirements
Requires 7 to 10 years of experience in data engineering with at least 3 years specifically in Databricks. Candidates must have strong expertise in Pyspark, data ingestion frameworks, and security protocols like ABAC and RBAC.
Full description
Senior Data engineer Key Responsibilities • 7 to 10 years of experience in data engineering and minimum of 3+ years of experience working in Databricks • 4+ years Working in leveraging Pyspark for building reusable Data ingestion and Data quality frameworks • Lakehouse & Data Engineering: Design scalable batch and real-time ETL pipelines using PySpark, Structured Streaming, and Delta Lake built on a Medallion Architecture (Bronze, Silver, Gold). • ABAC and RBAC Security - Experience in setting up ABAC and RBAC security for Agentic AI Journey use cases • Agentic AI & Orchestration: Build autonomous, multi-agent workflows and reasoning loops using MCP Protocol and Serverless compute . • RAG & Vector Integration: Develop end-to-end Retrieval-Augmented Generation (RAG) pipelines and integrate vector databases for semantic retrieval. • Governance & Operations: Manage data lineage, access controls, and security using Unity Catalog alongside CI/CD workflows and LLM evaluation frameworks
