Big Data Engineer (Cloudera Admin)

• Cloudera Infrastructure Design: Design and implement Cloudera infrastructure for data storage, processing, and analytics to meet the platform’s scalability, performance, and security requirements.

• System Integrations & Automation: Manage system integrations, automation, and CI/CD pipelines to streamline operations and improve platform efficiency.

• Scalability & Availability: Ensure system scalability, availability, and performance by proactively monitoring the infrastructure and addressing potential issues.

• Resource Utilization Optimization: Monitor and optimize resource utilization across the platform to ensure efficient operation and reduce costs.

• Collaboration: Work with cross-functional teams to ensure the platform meets the data and business needs of the organization.

• Cloudera Administration: Install, configure, and manage Cloudera Hadoop clusters (CDH/CDP).

• Monitor cluster health using Cloudera Manager.

• Manage Hadoop ecosystem components: HDFS, YARN, Hive, Impala, HBase, Spark, Kafka.

• Perform cluster upgrades, patching, and capacity planning

• Handle performance tuning and troubleshooting.

• Implement and maintain data security (Kerberos, Ranger, Sentry).

• Manage backup, disaster recovery, and data replication.

Requirements

• Education: Bachelor’s degree in computer science, Engineering, or a related field.

• Experience (3-5 years) : Proven experience with Cloudera infrastructure and containerization technologies such as Kubernetes.

• Technical Skills: Strong expertise in streaming, KAFKA), and system monitoring tools such as Dynatrace and Grafana.

• Expertise in designing, implementing, and maintaining scalable infrastructure for data platforms.

• Proficient with Cloudera and CI/CD automation.

• Strong experience in system monitoring, performance optimization, and ensuring high availability.

• Ability to work with cross-functional teams to meet platform needs.