Big Data Engineer
Select Minds LLC · Lansing, Michigan, United States · $110K–$125K/yr
IT Services and IT Consulting · 51-200 employees
About the role
Design, develop, and optimize large-scale data processing pipelines and real-time streaming solutions. Collaborate with cross-functional teams to ensure data quality, governance, and system health across the Big Data stack.
What they look for
Requirements
Requires 8+ years of experience in Big Data technologies with hands-on expertise in Spark, Scala, Hive, and the Hadoop ecosystem. Candidates must have experience in client-facing roles and a proven track record of delivering end-to-end Big Data implementations.
Benefits
Full description
Benefits:
- ONSITE
- Competitive salary
- Opportunity for advancement
Big Data Engineer Location: Chicago, IL (Day 1 Onsite) Job Type: Contract / Full Time Experience: 8+ yrs Duration: Long Term
We are seeking an experienced Big Data Engineer to design, develop, and optimize enterprise-scale big data solutions. The ideal candidate will have strong hands-on expertise in the Hadoop ecosystem, Spark, Scala, Hive, Apache NiFi, and Kafka, along with experience building scalable data pipelines and supporting modern CI/CD practices. Candidates should have worked directly with clients, participated in requirement gathering sessions, and delivered end-to-end Big Data implementations.
Key Responsibilities * Design, develop, and optimize large-scale data processing pipelines using Spark, Scala, and Hive. * Build and maintain data ingestion workflows using Apache NiFi. * Develop and support real-time data streaming solutions using Apache Kafka. * Monitor application logs and system health using Kibana, Elasticsearch, and Logstash. * Implement and support CI/CD pipelines using Git, Jenkins, and Nexus. * Ensure data quality, integrity, and governance across Big Data platforms. * Participate in architecture discussions and recommend scalable design solutions. * Troubleshoot and resolve complex production issues across the Big Data stack. * Collaborate with cross-functional teams in an Agile environment. * Mentor junior engineers and contribute to technical best practices.
Required Qualifications * 8+ years of experience in Big Data technologies. * Strong hands-on experience with Spark, Scala, Hive (HQL), and the Hadoop ecosystem. * Experience implementing Apache NiFi and Apache Kafka. * Proficiency in Shell Scripting or Python. * Experience with PostgreSQL. * Familiarity with scheduling tools such as Oozie and Control-M. * Working knowledge of Git, Jenkins, Nexus, Maven, Ansible, and SonarQube. * Strong understanding of CI/CD practices. * Experience in client-facing roles, including requirement gathering workshops. * Minimum of two end-to-end Big Data implementation projects. * Ability to work independently and contribute effectively within Agile teams.
