Data Engineer – Azure Databricks and Cloud Data Solutions (Safety)

Job Title: Data Engineer – Azure Databricks and Cloud Data Solutions (Safety)

Why this role

Southern Company’s Operations Data Analytics team supports the data foundations and integration capabilities needed to deliver reliable analytics and reporting across multiple organizations. This Data Engineer role will focus on Safety data and be responsible for creating data pipelines, curating source schemas, and supporting data product development in the Azure Databricks lakehouse platform.

The role is important in creating secure, reusable, and high-performing data services that align with business needs, Technology Organization strategy, and enterprise standards. You will work directly with business and technology stakeholders to turn data requirements into practical solutions while improving performance, resource utilization, capacity planning, and application and data security. This role will support multiple business units and work in multiple lakehouse environments with data provisioned through Unity catalog, to create a unified foundation of safety and human performance data.

About the role

As part of this team, you will design, develop, implement, integrate, and support cloud and data solutions, with hands-on responsibility for data pipelines, data models, ETL processes, lakehouse capabilities, and data storage layers.

You will partner with stakeholders, developers, and infrastructure teams to align solutions across technology platforms. The role also serves as a subject matter expert for data architecture and Databricks, supports operational issue resolution, and helps ensure that project documentation and change management requirements are met.

What you will do

• Design and develop robust data pipelines to ingest, store, and process data from multiple sources for analytics and reporting.

• Create and maintain logical and physical data models that organize data for efficient querying, analysis, and visualization.

• Build and maintain Extract, Transform, Load (ETL) processes that move data between systems while supporting data quality and consistency.

• Implement and support lakehouse solutions using medallion architecture, including data engineering pipelines and data storage layers.

• Optimize data lake storage, retrieval, performance, and resource utilization for analytical workloads.

• Develop dynamic, metadata-driven frameworks and scalable data ingestion patterns.

• Support streaming datasets and Databricks Delta Live Tables.

• Apply lakehouse performance-tuning techniques such as partitioning, aggregations, and materialized views.

• Architect, develop, implement, integrate, and support cloud technology solutions that meet business partner expectations for quality, cost, and timeliness.

• Work closely with developers and infrastructure teams to optimize application code, create reusable services, plan capacity, and protect application and data security.

• Become the team’s subject matter expert for data architecture and Databricks.

• Communicate effectively and promptly with customers, team members, and management, and build strong relationships with peers and business partners.

• Develop required project documentation, follow Change Management policies and procedures, and respond quickly and effectively to operational problems.

• Build a strong understanding of the company’s business challenges and goals and exhibit Our Values.

What success looks like

• Reliable, efficient data pipelines that support analytics and reporting needs.

• Reusable and well-aligned solutions across on-premises and Azure technology platforms.

• Consistent, high-quality data movement through maintainable ETL and lakehouse processes.

• Optimized application code, cloud resources, performance, and capacity utilization.

• Secure data and application solutions supported by complete project documentation and disciplined change management.

• Strong working relationships and timely communication with business partners, technology teams, and management.

• Fast, effective resolution of operational problems and growing team expertise in data architecture and Databricks.

What you bring (required)

Education: Bachelor’s degree required, preferably in Computer Science, Management Information Systems, Engineering, or another technology-related field.

Databricks: Experience with Databricks, including notebooks, clusters, and jobs.

Cloud platforms: Experience with a major cloud platform, preferably Microsoft Azure, and services including Cosmos DB, Storage Application Services, and Azure Data Factory.

Architecture: Experience architecting data-oriented solutions using cloud data architecture platforms and patterns, preferably Azure.

Lakehouse engineering: Experience implementing lakehouses with medallion architecture, data engineering pipelines, and data storage layers.

Data ingestion: Experience with dynamic metadata-driven frameworks and large-scale data ingestion patterns.

Data modeling: Experience developing complex logical and physical data models.

Performance and cost: Experience optimizing cloud PaaS resources to follow best practices and reduce cost, plus lakehouse performance tuning using partitioning, aggregations, materialized views, or similar techniques.

Delivery practices: Experience with Agile, DevOps, and CI/CD methodologies.

Data technologies: Familiarity with Apache Spark, Delta Lake, and MLflow, plus knowledge of structured and unstructured data across databases, data warehouses, and data lakes.

Software development: Knowledge of software development technologies, methodologies, and associated tools.

Problem solving: Excellent root cause analysis and problem-solving skills.

What sets you apart (preferred)

• Hands-on experience with Microsoft Azure data services and cloud PaaS architecture and resources.

• Familiarity with streaming datasets and Databricks Delta Live Tables.

• Strong technical writing skills.

What we value

Customer focus: You deliver cloud and data solutions that meet or exceed business partner expectations for quality, cost, and timeliness.

Ownership and responsiveness: You take responsibility for documentation, change management, and rapid resolution of operational problems.

Collaboration: You communicate effectively and build productive relationships with customers, peers, management, developers, infrastructure teams, and business partners.

Continuous learning: You become a subject matter expert, stay close to the business, and seek to understand the company’s challenges and goals.

Enterprise mindset: You design secure, reusable, aligned solutions and exhibit Our Values.

About Southern Company

Southern Company (NYSE: SO) is a leading energy provider serving 9 million customers across the Southeast and beyond through its family of companies. Providing clean, safe, reliable and affordable energy with excellent service is our mission. The company has electric operating companies in three states, natural gas distribution companies in four states, a competitive generation company, a leading distributed energy solutions provider with national capabilities, a fiber optics network and telecommunications services. Through an industry-leading commitment to innovation, resilience and sustainability, we are taking action to meet customers' and communities' needs while advancing our goal of net-zero greenhouse gas emissions by 2050. Our uncompromising values ensure we put the needs of those we serve at the center of everything we do and are the key to our sustained success. We are transforming energy into economic, environmental and social progress for tomorrow. Our corporate culture has been recognized by a variety of organizations, earning the company awards and recognitions that reflect Our Values and dedication to service. To learn more, visit www.southerncompany.com.

Southern Company invests in the well-being of its employees and their families through a comprehensive total rewards strategy that includes competitive base salary, annual incentive awards for eligible employees and health, welfare and retirement benefits designed to support physical, financial, and emotional/social well-being. This position may also be eligible for additional compensation, such as an incentive program, with the amount of any bonus/awards subject to the terms and conditions of the applicable incentive plan(s). A summary of the benefits offered for this position can be found here https://seo.nlx.org/southernco/pdf/SOCO-Benefits.pdf. Additional and specific details about total compensation and benefits will also be provided during the hiring process.

Southern Company is an equal opportunity employer where an applicant's qualifications are considered without regard to race, color, religion, sex, national origin, age, disability, veteran status, genetic information, sexual orientation, gender identity or expression, or any other basis prohibited by law.

Southern Company Services