Data Engineer – Ratings Engine & Billing Transformation

Data Engineer – Ratings Engine & Billing Transformation

Role Details

• Role: Data Engineer – Ratings Engine & Billing Transformation

• Primary Skills: AWS Glue, Snowflake, Python, PySpark, SQL, CI/CD

• Domain Exposure: Usage-Based Billing, Rating Engine, SAP Invoice Feeds, Financial Data Processing

• Experience / Location : 5 to 9 yrs/ Bengaluru- Hybrid

Role Overview

We are seeking an experienced Data Engineer to join a large-scale Billing Transformation programme. The engineer will be responsible for sourcing telemetry data, building scalable data pipelines, supporting rating logic, and preparing billing-ready outputs for SAP invoicing.

This role requires strong hands-on experience in AWS Glue, Snowflake, Python, PySpark, SQL, and CI/CD practices. The candidate will also be responsible for implementing data quality checks, reconciliation controls, audit trails, exception handling, and operational monitoring to ensure billing accuracy.

This is a critical engineering role supporting the transformation of raw telemetry usage data into rated, validated, and invoice-ready billing data.

Key Responsibilities

Data Ingestion & Pipeline Development

• Build and maintain data ingestion pipelines from telemetry and usage data sources.

• Develop scalable ETL/ELT pipelines using AWS Glue, PySpark, Python, and SQL.

• Support both batch and event-driven data ingestion patterns.

• Ensure data pipelines are reliable, performant, scalable, and cost-efficient.

Snowflake Data Engineering

• Create and maintain curated usage datasets in Snowflake.

• Design efficient data models, tables, views, and processing layers for downstream billing use cases.

• Optimize SQL queries, warehouse usage, partitioning strategies, and performance.

• Support analytics, validation, and reconciliation use cases on Snowflake.

Ratings Engine & Billing Transformation

• Support implementation of rating logic for usage-based billing.

• Prepare billing-ready datasets for invoice processing.

• Generate data outputs and feeds required for SAP invoicing.

• Work on pricing, usage calculation, billing rules, and financial data transformation logic.

• Ensure raw telemetry data is transformed into accurate, validated, and invoice-ready records.

Data Quality, Reconciliation & Audit Controls

• Design and implement data quality checks across ingestion, transformation, rating, and billing layers.

• Build reconciliation controls to validate data completeness, accuracy, and consistency.

• Implement audit trails for billing data movement and transformation.

• Create exception-handling and error-management processes for failed or inconsistent records.

• Support root cause analysis for billing discrepancies and data issues.

CI/CD, Monitoring & Operations

• Build and support CI/CD pipelines for data engineering deployments.

• Work with tools such as GitHub Actions, GitLab CI, Jenkins, AWS CodePipeline, or similar.

• Implement operational monitoring, alerts, dashboards, and pipeline health checks.

• Maintain technical documentation, runbooks, and support procedures.

• Collaborate with engineering, finance, billing, and operations teams to ensure smooth delivery.

Required Skills & Experience

Technical Skills

• Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.

• Experience with: • AWS Glue

• PySpark

• Python

• SQL

• Cloud-based data processing

• Strong experience working with Snowflake as a data warehouse or enterprise data platform.

• Advanced SQL skills for: • Data analysis

• Validation

• Troubleshooting

• Query optimization

• Performance tuning

• Experience with batch and event-driven ingestion patterns.

• Understanding of: • Data modelling

• Partitioning

• Performance tuning

• Cost optimization

• Scalable pipeline design

Billing & Financial Data Skills

• Good understanding of one or more of the following: • Usage-based billing

• Rating engines

• Invoice feeds

• SAP invoice integration

• Financial data processing

• Billing reconciliation

• Experience transforming usage/telemetry data into billing-ready outputs will be strongly preferred.

Data Quality & Governance

• Experience building: • Data quality checks

• Reconciliation frameworks

• Audit controls

• Exception handling processes

• Error logging and monitoring workflows

• Ability to ensure data accuracy, traceability, and reliability across the billing lifecycle.

DevOps & Engineering Practices

• Hands-on experience with CI/CD tools such as: • GitHub Actions

• GitLab CI

• Jenkins

• AWS CodePipeline

• Similar deployment tools

• Familiarity with Git-based workflows and release management.

• Ability to write clean, maintainable, and well-documented code.

• Experience working in Agile delivery environments.

Preferred Skills

• Experience working on large-scale Billing Transformation, Finance Transformation, or Order-to-Cash initiatives.

• Exposure to SAP invoicing, SAP billing integration, or downstream finance systems.

• Experience with operational dashboards and alerting tools.

• Knowledge of data governance, lineage, and metadata management.

• Experience handling high-volume telemetry or usage datasets.

Soft Skills

• Strong analytical and problem-solving skills.

• High attention to detail, especially around billing accuracy and reconciliation.

• Good communication and stakeholder collaboration skills.

• Ability to work with technical, finance, billing, and operations teams.

• Ownership mindset with the ability to deliver in transformation-driven environments.