Senior Software Engineer - Data Platform
NextTA New York, New York, United States
IT Services and IT Consulting · 11-50 employees
About the role
Design and operate scalable data pipelines for high-volume network telemetry and real-time event processing. Develop workflow engines and data lakehouse architectures to improve network monitoring and customer experience.
What they look for
Requirements
Requires 5+ years of software engineering experience with a focus on production data pipelines and stream processing. Proficiency in Java or Python and experience with Kubernetes and SQL-based query engines are essential.
Full description
This organization is a global communications solutions provider offering a comprehensive suite of fully managed services focused on secure connectivity, networking, and mobility. It simplifies communications and network operations for businesses and government agencies. Its customers include many Fortune 500 companies, and it is recognized as a leader in the industry. With one of the broadest portfolios of technology and integrated partnerships — along with a private network — the organization delivers tailored solutions across design, deployment, and ongoing management, driving cost savings, efficiency, innovation, and the ability for clients to focus on core objectives.
The team believes that each member plays a vital role in the success and sustainability of the group. To support this, it provides an environment where professionals can grow, develop their skills, collaborate with diverse colleagues, share knowledge, and build a rewarding career.
The organization is seeking a Senior Software Engineer – Data Platform to join the Data Engineering team in NYC.
In this role, you will help build the modern data processing platform behind proactive network monitoring — a core capability of the managed network services product used across thousands of enterprise and government sites. This is product engineering, not internal tooling: real-time visibility and automated detection are central to how the organization differentiates in the managed network services market. What this platform detects, and how quickly, directly impacts customer experience.
Role and Responsibilities
- Data Pipeline & Ingestion: Design, build, and operate reliable, scalable, data-intensive pipelines carrying high-volume device and network telemetry — including ingestion and orchestration across monitoring systems, device and satellite terminal telemetry, and third-party vendor APIs.
- Real-Time Event Processing: Build and evolve stateful, real-time event processing that transforms raw telemetry into meaningful operational events at scale.
- Workflow & Rule Engine: Extend and enhance the detection and workflow engine to support client-specific flow paths and parameters through managed configuration accessible in the product interface, including authoring, validation, and versioning.
- Event & Incident Flow Visibility: Partner with product and front-end engineers to expose event and incident monitoring flow in the product experience — clarifying what was detected, what it triggered, and its current status. Own the data models, APIs, and services behind these views.
- Observability & Data Lineage: Instrument the platform so engineers can trace events end-to-end and understand system behavior — including per-job telemetry, retry semantics, and lineage back to the source.
- Data Lakehouse & Storage: Contribute to the design and evolution of the data lakehouse — table and partition design, schema evolution, storage lifecycle, and query performance for analytics and reporting.
- Technology Stack Evolution: Support ongoing upgrades to the technology stack and architecture to meet evolving product and customer needs.
- Cloud & DevOps: Build and operate services on containerized infrastructure, deploying through CI/CD and following established deployment and infrastructure patterns.
- Design & Code Quality: Translate requirements into technical design and high-quality code. Write design docs and diagrams for significant changes, participate in code reviews, and share a light on-call rotation for owned pipelines.
- AI-Augmented Engineering: Use AI-assisted engineering tools as a core part of the development workflow, with the judgment to guide and validate their output.
Desired Qualifications
- Degree in Computer Science, Engineering, or equivalent experience
- 5+ years of professional software engineering, with substantial experience building production data pipelines and workflow or rule engines
- Strong Java or Python skills — ideally both
- Hands-on production stream processing experience (Flink, Kafka Streams, Spark Structured Streaming, or Beam)
- Practical Apache Kafka knowledge beyond basic consumer usage
- Experience with workflow or ETL orchestration tools (Dagster preferred; Airflow or Prefect acceptable)
- SQL fluency and experience with columnar or lakehouse query engines (Trino, Presto, Spark SQL, Snowflake, BigQuery, ClickHouse, or StarRocks)
- Comfort operating in Kubernetes
- Ability to debug distributed systems under pressure
- Familiarity with unit testing, automated testing, and CI/CD
- Proficiency with AI-assisted development tools
- Strong communication skills across technical and non-technical audiences
Requirements
null
