Systems & Data Infrastructure Engineer
Astrome Technologies · Bengaluru, Karnataka, India
Defense and Space Manufacturing · 51-200 employees
6 h ago
Senior (5-10 yrs)
Full-time
India
Log in
to apply, save this posting, or score it against your profile with AI.
About the role
Design and maintain robust data infrastructure, including crash-safe binary formats and relational database schemas. Manage HPC workload lifecycles and orchestrate real-time backend services with integrated AI/analytics capabilities.
What they look for
Python
Async/await
FastAPI
Relational DB modeling
SQL
WebSockets
Server-Sent Events
RBAC
JWT
OAuth
Subprocess management
HPC
Data engineering
PostgreSQL
Redis
Kafka
Requirements
Requires strong proficiency in Python, asynchronous programming, and relational database modeling. Candidates should have experience with HPC job management, API security, and streaming data architectures.
Benefits
Career growth
ESOPs
Gratuity
PF
Health Insurance
Full description
Functional Focus: Data Engineering, Real-Time Streaming, and HPC Job Orchestration
- Scientific Data Architecture: Design crash-safe, binary data formats capable of handling high-speed incremental writes from compute jobs and concurrent reads for query-time calculations.
- Operational Data Modeling: Design and maintain a relational database schema via an async ORM, backend-agnostic across database engines.
- HPC Workload Management: Manage the lifecycle of batch and interactivecompute jobs, handling subprocess monitoring, status polling, and cluster filesystem coherency.
- Real-Time API & Streaming: Build async backend services and long-lived, server-pushed event channels with backpressure handling, and enforce role-based access control on all APIs.
- AI Service Integration: Integrate and orchestrate calls to an AI/analytics service from the backend, coordinating with application state.
Mandatory:
- Python 3, async/await
- FastAPI (or similar async Python web framework)
- Relational DB modeling, async ORM (e.g. Tortoise, SQLAlchemy), SQL
- Server-Sent Events or WebSockets, backpressure handling
- Role-based access control (RBAC), token-based auth (JWT/OAuth)
- Subprocess management, batch job lifecycle/status polling
- Binary/streaming file format design, crash-safe writes
- REST/SDK integration with an external AI/LLM service
Optional:
- Go or another async-capable backend language
- GraphQL
- PostgreSQL administration, Alembic/migrations tooling
- Message queues/brokers (Redis, RabbitMQ, Kafka)
- Air-gapped/offline deployment experience
- HPC schedulers (SLURM, PBS), Linux cluster filesystems
- NumPy/columnar formats (Parquet, HDF5)
- Tool-calling / function-calling orchestration patterns
We offer great career growth, ESOPs, Gratuity, PF and Health Insurance.
