Senior QA Automation Engineer
G MASS New York, New York, United States · $260K–$390K/yr
IT Services and IT Consulting · 11-50 employees
Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.
About the role
Design and build automated test suites for compliance systems using Python, TypeScript, and AI-enabled tools. Ensure rigorous test coverage for trade surveillance, regulatory reporting, and data pipelines while maintaining full auditability.
What they look for
Requirements
Requires strong hands-on software engineering experience with a specialization in test automation and advanced proficiency in Python and SQL. Candidates must have experience applying large language models to production engineering problems and be able to work on-site in New York.
Full description
ace has partnered with a prestigious, leading global hedge fund to appoint a Senior QA Automation Engineer to its Legal & Compliance technology organization in New York.
This is a hands-on engineering role, not a test management one. You will design, build and run AI-enabled automated test capability for the systems the firm relies on to meet its regulatory obligations. You will also establish how AI-enabled compliance systems themselves are tested and evidenced. In-scope systems include trade and communications surveillance, transaction and regulatory reporting, control room and information barriers, and adjacent compliance platforms.
The role is deliberately not tied to one function. You will be deployed wherever the need is greatest: hardening a surveillance model release one quarter, assuring a reporting change against a regulatory deadline the next. You will be expected to get up to speed quickly in unfamiliar domains, work out what “correct” means from requirements and regulation, and turn that into rigorous, automated, auditable test coverage.
You will work with a high degree of autonomy in a demanding, fast-moving environment, where the expectation is delivery rather than management overhead.
Please note that this role requires presence in the client’s New York office five days per week. This is non-negotiable.
Responsibilities
Test engineering across compliance systems
- Design and build automated test suites, covering unit, integration, regression, end-to-end and data validation, for compliance applications and data pipelines across multiple functions.
- Engineer test frameworks and harnesses in Python for service, API and data-layer testing, and in TypeScript/JavaScript where front-end or UI testing is needed.
- Validate high-volume trade, order, position and transaction data at field level and record level. This includes reconciliation between source systems, enrichment layers and regulatory outputs.
- Build synthetic and scenario-based test data that exercises edge cases, such as complex instruments, lifecycle events, amendments, cancellations and late or out-of-sequence data, without exposing production data inappropriately.
- Test surveillance logic and alert generation. Confirm that rule and model changes behave as intended and that their downstream impact on alert volumes and coverage is understood before release.
AI-enabled testing and AI assurance
- Use large language models (Gemini on Google Cloud preferred) to interpret requirements, distinguish well-formed from poorly formed requirements, identify coverage gaps, and generate candidate test scripts, test data and expected results, with a human in the loop and full traceability.
- Design and orchestrate agent-based test execution, including how individual agents are scoped, coordinated across a single test scenario, and constrained by entitlements so that no agent can reach or surface data outside its permitted scope.
- Define and implement assurance approaches for LLM-enabled compliance capabilities, such as communications surveillance, alert triage, document classification and extraction. This covers evaluation datasets, accuracy and tolerance thresholds, regression checks when a model or prompt changes, and tracking of drift and consistency.
- Produce evidence of AI system behavior that will stand up to compliance oversight, internal audit and regulatory scrutiny, including versioning of models, prompts and evaluation results.
Platform engineering
- Contribute to the firm’s in-house automated testing platform and extend it, driving its adoption across the applications in scope.
- Integrate test execution with source control, pull-request workflows, requirements management and CI/CD, so that coverage is derived systematically rather than assembled manually.
- Own the infrastructure and environments for test tooling through a code-first, infrastructure-as-code approach.
Lineage, governance and assurance
- Deliver end-to-end lineage from requirement and rule change through test script, execution, results and UAT evidence, with full versioning and auditability.
- Produce test evidence that stands up to internal audit, compliance oversight and regulatory scrutiny.
- Implement entitlements and access controls within test tooling so that no test process can reach or surface data outside its permitted scope.
Ways of working
- Move between compliance functions as priorities shift, building working knowledge of each domain’s data, rules and regulatory drivers.
- Work directly with engineers, business analysts and compliance subject-matter experts to turn requirements, including ambiguous ones, into precise acceptance criteria.
- Raise automation standards across the teams you work with through code review, pairing and reusable components.
Essential
- Strong, current, hands-on software engineering experience with a specialism in test automation, at the level of a software engineer who specializes in quality rather than a tester who scripts.
- Advanced Python, covering test frameworks (e.g. pytest), service and API testing, and data manipulation at scale.
- Strong SQL and experience validating large, complex datasets, including reconciliation between systems.
- Proficiency in TypeScript/JavaScript and modern UI test tooling (e.g. Playwright, Cypress).
- Hands-on experience applying large language models to real engineering problems in production, rather than proofs of concept, with a clear view on controls, traceability and data access.
- CI/CD and infrastructure as code. GCP and Terraform are strongly preferred; containerization (Docker, Kubernetes) is an advantage.
- Disciplined engineering practice: Git, branching strategies, pull-request workflows, code review and repeatable release processes.
- Experience testing systems in a regulated financial services environment, ideally capital markets.
- A track record of taking automation into genuine production adoption and owning it thereafter.
- The ability to learn a new domain quickly and work independently at pace, making day-to-day implementation decisions without direction.
Advantageous
- Experience building with Gemini on Google Cloud (Gemini Enterprise Agent Platform, formerly Vertex AI), or comparable enterprise LLM platforms.
- Experience evaluating or testing ML/LLM-based systems, for example building evaluation sets, measuring precision and recall, testing for robustness, and handling outputs that vary between runs.
- Exposure to one or more compliance technology domains: trade or communications surveillance; transaction and regulatory reporting (e.g. CAT, CFTC Part 43/45, SEC Regulation SBSR, EMIR, MiFIR); control room and information barriers; books and records.
- Understanding of trade and order lifecycle data across asset classes.
- Familiarity with model risk management expectations (e.g. SR 11-7) as they apply to AI in regulated environments.
- Experience with event-driven or streaming architectures (e.g. Kafka).
Contract Length: 6 months, 3 months rolling
Day rate: $1,000-1,500/d, depending on experience
Similar roles
-
QA Supervisor (QC Documentation & Compliance)
United Concrete Wallingford, Connecticut, United States
-
QA Server Engineer
SuperPlay Warsaw, Masovian Voivodeship, Poland
-
Iterable Lifecycle QA Specialist- LATAM (Argentina/Colombia)
Algomarketing® Argentina
-
Principal QA Test Automation with TS + Playwright, Remote
Poland and Central-Eastern Europe Bulgaria
-
QA Manual
Poland and Central-Eastern Europe Bulgaria
-
QA Lead- Montoursville
Unique Industries, Inc. Montoursville, Pennsylvania, United States