Weekday AI

Senior SWE - Python / TypeScript

Weekday AI United States · $208K–$312K/yr

Technology, Information and Internet · 11-50 employees

6 h ago
Remote typescript Senior (5-10 yrs) Contractor United States
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Design, develop, and optimize evaluation rubrics and data pipelines to support AI and LLM projects. Collaborate with cross-functional teams to translate qualitative assessments into scalable engineering workflows.

What they look for

Python TypeScript Golang Java JavaScript Software Engineering Data Quality Machine Learning Data Workflows Evaluation Rubrics Infrastructure Optimization Scalable Engineering Technical Documentation Analytical Skills Problem Solving Communication

Requirements

Requires 5+ years of professional software engineering experience with proficiency in Python, TypeScript, or related languages. Candidates must have hands-on experience with preference data collection and building scalable software infrastructure.

Full description

This role is for one of our clients

Compensation: $100–$150 per hour

Location: North America

Employment Type: Freelance Contractor

Contract Duration: 3 months

Work Commitment: Minimum 20 hours per week, with at least 4 hours of daily availability.

Time Zone Overlap: Minimum 4 hours of overlap with Pacific Standard Time (PST).

Expected Start Date: Next week

Benefits: Fully remote engagement with the opportunity to contribute to cutting-edge AI and Large Language Model (LLM) projects. This contractor assignment does not include medical benefits or paid leave.

We are looking for a highly skilled Senior Software Engineer with strong technical expertise, exceptional attention to detail, and a keen understanding of software and data quality. The ideal candidate will have hands-on experience with preference data collection, evaluation rubric development, and building scalable engineering workflows.

You should be comfortable working in ambiguous, fast-paced environments, communicating complex technical concepts clearly, and making sound engineering decisions. We value engineers who combine strong technical execution with quality-driven judgment and prioritise delivering practical, high-impact solutions.

Key Responsibilities• Design, develop, refine, and continuously improve rubrics and evaluation criteria for preference data.

  • Review, assess, and label datasets while maintaining high quality standards and identifying subtle inconsistencies or quality gaps.
  • Build, maintain, and optimise pipelines and infrastructure supporting data generation, collection, processing, and evaluation workflows.
  • Develop reliable and scalable engineering solutions to improve data operations and evaluation processes.
  • Analyse findings from data and engineering activities and translate them into concise, structured reports, technical updates, and actionable recommendations.
  • Collaborate closely with researchers, engineers, and cross-functional teams to convert qualitative assessments into consistent, scalable processes.
  • Navigate complex codebases, identify improvement opportunities, and contribute to high-quality software development.

Required Skills and Qualifications• 5+ years of professional software engineering experience with proficiency in one or more of the following: Python, TypeScript, Golang, Java, or JavaScript.

  • Strong experience building, maintaining, and optimising software pipelines, infrastructure, or machine learning and data workflows.
  • Alternatively, extensive experience with TypeScript, Golang, or JavaScript, including contributing to or navigating large-scale codebases. Experience with projects such as Visual Studio Code is an added advantage.
  • Hands-on experience with preference data collection, preference modelling workflows, and/or evaluation rubric creation.
  • Strong understanding of data quality, evaluation methodologies, and the ability to distinguish high-quality examples from inadequate ones.
  • Excellent written and verbal communication skills, with the ability to communicate complex technical work through clear, concise, and actionable documentation.
  • Strong analytical and problem-solving abilities, with sound judgment and attention to detail.
  • Ability to work independently, adapt to ambiguity, and deliver results in fast-moving environments.

Evaluation ProcessThe selection process consists of one approximately 30-minute interview covering technical expertise, relevant experience, problem-solving approach, and cultural fit.

Similar roles