Internship - Machine Learning Research - Speech-to-Speech Translation

The Machine Translation and Multilingual AI team is looking for research interns to explore open problems in speech-to-speech translation: how to build models that let people talk naturally with anyone, in any language, in real time.

Description

Speech-to-speech translation is moving from cascaded pipelines to end-to-end models that listen, translate and speak in a single system. Many core questions remain open. How do you translate simultaneously, before the speaker has finished? How do you keep translated speech natural and expressive? How do large multimodal models best combine speech understanding, translation and speech generation? And how should these systems be evaluated?

As an intern, you will own a focused research project on one of these questions, working closely with researchers and engineers on the team. You will have access to large-scale data and compute, and we encourage publishing results at top venues.

Example research directions include:

- Streaming and simultaneous speech translation: policies for when to listen and when to speak, and the trade-off between quality and latency. - Speech generation for translation: natural, expressive and prosodically appropriate output, including speech tokenizers and audio codecs. - Multimodal foundation models for speech: how pre-training, fine-tuning and reinforcement learning shape speech-to-speech ability across languages. - Multilingual scaling: extending to new languages and handling code-switching and unknown source languages. - Evaluation: automatic metrics and human evaluation methods that reflect real conversational experience.

Minimum Qualifications

Currently enrolled in a PhD or Master's program in Computer Science, Electrical Engineering, Linguistics or a related field. Research experience in at least one of: speech recognition, speech synthesis, machine translation, NLP or multimodal learning. Hands-on experience with deep learning toolkits such as PyTorch, JAX or TensorFlow. Strong programming skills in Python.

Preferred Qualifications

Research experience in speech-to-speech or speech-to-text translation, especially end-to-end, streaming or simultaneous approaches. Research experience in text-to-speech or speech generation, such as neural TTS, speech tokenizers, audio codecs, or expressive and multilingual synthesis. Experience training or fine-tuning large language models or multimodal (audio + text) models. Familiarity with speech and translation evaluation, such as WER, BLEU/chrF/COMET-style metrics, MOS and human preference studies, and latency metrics. Publications at relevant venues (e.g. Interspeech, ICASSP, ACL, EMNLP, NeurIPS, ICML, ICLR). Excellent spoken and written communication skills.