Apertus Engineer: Post-training

Join a Leading AI Initiative

We are on the lookout for a highly skilled engineer to become an integral part of the Apertus post-training team. In this dynamic role, the successful candidate will focus on developing, executing, and evaluating sophisticated pipelines designed to transform Apertus base models into highly capable assistants.

The ideal individual will possess a robust background in LLM post-training, exceptional software engineering skills, and a collaborative spirit that thrives in a research-driven HPC setting.

About Apertus

Apertus trains open foundation models with vast parameter scales on some of Europe’s most advanced AI-ready supercomputers. Collaborating with over thirty academic partners, our team of more than a dozen dedicated engineers works alongside top-tier researchers from EPFL and ETH Zürich, releasing innovative models such as Apertus 1 and Apertus 1.5.

Role Responsibilities

• Infrastructure and Systems Engineering:
  • Build and maintain containerized environments for LLM post-training and reinforcement learning workloads.
  • Adapt containers and dependencies to function on the Alps / CSCS infrastructure.
  • Run and monitor Slurm-based training and evaluation jobs.
  • Debug distributed execution failures, performance issues, and networking challenges.
  • Ensure reproducible training recipes, configuration files, and documentation are maintained.
  • Collaborate with researchers and infrastructure engineers to enhance large-scale experiment performance.
• LLM Post-training and Reinforcement Learning:
  • Support SFT, preference optimization, and reinforcement learning workflows.
  • Develop RL environments for various tasks, including mathematics and coding.
  • Implement and run reward modeling and calibration processes.
  • Evaluate and debug post-training processes, addressing common challenges.
  • Conduct ablation studies and assess model behavior across various benchmarks.

Essential Qualifications

• MSc or PhD in a relevant field such as Computer Science, Data Science, or Machine Learning. Exceptional BSc candidates with substantial engineering experience will also be considered.
• Demonstrated experience in AI and neural network architectures.
• Strong communication and collaboration skills for effective teamwork.
• Flexibility to adapt to shifting priorities and workflows in a fast-paced environment.
• Hands-on experience with LLM post-training processes.

Preferred Background

• Familiarity with distributed training concepts and HPC workload managers like Slurm.
• Experience adapting containers for HPC or GPU clusters.
• Publication in relevant research domains or familiarity with recent advancements.

What We Offer

• A stimulating academic setting at a premier technical university.
• Access to cutting-edge supercomputing infrastructure and AI research.
• Collaboration opportunities with esteemed researchers and engineers.
• Flexible working arrangements, including remote work options.
• Professional development opportunities, including conferences and specialized training.
• A chance to contribute to open-source projects with global implications.

The role is available in either Lausanne at EPFL or in Zürich at ETH Zürich, positioning you in the heart of Switzerland's AI development landscape.

Explore an exciting opportunity to shape technology with significant national impact while expanding your professional horizons.