AI Evaluation Infrastructure Engineer

About this job

The Role We build AI products, and the quality of our evaluations sets the ceiling for how good those products can be. The speed of our evaluations determines how quickly we can improve them. We are looking for an engineer to build the infrastructure and tooling that make high-quality AI evaluation possible at Block's scale. You will help teams understand whether a model or product change is actually better, whether a result is statistically meaningful, and whether offline evaluation is predicting what happens with real users. Our evaluation approach combines offline evals that encode our definition of a good response, online evals that show how people actually respond, and a feedback loop that keeps the two converging.…

This is a short summary.

Want to know if you're a fit? Check your CV against this role — free, in 30 seconds.

Most-requested skills in the United States

Based on 428 United States vacancies that list requirements, these are the skills employers ask for most often.

• Project Management36% of postings
• CAD22% of postings
• AutoCAD20% of postings
• Lean17% of postings
• MATLAB14% of postings
• Six Sigma12% of postings
• SolidWorks10% of postings
• Revit10% of postings

Similar jobs

• Field Service Engineer III - Mass Spectrometry (On-Site) · Thermo Fisher Scientific
• Senior Application Security Engineer (Java), VP · Citi
• Head of Product Engineering, HR Total Rewards - C15 - Director - C15 - NEW YORK · Citi
• Mechanical Maintenance Engineer (Experienced Professional) · Michelin
• Quality Engineer Manager · Airbus
• Senior Platform Software Engineer – AI/ML & GPU Infrastructure · Cisco