Staff Product Data Scientist, Workspace

In this role, you will dive deep, dissecting model performance from foundational layers to product specific fine-tuning. You will balance how to blend techniques like autoratings, classic human eval, and reveal preference modeling to illuminate where we need to do better, and where we are doing well. Your findings will drive cross-organizational engineering efforts, shaping the very core of our AI-powered future. You will have the opportunity to work with technology and contribute to the development of a product that has the potential to transform the way people work.The US base salary range for this full-time position is $183,000-$271,000 + bonus + equity + benefits. Our salary ranges are determined by role, level, and location. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.

Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits. Learn more about benefits at Google.

Responsibilities

• Conduct evaluations of our model performance, focusing on productivity-related tasks and workflows across modalities (text, video, images, speech).
• Develop and implement evaluation frameworks and metrics to assess the model's strengths and weaknesses.
• Analyze evaluation data and generate reports to communicate findings to leadership.
• Collaborate closely with research and engineering teams to identify areas for improvement and contribute to the development of new evaluation methodologies.
• Stay up-to-date with the latest advancements in LLM technology and evaluation techniques, and be conversant in (and understand the shortcomings of) industry benchmarks.

Minimum qualifications:

• Bachelor's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
  
• 10 years of experience using analytics to solve product or business problems, performing statistical analysis, and coding (e.g., Python, R, SQL) or 8 years work experience with a Master's degree.

Preferred qualifications:

• Master's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
  
• 12 years of experience using analytics to solve product or business problems, performing statistical analysis, and coding (e.g., Python, R, SQL).