Job Description

Job Title: Data Scientist, Foundation AI - PhD Early Career

Company: Roblox

Years of Experience: 0–3 years / Research-oriented (PhD preferred or pursuing)

Location: San Mateo

Role Type: Full-time

Salary: $185,860 – $221,380 USD (plus equity & benefits)

Eligibility

• Pursuing or completed a PhD (or equivalent) in:
• Statistics
• Economics
• Computer Science
• Applied Mathematics
• Physics
• Engineering or related quantitative disciplines

Role Overview

We are hiring an Applied Data Scientist – GenAI Evaluation to build and scale rigorous evaluation systems for next-generation AI products. This role focuses on measuring the quality, safety, and business impact of Generative AI features across text, image, video, and multimodal experiences.

You will work at the intersection of GenAI research, experimentation, causal inference, and product analytics, helping shape the future of AI evaluation frameworks and user experience.

Key Responsibilities

• Develop evaluation frameworks for GenAI systems across text, image, video, 3D, and multimodal outputs.
• Design experiments including dataset creation, reliability analysis, and evaluation pipelines.
• Build and fine-tune LLM-as-a-judge systems for automated quality assessment.
• Conduct A/B testing and causal inference experiments to measure feature impact.
• Define leading and lagging success metrics for product performance, user satisfaction, and safety.
• Build reproducible and automated evaluation tooling for company-wide use.
• Conduct frontier applied research in Generative AI + Data Science.
• Contribute new methodologies and best practices to the technical community.

Skills and Qualifications

• Strong experience with SQL (Hive/Spark) for large-scale data analysis.
• Strong coding skills in Python or R for statistical modeling and analytics.
• Expertise in experimentation design, causal inference, and statistical analysis.
• Familiarity with Generative AI models and evaluation systems.
• Knowledge of:
• Fine-tuning
• RLHF
• Synthetic data generation
• Experience in applied research, publications, or technical innovation.
• Ability to solve open-ended business problems with strong analytical frameworks.
• Comfortable using AI tools to accelerate workflows and experimentation.
• Strong collaboration and communication across technical and business teams.