Responsibilities: • Measurement: Define, invent, and deliver metrics which accurately measure the quality of online information, and the satisfaction/success of our customers. • Models: Develop ML/Statistical models to measure/predict the quality of online content and/or user interactions with large scale AI systems. • Experimental Design: Think critically about sampling and experimental design. Developing innovative strategies and products in these areas. • Crowdsourcing: Apply behavioral insights, game theory, and social scientific understanding to measure and improve the performance of a global workforce employed to label online content. • Strategy: Translate strategy into plans that are clear and measurable, with progress shared out monthly to stakeholders. • Cooperation: Partner effectively with program management, engineers, and other areas of the business. Qualifications: • MS or PhD in data science or related field (statistics, computer science, economics, etc). • Experience with human-in-the-loop machine learning. • Experience with real world system building and data collection, including design, coding and evaluation. • Customer focused, strategic, drives for results, is self-motivated, and has a propensity for action. • Fantastic problem solver: ability to solve problems that the world has not solved before. • Industry data science experience.