Senior Experimentation Analyst

Preply

Preply

IT

Barcelona, Spain

Posted 6+ months ago

Preply is a global language learning marketplace, connecting 50,000+ tutors with millions of learners from all over the world. Since the company launched in 2012, students have signed up to more than 15 million lessons.

Meet the team!

We A/B test most of our changes on production, both product and technical, before scaling it to all customers. The Experimentation Squad owns the whole infrastructure and the framework that are used to make those business-vital decisions.

We define the framework rules, including the overall evaluation criterion, set of target metrics, available test types: student-sided, tutor-sided, clustered; frequentist, bayesian, interleaving; superiority, non-inferiority. We analyze performance of the experimentation program using multiple holdout groups and push product management in the directions, which will improve its results. Finally we’re in charge of improving the company's experimentation culture and education, and keeping institutional memory.

We have diverse technical challenges (sometimes we write about them on our Engineering Blog) that will allow you to develop your skills across the stack.

Visit our Tech Radar to learn more about the technologies we use at Preply!

What you’ll be doing:

  • Unlock the potential of product teams by researching and implementing new approaches for hypothesis testing (clustering, interleaving..);
  • Identifying and designing value-adding insights solutions, leveraging data to optimize our strategy;
  • Establish credibility and quality of the A/B testing system;
  • Quantify the impact of product development and distinct initiatives in order to identify growth opportunities and help top management to prioritize Preply’s roadmap;
  • Level up the A/B testing culture across the company;
  • Support engineers to build and maintain high-quality data pipelines;
  • Collaborate with product teams to help them tackle technical issues with A/B testing.

What you need to succeed:

  • Extensive experience in the product data science field (5+ years’ experience at least);
  • Experience in A/B testing and desire to dive deeper into the sphere and research best industry practices to build state-of-the-art experimentation in Preply;
  • Strong curiosity, problem solving and problem finding skills;
  • Solid SQL hands-on experience, strong knowledge of Python;
  • Good understanding of statistical analysis concepts such as: p-value, null hypothesis, confidence interval;
  • Eagerness to take responsibility for systems that directly impact the company's decision-making process;
  • Passion for analyzing data to inform decisions, measuring and monitoring;
  • Willingness to educate colleagues (doing knowledge-sharing sessions);
  • Excellent written and verbal communication skills. Fluent English is a must.

Nice to have:

  • Experience applying advanced experimentation - bayesian testing, causal impact analysis;
  • Knowledge of Airflow, Redshift (or any other Data Warehouse DBs - Snowflake, BigQuery..);
  • Knowledge of Looker, Amplitude, Databricks or other BI tools.

Why you’ll love it at Preply:

  • An open, collaborative, dynamic and diverse culture;
  • A generous monthly allowance for lessons on Preply.com, Learning & Development budget and time off for your self-development;
  • A competitive financial package with equity, leave allowance and health insurance;
  • Access to free mental health support platforms;
  • A flexible set-up with the option of paid relocation to Barcelona or working remotely within CET or 3 hours from CET;
  • The opportunity to shape the lives of learners and tutors through language learning and teaching in 175 countries (and counting!).

About Preply:

At Preply, we focus on doing better every day – pushing the limits of what we know is possible. And we know our purpose is strong. We’re re-engineering education, providing outstanding experiences for our global community of language learners and building authentic connections across the world.

Propelled by a tenfold increase in revenues over the last three years, thanks to our best-in-class consumer and b2b products, Preply is now leading the online language tutoring segment globally and has 400+ employees of over 50 nationalities based across Barcelona, Kyiv and the US. We do extraordinary things at Preply – and we want you to join us. If you’re committed to curiosity, love experimentation and relish the challenge of exploring new ideas, then it’s time we said:

Welcome to The World Class.

Our values:

  • Human: We embrace diversity by fostering inclusive environments where people feel respected and safe to express their opinions.
  • Hungry: We make bold, data-informed decisions to deliver high-impact solutions for our customers.
  • Humble: We experiment continuously by staying open to being challenged, learning from failures and making data-informed decisions.
  • Hackers: We learn by doing, focus on things that matter and move with incredible speed to create simple, creative, and scalable solutions.
  • Curious: We thrive on facing challenges, diving deep to identify root causes and uncovering insights to continuously improve.
  • Learners Obsessed: We put our learners at the core of everything we do to improve the effectiveness of their language learning with us.

Diversity, Equity, and Inclusion

Preply is committed to creating a diverse and inclusive environment where people from all backgrounds can thrive. Different opinions and viewpoints are key ingredients in our success as a multicultural Ed-Tech company.

That means that Preply will consider all applications for employment without regard to race, color, religion, gender identity or expression, sexual orientation, national origin, disability, age or veteran status. Together, we are The World Class.

#LI-VS1