Lead ML Systems Engineer
Twelve Labs
Software Engineering, Data Science
San Francisco, CA, USA
Posted on Wednesday, September 6, 2023
Who we are
We’re a fast-moving, diverse team pushing the frontiers of artificial intelligence. At Twelve Labs, our mission is to help developers build programs that can see, listen, and understand the world as we do by bringing the world’s most powerful video understanding infrastructure to market. As a part of achieving this mission, we are building foundation AI models that can accurately and instantly search exact moments within petabytes of video archives, generate coherent text summaries of videos, perform prompt-based video generation, and many more. The Twelve Labs platform provides access to its Large Visual Language Models (VLMs) through a suite of APIs that are trained on massive video datasets and learn to understand the meaning and context behind the visuals, conversations, and sounds within videos.
Twelve Labs recently raised $17M in seed funding, recognized as one of CB Insights’ AI 100 companies within a year of its founding, and secured a massive compute resource through partnering with Oracle. We are hyper focused on delivering the Twelve Labs platform to our customers so they can build video understanding into their products and power dream features they could have only imagined.
Part of the pathway to our rapid growth has been paved by the outstanding group of people united by the company’s mission. Beyond prominent venture capital firms such as Index Ventures and Radical Ventures, the Twelve Labs mission is backed by category building luminaries like Fei-Fei Li (Stanford HAI), Silvio Savarese (Salesforce), Oren Etzioni (AI2), Alexandr Wang (Scale), Lukas Biewald (W&B), Jack Conte (Patreon) and more.
We are committed to creating a diverse and inclusive work environment where our team members can bring their full selves to work, bring out their potential, and most importantly, thrive together. We welcome kind, brilliant, and open minded people from all walks of life to our team. If joining this mission speaks to you, we encourage you to apply!
About the Role:
As the Lead ML Systems Engineer at Twelve Labs, you will lead the ML Engineering team, driving the development of optimal machine learning systems for video foundation (VFM) and language model (VLM) in production. Your role encompasses the entire spectrum of machine learning engineering, from optimizing and scaling the inference infrastructure, which involves extensive video processing both in the cloud and on-premise, to model deployment and operations, and data infrastructure. VFMOps & VLMOps is central to our user experience as it dictates the latency and deployment speed of the trained model.
For the first 3 to 6 months, you will be hands on and actively contribute as an individual contributor in our development process. As the lead, you will set the technical strategies and goals, recruit top talent, and be responsible for your team’s success, ensuring our machine learning systems exceed user expectations in terms of speed, efficiency, and reliability. Your expertise will be key in overcoming challenges related to processing vast amounts of video data and deploying sophisticated models in production. Together with your team, you will work to enhance our VFMOps & VLMOps, contributing to a superior user experience that distinguishes Twelve Labs from its competitors. Your leadership, technical expertise, and commitment to excellence will be critical to our team's success and our users' satisfaction.
You will:
- Prioritize the team’s work in building and improving our machine learning systems in production for video foundation and language model (VFM & VLM), in collaboration with senior engineers and other stakeholders
- Inference Infrastructure: Construct the most performant, scalable, and reliable inference engine optimized for Twelve Lab’s video foundation and language models.
- ML Deployment & Operations (VFMOps / VLMOps): Lead the initiative in serving the model in the most optimized manner, deploying the pipeline, and automating the model training to deployment process.
- Data: Oversee the data infrastructure and preparation of high-quality video data for our training runs.
- Design processes (e.g. postmortem review, incident response, on-call rotations) that help the team operate effectively
- Coach and develop your reports to decide how they would like to advance in their careers and help them do so
- Run the team’s recruiting efforts through a period of rapid growth
You may be a good fit if you have:
- 10+ years of software development experience, including experience in machine learning engineering
- 5+ years of experience in building end-to-end machine learning systems encompassing infrastructure, MLOps, and data management
- You have experience working with engineers at different levels and have coached them in their career development
- 2+ years of experience managing high output engineering teams
- Proficiency in working with video processing and data pipelining
- Experience in establishing and maintaining secure software and system development environments
Desired Experience:
- MS or PhD in Computer Science, Math, or equivalent real-world experience
- Fast-paced startup engineering experience
- Experience working with large scale models
- Experience working with both cloud and on-premise environment
- ML research experience would be helpful, as this role requires interchangeable effort on both research side and software side
- Experience in handling large-scale computing system and firm understanding on scale-up and scale-out approach in cloud environment
Relevant Tech Stack:
- Language: Python, Golang, C++, CUDA
- ML / Platform: PyTorch, Docker, Kubernetes, Terraform
- ML Demo page: Gradio, Streamlit
- MLOps: MLFlow, Weights and Biases
- Data: Pachyderm, DVC
- Automation: Airflow, Kubeflow
- Model serving: Triton, FasterTransformer
Interview and Onboarding Process
Recruiter Phone Screen -> Phone Interview -> Technical Screen -> Onsite Interview -> Reference Checks
We're also excited to share that we'll do global onboarding in Seoul for all new hires (company-sponsored travel).
Even If there are a few checkboxes that aren’t ticked through your prior experience, we still encourage you to apply! If you are a 0-to-1 achiever, a ferocious learner, and a kind and fun team player who motivates others, you will find a home at Twelve Labs.
We welcome applicants from all walks of life and are committed to equal opportunity employment. We cherish and celebrate diversity not just because it is the right thing to do, but because it makes our company much stronger.
Benefits and Perks
🤝 An open and inclusive culture and work environment.
🧑💻 Work closely with a collaborative, mission-driven team on cutting-edge AI technology.
🦷 Full health, dental, and vision benefits
✈️ Extremely flexible PTO and parental leave policy. Office closed the week of Christmas and New Years.
🏙 Remote-flexible, offices in San Francisco and Seoul and coworking stipend
🛂 VISA support (such as H1B and OPT transfer for US employees)
