Developing Feast, the Leading Open Source Feature Store, with Willem Pienaar (Gojek, Tecton)

Machine Learning Engineered

Mar 9 2021 • 1 hr 11 mins

Willem Pienaar is the co-creator of Feast, the leading open source feature store, which he leads the development of as a tech lead at Tecton. Previously, he led the ML platform team at Gojek, a super-app in Southeast Asia. Learn more: https://twitter.com/willpienaar (https://twitter.com/willpienaar) https://feast.dev/ (https://feast.dev/) Every Thursday I send out the most useful things I’ve learned, curated specifically for the busy machine learning engineer. Sign up here: https://www.cyou.ai/newsletter (https://www.cyou.ai/newsletter) Follow Charlie on Twitter: https://twitter.com/CharlieYouAI (https://twitter.com/CharlieYouAI) Subscribe to ML Engineered: https://mlengineered.com/listen (https://mlengineered.com/listen) Comments? Questions? Submit them here: http://bit.ly/mle-survey (http://bit.ly/mle-survey) Take the Giving What We Can Pledge: https://www.givingwhatwecan.org/ (https://www.givingwhatwecan.org/) Timestamps: 02:15 How Willem got started in computer science 03:40 Paying for college by starting an ISP 05:25 Willem's experience creating Gojek's ML platform 21:45 Issues faced that led to the creation of Feast 26:45 Lessons learned building Feast 33:45 Integrating Feast with data quality monitoring tools 40:10 What it looks like for a team to adopt Feast 44:20 Feast's current integrations and future roadmap 46:05 How a data scientist would use Feast when creating a model 49:40 How the feature store pattern handles DAGs of models 52:00 Priorities for a startup's data infrastructure 55:00 Integrating with Amundsen, Lyft's data catalog 57:15 The evolution of data and MLOps tool standards for interoperability 01:01:35 Other tools in the modern data stack 01:04:30 The interplay between open and closed source offerings Links: https://github.com/feast-dev/feast (Feast's Github) https://blog.gojekengineering.com/data-science/home (Gojek Data Science Blog) https://www.getdbt.com/ (Data Build Tool (DBT)) https://www.tensorflow.org/tfx/data_validation/get_started (Tensorflow Data Validation (TFDV)) https://feast.dev/post/a-state-of-feast/ (A State of Feast) https://cloud.google.com/bigquery (Google BigQuery) https://www.amundsen.io/ (Lyft Amundsen) https://www.cortex.dev/ (Cortex) https://www.kubeflow.org/ (Kubeflow) https://mlflow.org/ (MLFlow)