Moin Nadeem (MIT): The extraordinary future of natural language models

Machine Learning Engineered

Nov 3 2020 • 1 hr 24 mins

Moin Nadeem is a masters student at MIT, where he studies natural language generation. His research interests broadly include natural language processing, information retrieval, and software systems for machine learning. Learn more about Moin: https://moinnadeem.com/ (https://moinnadeem.com/) https://twitter.com/moinnadeem (https://twitter.com/moinnadeem) Want to level-up your skills in machine learning and software engineering? Join the ML Engineered Newsletter: http://bit.ly/mle-newsletter (http://bit.ly/mle-newsletter) Comments? Questions? Submit them here: http://bit.ly/mle-survey (http://bit.ly/mle-survey) Follow Charlie on Twitter: https://twitter.com/CharlieYouAI (https://twitter.com/CharlieYouAI) Take the Giving What We Can Pledge: https://www.givingwhatwecan.org/ (https://www.givingwhatwecan.org/) Subscribe to ML Engineered: https://mlengineered.com/listen (https://mlengineered.com/listen) Timestamps: 01:35 Follow Charlie on Twitter (https://twitter.com/CharlieYouAI (https://twitter.com/CharlieYouAI)) 03:10 How Moin got started in computer science 05:50 Using ML to identify depression on Twitter in high school 11:00 Building a system to track phone locations on MIT’s campus 14:35 Specializing in NLP 17:20 Building an end-to-end fact-checking system (https://www.aclweb.org/anthology/N19-4014/ (https://www.aclweb.org/anthology/N19-4014/)) 25:15 Predicting statement stance with neural multi-task learning (https://www.aclweb.org/anthology/D19-6603/ (https://www.aclweb.org/anthology/D19-6603/)) 27:20 Is feature engineering in NLP dead? 29:40 Reconciling language models with existing knowledge graphs 35:20 How advances in AI hardware will affect NLP research (crazy!) 47:25 Moin’s research into sampling algorithms for natural language generation (https://arxiv.org/abs/2009.07243 (https://arxiv.org/abs/2009.07243)) 57:10 Under-rated areas of ML research 01:00:10 How research works at MIT CSAIL 01:04:35 How Moin keeps up in such a fast-moving field 01:11:30 Starting the MIT Machine Intelligence Community 01:16:30 Rapid Fire Questions Links: https://www.aclweb.org/anthology/N19-4014/ (FAKTA: An Automatic End-to-End Fact Checking System) https://stereoset.mit.edu/ (StereoSet: Measuring stereotypical bias in pretrained language models) https://www.aclweb.org/anthology/D19-6603/ (Neural Multi-Task Learning for Stance Prediction) http://www.incompleteideas.net/IncIdeas/BitterLesson.html (Rich Sutton - The Bitter Lesson) https://arxiv.org/abs/2009.07243 (A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation) https://arxiv.org/abs/1905.12265 (Strategies for Pre-training Graph Neural Networks) https://openreview.net/pdf?id=YicbFdNTTy (Transformers For Image Recognition at Scale) https://www.cerebras.net/product/ (Cerebras CS-1) https://www.tryklarity.com/ (Klarity: AI for Law Contract Review) https://www.mit.edu/~jda/ (Jacob Andreas) https://cs.stanford.edu/people/jure/ (Jure Leskovec) https://www.simonandschuster.com/books/Shoe-Dog/Phil-Knight/9781501135927 (Shoe Dog) https://en.m.wikipedia.org/wiki/Alexander_Hamilton_(book) (Hamilton) https://becomingmichelleobama.com/ (Becoming) https://www.penguinrandomhouse.com/books/44330/mindset-by-carol-s-dweck-phd/ (Mindset) https://en.m.wikipedia.org/wiki/The_Innovators_(book) (The Innovators)