Guest Percy Liang is an authority on AI who says that we are undergoing a paradigm shift in AI powered by foundation models, which are general-purpose models trained at immense scale, such as ChatGPT. In this episode of Stanford Engineering’s The Future of Everything podcast, Liang tells host Russ Altman about how foundation models are built, how to evaluate them, and the growing concerns with lack of openness and transparency.Connect With Us:Episode Transcripts >>> The Future of Everything WebsiteConnect with Russ >>> Threads / Bluesky / MastodonConnect with School of Engineering >>> Twitter/X / Instagram / LinkedIn / FacebookChapters:(00:00:00) IntroductionHost Russ Altman introduces Percy Liang, who runs the Stanford Center on Foundation Models (00:02:26) Defining Foundation ModelsPercy Liang explains the concept of foundation models and the paradigm shift they represent. (00:04:22) How are Foundation Models Built & Trained?Explanation of the training data sources and the scale of training data: training on trillions of words. Details on the network architecture, parameters, and the objective function.(00:10:36) Context Length & Predictive CapabilitiesDiscussion on context length and its role in predictions. Examples illustrating the influence of context length on predictive accuracy. (00:12:28) Understanding HallucinationPercy Liang explains how foundation models “hallucinate”, and the need for both truth and creative tasks which requires “lying”. (00:15:19) Alignment and Reinforcement in TrainingThe role of alignment and reinforcement learning from human feedback in controlling model outputs. (00:18:14) Evaluating Foundation ModelsThe shift from task-specific evaluations to comprehensive model evaluations, Introduction of HELM & the challenges in evaluation these models. (00:25:09) Foundation Models Transparency IndexPercy Liang details the Foundation Models Transparency Index, the initial results and reactions by the companies evaluated by it.(00:29:42) Open vs. Closed AI Models: Benefits & RisksThe spectrum between open and closed AI models , benefits and security impacts Connect With Us:Episode Transcripts >>> The Future of Everything WebsiteConnect with Russ >>> Threads / Bluesky / MastodonConnect with School of Engineering >>>Twitter/X / Instagram / LinkedIn / Facebook Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.