Foundation Model

What is Foundation Model?

• AI > Machine Learning > Deep Learning > Foundation Models > LLM
  • Machine Learning includes SL (Supervised Learning), UL (Unsupervised Learning), & RL (Reinforcement Learning)
  • Traditional Machine learning plasy a pivotal role, shcu as LR(linear regression_ DT (Decision Tree), SVM (Support Vector Machine), CS (Clustering Algorithm), etc.
  • Foundation Models: Large scale neural networks trained on vast amounts of data

Features of Foundation Model

• Serve as a base or a foundation for a multitude of applications
• Instead of training a model from scratch for each specific task, take a pre-trained foundation model and fine-tune it for a particular application
• Save a lot of time & resources
• Trained on diverse datasets, capturing a broad range of knowledge
• Adaptable to tasks ranging from language translation to content generation to image recognition
• Sit within the DL category but represent a shift towards more generalized models

LLM (Large Language Models)

A specific type of foundation model centered around processing and generating human-like text.

• Large refers to the scale of the model
  • Possess a vast number of parameters, billions or even more
  • The enormity of these models contributes to their nuanced understanding and capabilities

    • Enormity: the great or extreme scale, seriousness, or extent of something perceived as bad or morally wrong. 

• Language is designed to understand and interact using human languages
  • Trained on massive datasets
  • Can grasp grammar, context, idioms, and even cultural references
• Model refers to the computational models at the core
  • A series of algorithms and parameters working together to process input and produce output

Other Applications

Vision

• Can interpret and generate images
• Some specific models can perform specific tasks
• Examples include:
  • Biology models for predicting how proteins fold into 3D shapes
  • Audio models for generating human-sounding speech or composing new music

Generative AI

• Pertains to models and algorithms specifically crafted to generate new content
• While foundation models provide the underlying structure and understanding, generative AI focuses on harnessing that knowledge to produce something new
• It's the creative expression that emerges from the vast knowledge base of these foundation models

Rerence
[1] IBM Technology, Machine Learning vs. Deep Learning vs. Foundation Models, https://www.youtube.com/watch?v=Beh13Cd_QbY