1  INTRODUCTION TO MACHINE LEARNING

1.1 Machine Learning

Machine Learning

What is the Learning Problem?

Learning \equiv Improving with experience at some task

  • Improve over task T,
  • with respect to performance measure P,
  • based on experience E.

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” – Tom Mitchell

Inductive Learning

  • Induction: Generalizing from specific examples to general rules (Machine Learning).
  • Deduction: Deriving specific conclusions from general premises (Traditional Logic).

Machine Learning vs. Traditional Programming

  • Traditional Programming: You provide the Data and the Rules (Program). The computer outputs the Answers.
  • Machine Learning: You provide the Data and the Answers (Labels). The computer outputs the Rules (Model).

Machine Learning vs. Artificial Intelligence

  • Artificial Intelligence (AI): The broad science of mimicking human abilities.
  • Machine Learning (ML): A subset of AI that trains a machine how to learn.
  • Deep Learning (DL): A subset of ML using multi-layered neural networks.
  • General AI (GenAI): A subset of AI that can perform any intellectual task that a human can perform.

Interpretable Machine Learning

Explainable Artificial Intelligence (XAI)

As models (like Deep Neural Networks) become more complex/accurate, they often become less interpretable (“Black Box”).

  • Interpretability: The degree to which a human can understand the cause of a decision.
  • Explainability: Techniques to explain the predictions of black-box models.

Goals of XAI:

  1. Trust: Why did the model make this decision?
  2. Safety: Will it fail in edge cases?
  3. Ethics: Is the decision biased or discriminatory?

Human-AI Interaction

Effective collaboration requires mutual understanding:

  1. System’s Model of User: The AI needs to understand human intent and context.
  2. User’s Model of System: The human needs a mental model of how the AI works to use it effectively.

1.2 Learning Agent

Why Agents Need Learning

  1. Unknown Environments: Designers cannot anticipate every possible state or interaction.
  2. Changing Environments: The world changes over time; fixed programs fail.
  3. Efficiency: It is often easier to let an agent “learn” a complex task (like walking or recognizing faces) from data than to manually code the rules.

Forms of Learning

How an agent learns depends on four factors:

  1. Component: Which part of the agent is improving? (e.g., world model, policy, utility).
  2. Prior Knowledge: What does the agent already know?
  3. Representation: How are data and functions represented? (Linear weights, Neural Networks, etc.).
  4. Feedback: What signal is used to learn?

Components that can be Learned

Virtually any part of an intelligent agent can be learned:

  1. Atomic Components: Mapping states directly to actions.
  2. World Models: Predicting the next state given current state and action (Transition Model).
  3. Utility Functions: Learning how desirable a state is.
  4. Action-Value Functions: Learning the expected future reward of an action (Q-values).
  5. Goals: Inferring high-level objectives.

Types of Feedback (Learning Paradigms)

The type of feedback dictates the learning algorithm:

  • Supervised Learning: The agent observes Input-Output pairs (Teacher gives the correct answer).
    • Task: Learn a function f(x) \to y.
    • Example: Image Classification, Spam Detection.
  • Unsupervised Learning: The agent observes Patterns in data without labels.
    • Task: Find structure in input x.
    • Example: Clustering, Dimensionality Reduction.
  • Reinforcement Learning: The agent learns from Rewards/Punishments.
    • Task: Learn a policy to maximize long-term reward.
    • Example: Playing Chess, Robot Navigation.
  • Semi-Supervised Learning: A mix of a small amount of labeled data and a large amount of unlabeled data.

Machine Learning Workflow

A systematic approach to building ML systems:

  1. Data Collection: Gather raw data (the fuel for ML).

  2. Data Preparation: Clean, normalize, and transform data. Handle missing values.

    Exploratory Data Analysis (EDA): Visualize data to understand distributions and correlations.

  3. Model Building: Select algorithms and train models on the training set.

  4. Evaluation: Test the model on unseen data (validation/test sets) to measure performance.

  5. Deployment: Integrate the model into the production environment.

  6. Monitoring: Continuously track performance to detect “drift” and retrain when necessary.

1.3 Data

Types of Data

  • Record

    • Relational records

    • Data matrix: numerical matrix

    • Document data: text documents

    • Transaction data

  • Graph and network

    • World Wide Web

    • Social or information networks

    • Molecular Structures

  • Ordered

    • Video data: sequence of images

    • Temporal data: time-series

    • Sequential Data: transaction sequences

    • Genetic sequence data

  • Spatial, image and multimedia

    • Spatial data: maps

    • Image data

    • Video data

Data Sets

  • Data sets are made up of data objects.

  • A data object represents an entity (also called samples, examples, instances, data points, objects, tuples).

  • Data objects are described by attributes.

Attributes

An attribute (also called dimension, feature, variable) is a data field, representing a characteristic or feature of a data object.

Data set Attributes
Sales database customers, store items, sales
Medical database patients, treatments
University database students, professors, courses

Attribute Types

  • Nominal: categories, states, or “names of things”

    • hair color = {auburn, black, blond, brown, grey, red, white}

    • marital status, occupation, ID numbers, zip codes

  • Binary: nominal attribute with only two states

    • gender (symmetric: both outcomes equally important)

    • medical test (positive vs. negative) (asymmetric: outcomes not equally important)

  • Ordinal: values have a meaningful order but magnitude between successive values is not known.

    • size = {small, medium, large}

    • grades, rankings

Numeric Attribute Types

  • Quantity (integer or real-valued)

  • Interval scale: Measured on a scale of equal-sized units. No true zero-point. Values have order

    • temperature in Celsius or Fahrenheit

    • calendar dates

  • Ratio scale: Inherent zero-point

    • temperature in Kelvin

    • length

    • counts

    • monetary quantities

Discrete vs. Continuous Attributes

  • Discrete Attribute: Has only a finite or countably infinite set of values

    • zip codes

    • profession

    Note: Sometimes, represented as integer variables; binary attributes are a special case of discrete attributes

  • Continuous Attribute: Has real numbers as attribute values

    • temperature

    • height, or weight

Sample datasets

  • Housing data for 506 census tracts of Boston from the 1970 census

  • Iris data set gives the measurements in centimeters of the variables sepal length and width and petal length and width, respectively, for 50 flowers from each of 3 species of iris.

  • BBC news dataset

    # sentences class
    0 Labour plans maternity pay rise\n\nMaternity p... politics
    1 Watchdog probes e-mail deletions\n\nThe inform... politics
    2 Hewitt decries ‘career sexism’\n\nPlans to ext... politics
    3 Labour chooses Manchester\n\nThe Labour Party ... politics
    4 Brown ally rejects Budget spree\n\nChancellor ... politics
    ... ... ...
    2220 Trial begins of Spain’s top banker\n\nThe tria... business
    2221 UK economy ends year with spurt\n\nThe UK econ... business
    2222 HealthSouth ex-boss goes on trial\n\nThe forme... business
    2223 Euro firms miss out on optimism\n\nMore than 9... business
    2224 Lacroix label bought by US firm\n\nLuxury good... business

Large Language Models

Year Model # parameters dataset size
2018 ELMo 9.36*10^{7} \approx 6GB *
2019 BERT 3.4*10^{8} 16 GB
2019 Megatron-LM 8.3*10^{9} 174 GB
2020 GPT-3 1.75*10^{11} 570 GB
2020 GShard 6.00*10^{11} \dagger
2021 Switch-C 1.57*10^{12} 745 GB
2021 Gopher 2.8*10^{11} \approx 1800 GB \ddagger
2022 PaLM 5.4*10^{11} \approx 4680 GB \$
*

1 billion words

\dagger

25 billion training examples (100 languages)

\ddagger

300 billion tokens

\$

780 billion tokens

How Much Data is that?

Someone who reads a 400 page book each day will take how long to read the same amount of information as GPT-3 was trained on?

  1. 2 years

  2. 20 years

  3. 200 years

  4. > 2000 years

Tensor-based Attributes

A tensor is a generalized matrix, a finite table of numerical values indexed along several discrete dimensions.

  • A 0d tensor is a scalar

  • A 1d tensor is a vector (e.g. a sound sample)

  • A 2d tensor is a matrix (e.g. a grayscale image)

  • A 3d tensor (e.g. a multi-channel image)

  • A 4d tensor (e.g. a sequence of multi-channel images)

Sample dataset

  • 28\times28 grayscale images, 60k train samples, 10k test samples.

1.4 Exploration

Data Visualization

Visualization is the conversion of data into a visual or tabular format so that the characteristics of the data and the relationships among data items or attributes can be analyzed or reported.

Humans have a well developed ability to analyze large amounts of information that is presented visually

  • Can detect general patterns and trends

  • Can detect outliers and unusual patterns

1.5 Type of Learnings

Basic Premise of Learning

“Using a set of observations to uncover an underlying process or rule”

broad premise \Longrightarrow many variations

  • Supervised learning

  • Unsupervised learning

  • Reinforcement learning

Supervised Learning

  • We get data \mathcal{D}: (input, correct ouput)

    • When the output is one of a finite set of values, the learning problem is called classification

    • When the output is a number, the learning problem is called regression

  • Example from vending machine - coin classification

Unsupervised Learning

  • Instead of (input, correct output), we get (input, ?)

Reinforcement Learning

  • We get data \mathcal{D}: (observation, action, and reward)

1.6 Successes of Machine Learning

Vision

Detection and Segmentation

  • Object Detection: State-of-the-art models like YOLO (You Only Look Once) and EfficientDet can detect objects in real-time with high accuracy.
  • Segmentation: The Segment Anything Model (SAM) from Meta (2023) can “cut out” any object in any image with a single click, demonstrating zero-shot generalization.

Face recognition

  • Modern systems achieve near-perfect accuracy in unconstrained environments.
  • Widely used in security, authentication (FaceID), and photo organization.

Generative Vision

  • Image Generation: Models like DALL-E 3, Midjourney, and Stable Diffusion can generate photorealistic images from text descriptions.
  • Video Generation: Models like Sora (OpenAI) and Gen-3 Alpha (Runway) are pushing boundaries in generating coherent video clips from text.

Language

Large Language Models (LLMs) & Chatbots

  • ChatGPT (OpenAI): Based on GPT-4, it exhibits human-level performance on various professional and academic benchmarks.
  • Claude (Anthropic): Known for large context windows and strong reasoning capabilities.
  • Gemini (Google): A multimodal model capable of understanding and reasoning across text, images, audio, video, and code.

Translation

  • Real-time Translation: Models like Meta’s SeamlessM4T enable nearly instantaneous speech-to-speech translation across nearly 100 languages.

“The reason Boeing are doing this is to cram more seats in to make their plane more competitive with our products,” said Kevin Keniston, head of passenger comfort at Europe’s Airbus.

\downarrow

Kevin Keniston, người đứng đầu bộ phận thoải mái của hành khách tại Europe’s Airbus, cho biết: “Lý do Boeing làm điều này là để nhồi nhét thêm ghế để máy bay của họ cạnh tranh hơn với các sản phẩm của chúng tôi.”

Code Generation

  • Coding Assistants: Tools like GitHub Copilot and Cursor, powered by models like OpenAI’s Codex and GPT-4, assist developers by writing entire functions, debugging code, and translating between languages.

Games

Strategic Gaming

  • AlphaGo (2016): Beat world champion Lee Sedol at Go.
  • AlphaStar (2019): Achieved Grandmaster level in StarCraft II.
  • Cicero (2022): Mastered the game of Diplomacy, combining strategic reasoning with natural language negotiation to cooperate with humans.

Robotics

Embodied AI

  • Robotic Transformers (RT-2): Google’s vision-language-action (VLA) model that controls robots using web-scale data.
  • Humanoid Robots: Tesla Optimus and Boston Dynamics’ Atlas are rapidly evolving to perform complex, dynamic tasks in human environments.

Science

Accelerating Discovery

  • AlphaFold 3 (2024): Predicts the structure and interactions of all life’s molecules (proteins, DNA, RNA, ligands) with high accuracy.
  • GNoME (2023): Discovered 2.2 million new crystals, accelerating materials science by orders of magnitude.

1.7 References