L-3

Large Language Models

HKU School of Computing and Data Science

L2 Exit Ticket Results

Language

Library interior with long rows of books
Language
Our Most Powerful Invention

Language: The Meta-Tool

Our Most Powerful Tool

Foundation for human progress and innovation

Meta-Tool: A Tool for Creating Other Tools

Enables creation of new tools

Allows sharing of knowledge

Supports refinement and improvement over time

From mastering fire to landing on the Moon,
language lets us build on each other’s ideas

Language and Machines

Language lets us pass knowledge across generations.
Train AI on humanity’s
accumulated knowledge
Use language to build
on that knowledge

Why Is Language Difficult?

Complex
Irregularities, exceptions, idioms
“I broke the ice.”
Ambiguous
Context, tone, and culture shape meaning
“I saw her duck.”
Creative
New words, slang, evolving structures
“My brain has 47 tabs open, and I can’t
find which one is playing music.”
Layered
Emotional, cultural, and social dimensions
“YOU ate my cake?” → You, of all people?
“You ATE my cake?” → You ate it rather than saving it?
“You ate MY cake?” → Mine, rather than yours?

How Do Machines Learn Language?

Fundamental Idea: Predict What Comes Next

The robot made breakfast, but burned the …

·toast2170758.3%
·eggs1823716.1%
·bread155303.0%
·bacon390152.1%
·pancakes771091.9%
·egg180751.8%
·coffee104821.8%
·food34871.7%
Next-token probabilities

Different Inputs, Different Outputs

Last week: predict a score
This week: predict what comes next in text

From One Prediction to a Reply

How can I stop my toast from burning?

Qwen3.5-9B Base
Vocabulary: 248,320 tokens
DEMO 2 · Tokenization

What Is a Token?

unbelievable unbelievably unbelievable!
How many tokens?
un354belie30754vable23017·unbelie37649vably85098·unbelievable49408!0
7 tokens
Tokenization varies by model · Qwen3.5-9B Base
A token is a piece of text:
a whole word, part of a word, or punctuation
Each token has an ID that the model uses
Token ID → Embedding → List of numbers

Qwen3.5-9B Base
DEMO 3 · Choosing the Next Token

How Is the Model Trained?

Stage IText CollectionsEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Base modelStage IIExample ConversationsEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Instruct modelStage IIIHuman FeedbackEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Aligned modelPre-trainingFine-tuning

Stage I

FineWeb
How Was FineWeb Created?
An example of pre-training data
Common Crawl
One crawl · August 2026
2.14 billion web pages
33.1 million registered domains
360 TiB uncompressed
Remove websites
on the blocklist
Extract readable text
from HTML pages
Keep English text
(language score ≥ 0.65)
Filter low-quality text
and remove duplicates
Mask personal information
(email and IP addresses)
FineWeb2 extends this approach
to 1,000+ languages

Stage I

My cat is secretly stealing my socksTarget token·stealing36742EmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network My4888·cat7993·is369·secretly40361·eating1183410.9%·stealing367427.0%·hiding242445.2%·taking44833.7%·drinking156872.4%·afraid160872.0%·using16082.0%·feeding241991.8%Base model

Qwen3.5-9B Base
DEMO 4 · Continuous Generation

Stage II

Same neural network · Different training data
User
Explain monopsony in simple terms
Assistant
Monopsony means there is one main buyer.
For example, a town’s only large employer
may have power to keep wages low
People writing and reviewing examples
Human-written
or generated
and reviewed
Collect carefully checked request–response examples

Stage II

Monopsony means there is one main buyerTarget token·one799EmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network ·only113288.2%·just10666.0%·one7994.2%·a2641.1%·usually58020.1%·**29720.0%·very15460.0%·basically128370.0%Reviewed exampleExplain monopsonyin simple termsMonopsony meansthere isInstruct model
Qwen3.5-9B
DEMO 5 · Base vs. Instruct

Stage III

Explain monopsony to a first-year student
A

Monopsony is a market structure characterised by a single buyer with market power over suppliers

B

When one employer is the main buyer of labour in a town, workers have fewer alternatives and less power to negotiate pay

Which is clearer for this reader, and why?

Stage III

People compare answers Across many examples Train Reward model Predicts preference scores Predicts next tokens LLM Text Formal definition Concrete example Prompt + answer Score Scores guide LLM weight updates

Training Complete

Stage IText CollectionsEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Base modelStage IIExample ConversationsEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Instruct modelStage IIIHuman FeedbackEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Aligned modelPre-trainingFine-tuning

Training and Inference

Training Learn from examples and feedbackEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Weights change Inference Use context to generate the next tokenEmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer Network Weights stay fixed A longer conversation changes the context

Attention Uses Context

Attention helps predict the next token by using relevant earlier tokensWhat comes next?Which earlier tokens matter?EmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer NetworkAttention Inside the TransformerCurrent token:·the279Attention close-upNext-token probabilities·money311732.3%·cash809111.6%·envelope332565.6%·cheque743774.7%·large33493.3%·bag86972.9%·bundle125242.5%·new4911.9%What comes next?Which earlier tokens matter?EmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣTransformer NetworkAttention Inside the TransformerCurrent token:·the279Attention close-upNext-token probabilities·fish739772.0%·catch22135.7%·______315861.9%·basket136211.0%·bag86970.9%·heavy85450.7%·____297390.6%·__12590.4%She7691·cash8091ed290·the279·cheque74377,11·then1179·left2047·the279·bank5883·carrying14885·the279She7691·caught10255·the279·fish7397,11·then1179·left2047·the279·bank5883·carrying14885·the279

From LLM to Agent

An LLM generates tokens. An agent uses them to act. LLM EmbeddingΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣΣ Input click ( … ) Generated tokens Tool instruction Software acts Result becomes new input Agent

Activity