Machine Learning - Tag - Daily Deep Think https://blog.baifan.site/en/tags/machine-learning/ Machine Learning - Tag - Daily Deep Think Hugo -- gohugo.ioenblog@baifan.site (ByF) blog@baifan.site (ByF)Wed, 30 Jul 2025 14:00:00 +0800 AI Technical Glossary: A Complete Guide to 270+ Terms https://blog.baifan.site/en/ai-technical-glossary-complete-guide/ Wed, 30 Jul 2025 14:00:00 +0800 ByF https://blog.baifan.site/en/ai-technical-glossary-complete-guide/

AI Technical Glossary

This reference gathers the core terminology of the LLM field, from basic concepts to advanced technical architecture, to help you build a systematic understanding of the AI technology landscape.

πŸ“š Fundamentals

Term Technical definition Plain-language explanation Example
AGI (Artificial General Intelligence) An AI system with human-level intelligence An all-capable AI that can think, learn, and create like a person A robot that can write poetry, code, cook, and chat at the same time
LLM (Large Language Model) A large neural network model trained on massive data A “super brain” that understands and generates human language GPT-4, Claude, ERNIE Bot, and the like are all LLMs
Training The process of fitting neural network parameters on large data The AI’s “study phase” β€” like a person absorbing knowledge from books Training a model on all text on the internet until it learns language
Inference A trained model generating output from input The AI’s “application phase” β€” like a person answering questions from what they learned The model generating an answer after you ask it a question
Token The smallest unit of text a model processes, a fragment split by a tokenization algorithm The particles of AI language, processed one at a time "ζˆ‘ε–œζ¬’θ‹Ήζžœ" β†’ ["ζˆ‘", "ε–œζ¬’", "θ‹Ήζžœ"]

πŸ—οΈ Architecture

Term Technical definition Plain-language explanation Example
Transformer A deep learning architecture based on self-attention, proposed by Google in 2017 The “neural skeleton” of modern AI that lets models understand language efficiently GPT, BERT, and every other large model is built on Transformer
Encoder A neural network component that encodes an input sequence into semantic representations The AI’s “understanding unit” β€” turns text into vectors machines understand BERT uses an encoder for text understanding tasks
Decoder A neural network component that generates output token by token based on context The AI’s “writing unit” β€” generates answers from what it understood The GPT series are all decoder-only models
Self-Attention A mechanism that computes how much each element in a sequence relates to the others The AI automatically “focuses on what matters”, like a person picking out key points while reading In “deposit money at the bank”, “bank” attends to “money”; in “fish by the river bank”, it attends to “river”
Multi-Head Attention Several self-attention mechanisms run in parallel to capture different types of dependencies The AI understands text from multiple angles at once One head tracks syntax while another tracks semantics
Positional Encoding Vector representations that add position information to each token Lets the model know “who comes first, who comes later” “The dog bit the man” and “the man bit the dog” mean different things
Query The vector that actively asks for related information β€” what the current word needs The numeric expression of “what am I looking for” “Apple” queries attributes like taste and color
Key The identifier vector for information being queried β€” what each word can offer The label of “what I can provide” “Sweet” serves as the Key for a taste feature, waiting to be queried
Value The representation vector holding the actual content and true semantic information “My actual content”, in numbers The actual semantic representation of “sweet”: [0.8, 0.2, -0.1]
Attention Weight Importance scores expressing how much to attend, usually normalized via softmax “How much to pay attention”, quantified 0.8 means strong attention, 0.1 weak; all weights sum to 1
Cross-Attention Attention across two sequences β€” Query comes from one, Key/Value from another Cross-modal information exchange In image-text matching, text Queries attend to image Keys/Values
Causal Attention Attention restricted to the current position and earlier, preventing future information leaks Attention that can “only look backward” When GPT generates the 5th word it can only see the previous 4
Softmax Function An activation function that turns any real-valued vector into a probability distribution Converts scores into “importance percentages” [2,1,0] β†’ [0.67,0.24,0.09], preserving relative magnitudes

πŸ”’ Mathematical Representation

Term Technical definition Plain-language explanation Example
Vector A mathematical object with magnitude and direction; an ordered list of numbers A “numeric ID card” that describes a thing with numbers [25, 180, 70] can represent a person’s age, height, and weight
Embedding The technique of mapping discrete symbols into a continuous vector space Turns words into “numeric coordinates” "king"β†’[0.25, -0.12, 0.78, ...]
Query / Key / Value The three core vector matrices in self-attention: what is asked, what is labeled, what is delivered Query = what I want, Key = what I can offer, Value = my actual content Query=[0.1,0.2] asks about taste, Key=[0.8,0.1] labels sweetness, Value=[0.9,0.05] is the actual representation of sweetness
Feed-Forward Network Applies an independent nonlinear transform at each position Deepens the model’s understanding of each word From “spring” the model further associates “warmth, growth”
Layer Normalization Standardizes a layer’s inputs A “stabilizer” for training Prevents gradient explosion or divergence
Residual Connection A cross-layer connection that preserves the original information An “express lane” for information, preventing loss Like a shortcut path that keeps deep networks from degrading

πŸ”„ Processing Pipeline

Term Technical definition Plain-language explanation Example
Tokenizer Converts text into a sequence of tokens A “knife for chopping text” "Hello world" β†’ ["Hello", " world"]
Context Window The maximum number of tokens a model can process The AI’s “memory limit” GPT-4 has a 128K context
Decoding Generates text token by token from a probability distribution The AI’s “writing process” Starts generating from the most probable word
Temperature A parameter controlling generation randomness A “creativity dial” High temperature is more creative, low more stable
Top-p Sampling A sampling strategy based on cumulative probability An “essence filter” Only considers candidates whose cumulative probability reaches 90%
Max Tokens Caps the length of generated output A “word-count limiter” Keeps the AI from answering too long

πŸ› οΈ Engineering Practice

Term Technical definition Plain-language explanation Example
RAG (Retrieval-Augmented Generation) An AI approach combining retrieval and generation An “open-book exam” AI Look up references first, then answer the question
Prompt Engineering The craft of designing and optimizing prompts “The art of asking” Helping the AI understand your needs better
Fine-tuning Training a pretrained model on a specific task “Targeted job training” Turning a general model into a medical assistant
BPE (Byte Pair Encoding) A common tokenization algorithm A “text compression technique” "unhappiness" β†’ ["un","happi","ness"]
Detokenization Turns a token sequence back into readable text “Reassembling the pieces” ["ζˆ‘","ε–œζ¬’","θ‹Ήζžœ"]β†’"ζˆ‘ε–œζ¬’θ‹Ήζžœ"
Streaming Generates output token by token in real time The “typewriter effect” A chatbot thinking while it types

🧠 Classic Models Compared

Term Technical definition Plain-language explanation Example
RNN (Recurrent Neural Network) A neural network that processes sequences step by step A “read-one-word-at-a-time AI” Translating "ζˆ‘ηˆ±δ½ " word by word
LSTM (Long Short-Term Memory) An improved RNN that handles long-range dependencies “A better memory” Can remember content from the beginning
CNN (Convolutional Neural Network) A neural network that excels at image patterns An “image specialist” Recognizing cats, dogs, and faces
Encoder-Decoder Architecture A model containing both understanding and generation modules An “all-round AI” Machine translation models

πŸ“Š Application Scenarios

Term Technical definition Plain-language explanation Example
Chat product A user-facing AI application interface An “AI chat shell” ChatGPT, Claude
API call An interface for program-to-program communication The “AI phone line” An application calling the OpenAI API
Context management The technique of maintaining conversation history The “AI’s memory” A chatbot remembers what you said
Multi-turn dialogue A continuous human-machine interaction mode “Ongoing conversation” Ask about the weather, then what to wear
Function Calling The model invoking external APIs to perform tasks The “AI’s ability to act” The AI checks the weather or searches automatically

🧩 Model Optimization and Training Techniques

Term Technical definition Plain-language explanation Example
LoRA (Low-Rank Adaptation) Fine-tunes model parameters via low-rank matrices “Lightweight fine-tuning” Lets an LLM quickly adapt to a new domain
Quantization Represents model parameters at lower precision “Slimming the model down” FP32β†’INT8 speeds up inference
Pruning Removes redundant neurons or connections “Trimming the branches” Cutting useless parameters
Distillation (Knowledge Distillation) A large model teaches a small one “Teacher trains the student” GPT-4 teaching a small model
Checkpoint A saved intermediate state during model training A “training save point” Prevents losing progress on a power cut

πŸ” Vector Retrieval and Knowledge Integration

Term Technical definition Plain-language explanation Example
Embedding Model A model that converts text into semantic vectors A “semantic coordinate machine” text-embedding-3-large
Vector Database A database supporting vector retrieval A “semantic warehouse” Milvus, Pinecone, FAISS
Cosine Similarity Measures how similar two vectors’ directions are A “semantic similarity meter” A cat is sleeping β‰ˆ The cat is resting
Knowledge Graph Stores knowledge as nodes and relationships A “knowledge map” apple β†’ is a β†’ fruit
Hybrid Search Combines semantic retrieval with keyword matching “Belt-and-suspenders search” Searching both cat and pet animal at once

🧩 Multimodal and Agents

Term Technical definition Plain-language explanation Example
Multimodal Model Handles text, images, audio, and other modalities at once A “full-senses AI” GPT-4V, Gemini
VLM (Vision-Language Model) Vision-Language Model An AI that can “see pictures” A visual question-answering AI
Speech Recognition Converts speech to text A “dictation AI” Voice input methods
TTS (Text-to-Speech) Converts text to speech An “AI announcer” The AI reads its answer aloud
AI Agent An AI capable of autonomous action and decision-making An AI assistant “that can act” Devin, AutoGPT

βš™οΈ Model Evaluation and Safety

Term Technical definition Plain-language explanation Example
Hallucination The model generating false information “Confident nonsense” Inventing papers or facts
Alignment Bringing the model in line with human values “Values training” Tuning a model with RLHF
RLHF (Reinforcement Learning from Human Feedback) Optimizes a model with human preferences “Humans teaching AI to speak” How ChatGPT was trained
Red Teaming Adversarial testing of model safety A “security penetration test” Testing whether the model leaks secrets
Bias Systematic prejudice in model outputs “The AI plays favorites” Preference for a gender or language

🧰 Emerging Trends and Future Directions

Term Technical definition Plain-language explanation Example
Mixture of Experts A structure with multiple sub-models activated dynamically A “panel-of-experts AI” The Gemini 1.5 Pro architecture
Context Compression Compresses conversation history to save tokens “Memory compression” Summarizing long conversations
Memory-Augmented Model An AI combined with long-term memory mechanisms An AI “with a memory” ChatGPT long-term memory
Autonomous Agent An AI that can plan and execute tasks on its own A “self-managing AI” AutoGPT, Devin
Synthetic Data Virtual training data generated by AI “AI-made textbooks” Expanding a training set with AI

πŸ’‘ Study Advice

🎯 Priority for Mastering Core Concepts

  1. Beginner (must know): Token, Embedding, Transformer, LLM
  2. Intermediate (important): Self-Attention, RAG, Context Window
  3. Advanced (optional): LoRA, Mixture of Experts, Red Teaming

πŸ“– Suggested Learning Path

  1. Understand the basics: what a token is, and why vector representation is needed
  2. Master the core architecture: the Transformer encoder-decoder structure
  3. Practice application techniques: combining prompt engineering with RAG
  4. Go deeper into technical details: attention mechanisms and alignment training

πŸ”— Concept Map