Saturday, July 11, 2026
- Group Sequence Policy Optimization (GSPO)v2Group Sequence Policy Optimization (GSPO) is a reinforcement learning algorithm for training large language models, introduced by the Qwen team at Alibaba in...
- Gradient checkpointingv4Gradient checkpointing, also called activation checkpointing, activation recomputation, or rematerialization, is a memory-saving technique for training deep...
- LongRoPEv3LongRoPE is a context-window extension technique for large language models (LLMs) that use rotary position embeddings (RoPE). Introduced by researchers at...
- Score matchingv4Score matching is a method for fitting a probabilistic model by matching the gradient of its log-density, the so-called score function , to the same gradient...
- xLSTMv3xLSTM (Extended Long Short-Term Memory) is a recurrent neural network architecture introduced in May 2024 by Maximilian Beck, Korbinian Pöppel, Sepp...
- DoRA (Weight-Decomposed Low-Rank Adaptation)v5DoRA (Weight-Decomposed Low-Rank Adaptation) is a parameter-efficient fine-tuning (PEFT) method for large neural networks introduced in February 2024 by...
- Tanh (hyperbolic tangent)v3The hyperbolic tangent, written tanh, is a smooth, S-shaped activation function that maps any real number into the open interval , passing through the origin...
- Dynamic Programmingv4Dynamic programming (DP) is an algorithmic technique that solves a complex problem by breaking it into simpler overlapping subproblems, solving each subproblem...
- Weighted Alternating Least Squares (WALS)v6Weighted Alternating Least Squares (WALS), also called implicit Alternating Least Squares (iALS) or weighted regularized matrix factorization (WRMF), is a...
- Wasserstein Lossv5Wasserstein loss is a loss function for training generative models that measures the distance between two probability distributions as the Wasserstein-1...
- Output Layerv10See also: neural network, activation function, loss function, hidden layer, softmax, backpropagation The output layer is the final layer of a neural network:...
- Mean Squared Error (MSE)v7See also: Machine learning terms Mean Squared Error (MSE), also called mean squared deviation (MSD), is the average of the squared differences between...
- Feature Vectorv8A feature vector is an n-dimensional, ordered list of numerical values that represents the measurable properties of an object, data point, or observation in a...
- Centroidv5See also: Machine learning terms A centroid is the geometric center of a set of points, computed as the arithmetic mean of their coordinates: each component of...
- Embedding Layerv8An embedding layer is a neural network component that acts as a trainable lookup table, mapping discrete integer indices (such as word IDs, user IDs, or...
- Baggingv6See also: Machine learning terms, Ensemble methods, Random forest, Boosting Bagging, short for bootstrap aggregating, is an ensemble learning technique in...
- Diffusion modelv11A diffusion model is a type of generative model that produces data by learning to reverse a gradual noising process: it is trained so that if Gaussian noise is...
- Task arithmeticv3Task arithmetic is a model-editing technique that steers the behavior of a neural network by adding or subtracting vectors in its weight space. The central...
- LongNetv3LongNet is a transformer variant introduced by Microsoft Research in July 2023 that is designed to scale attention to sequences exceeding one billion tokens...
- Medusav4Medusa is a large language model inference acceleration framework that speeds up text generation by adding multiple lightweight decoding heads on top of an...
- Latent Dirichlet allocationv4Latent Dirichlet allocation (LDA) is a generative probabilistic model that discovers the hidden thematic structure in a collection of documents by treating...
- Policy gradient methodsv3Policy gradient methods are a family of reinforcement learning algorithms that directly parameterise the agent's policy and optimise it by stochastic gradient...
- Hidden Markov Modelv4A Hidden Markov Model (HMM) is a statistical model of sequential data in which an unobserved (hidden) sequence of discrete states follows a Markov process, and...
- Layer normalizationv7Layer normalization is a technique for normalizing the activations of a neural network across the feature dimension of each individual sample, rather than...
- Step sizev5See also: learning rate, parameter update, Adam, SGD In machine learning, the step size (also called the learning rate, usually written as the Greek letter ...
- User matrixv6See also: Machine learning terms In collaborative filtering and matrix factorization recommender systems, the user matrix (commonly written U or P) is the...
- Model Parallelismv8See also: Machine learning terms, Data parallelism, GPU computing, Deep learning Model parallelism is a distributed training and inference technique that...
- Reinforcement Learning from Human Feedback (RLHF)v9Reinforcement Learning from Human Feedback (RLHF) is a machine learning technique that trains artificial intelligence systems to behave according to human...
- Hyperparameterv11See also: Machine learning terms A hyperparameter is a configuration setting in a machine learning algorithm that is fixed by the practitioner before training...
- Exploding Gradient Problemv11See also: Machine learning terms, Vanishing gradient problem, Gradient clipping, Backpropagation The exploding gradient problem is a training failure in deep...
- Criticv7A critic in reinforcement learning (RL) is the component of an actor-critic system that estimates a value function, scoring how good the actor's chosen actions...
- L2 Lossv4L2 loss is the squared-error loss function: for a true value and a predicted value , it is the squared difference , and averaging it across a dataset gives...
- State (Reinforcement Learning)v7In reinforcement learning (RL), a state is a complete description of the environment at a particular point in time, containing all the information an agent...
- DeepNorm / DeepNetv2DeepNorm is a normalization and weight initialization scheme for Transformer networks that makes the training of very deep models stable. It was introduced by...
- EnCodecv4EnCodec is a real-time neural audio codec developed by Meta AI's FAIR (Fundamental AI Research) team that compresses speech, ambient sound, and music into a...
- Sparse Codingv3Sparse coding is a representation learning principle in which a signal is encoded as a linear combination of a small number of elements drawn from a (usually...
- AdaBoostv3AdaBoost (short for Adaptive Boosting) is a machine learning ensemble algorithm that combines many weak classifiers into a single strong classifier through a...
- k-Nearest Neighborsv6k-Nearest Neighbors (often abbreviated k-NN or KNN) is a non-parametric, instance-based supervised learning algorithm that classifies a new data point by...
- Recursive reward modelingv2Recursive reward modeling (RRM) is a proposed approach to the scalable oversight problem in AI alignment, in which agents trained by reward modeling are...
- Florence-2v3Florence-2 is a vision foundation model developed by Microsoft Research that handles a wide range of computer vision and vision-language tasks through a single...
- Evan Hubingerv3Evan Hubinger is an American AI safety researcher who leads the alignment stress-testing team at Anthropic, where he serves as a Member of Technical Staff and...
- BitNet b1.58v3BitNet b1.58 is a ternary-weight large language model architecture from Microsoft Research in which every weight is constrained to one of three values, -1, 0,...
- Bias-variance tradeoffv8The bias-variance tradeoff is a foundational concept in machine learning and statistics that describes the tension between two competing sources of error in...
- Decoding strategiesv5Decoding strategies are the algorithms that select output tokens from a language model's next-token probability distribution during text generation. At each...
- Expectation-Maximization (EM) Algorithmv7The Expectation-Maximization (EM) algorithm is an iterative method for finding maximum likelihood or maximum a posteriori (MAP) estimates of the parameters of...
- Grad-CAMv4Grad-CAM (Gradient-weighted Class Activation Mapping) is a technique for producing visual explanations from convolutional neural network (CNN) models by using...
- Grouped-Query Attentionv8Grouped-query attention (GQA) is an attention mechanism for transformer language models that partitions the query heads into a small number of groups, where...
- Rotary Position Embeddingv10Rotary Position Embedding (RoPE) is a positional encoding method for transformer models that encodes a token's absolute position by rotating its query and key...
- Sigmoid Functionv11See also: Machine learning terms The sigmoid function is a mathematical function that maps any real number to a value between 0 and 1 using the formula ,...
- Online learningv6See also: Machine learning terms Online learning is a machine learning paradigm in which a model receives data sequentially, one example or one mini-batch at a...
- Loss Functionv11See also: Machine learning terms A loss function is a mathematical formula that measures how far a machine learning model's prediction is from the correct...
- Cross-Entropyv6See also: Machine learning terms, Loss function, Entropy Cross-entropy is a measure from information theory of how many bits (or nats) are needed to encode...
- Entropyv5Entropy is a quantitative measure of the uncertainty, randomness, or average information content in a probability distribution: for a discrete random variable...
- Pruningv7Pruning is a family of techniques used in machine learning and artificial intelligence to remove parts of a model or search space that are estimated to be...
- ZeRO (Zero Redundancy Optimizer)v3ZeRO (Zero Redundancy Optimizer) is a family of memory-optimization techniques for training large neural networks introduced by Samyam Rajbhandari, Jeff...
- Cosine similarityv5Cosine similarity is a measure of similarity between two non-zero vectors that calculates the cosine of the angle between them, defined as the dot product of...
- ROC (Receiver Operating Characteristic) Curvev7A Receiver Operating Characteristic (ROC) curve is a graph that measures how well a binary classification system separates two classes by plotting its true...
- Multinomial regressionv4See also: Machine learning terms Multinomial regression is a statistical model that predicts which one of K possible categories an observation belongs to,...
- Logitsv7In machine learning and statistics, logits refer to the raw, unnormalized scores output by a model before they are converted into probabilities. The term has...
- Least Squares Regressionv5Least squares regression is a statistical method that fits a model to data by choosing the parameters that minimize the sum of the squared residuals, the...
- Classification Thresholdv6A classification threshold (also called a decision threshold or cut-off point) is a numeric value used to convert the continuous probability output of a...
- DDIM (Denoising Diffusion Implicit Models)v4Denoising Diffusion Implicit Models (DDIM) are a class of iterative generative models, introduced by Jiaming Song, Chenlin Meng, and Stefano Ermon of Stanford...
- Adafactorv3Adafactor is a memory-efficient adaptive learning-rate optimizer for training deep neural networks, introduced by Noam Shazeer and Mitchell Stern in the 2018...
- Principal Component Analysis (PCA)v6Principal component analysis (PCA) is an unsupervised learning technique for dimensionality reduction that identifies the orthogonal directions of maximum...
- Adversarial attackv7An adversarial attack is a technique for crafting inputs that are deliberately designed to cause artificial intelligence systems, particularly machine learning...
- Kolmogorov-Arnold Networkv4A Kolmogorov-Arnold Network (KAN) is a type of neural network architecture proposed as an alternative to the traditional Multi-Layer Perceptron (MLP)....
- Target Networkv5A target network is a separate, slowly updated copy of a neural network used in deep reinforcement learning to compute stable learning targets, decoupling the...
- Elastic Netv4See also: Regularization, Linear regression Elastic Net is a regularization and variable selection method for linear regression and other generalized linear...
- Temperature samplingv5Temperature sampling is the most common decoding control in large language models: a single hyperparameter, written T, that divides the model's output logits...
- SHAP (SHapley Additive exPlanations)v5See also: explainable AI, feature importance, LIME, permutation feature importance SHAP (SHapley Additive exPlanations) is a game-theoretic method that...
- Adam optimizerv6The Adam optimizer (short for Adaptive Moment Estimation) is an algorithm for first-order gradient descent-based optimization of stochastic objective...
- Sparse attentionv6Sparse attention is a family of techniques that cut the computational and memory cost of the attention mechanism in transformer models by letting each token...
- Full Softmaxv6Full softmax (also called the standard softmax or exact softmax) is the softmax computation that calculates a probability for every possible output class in a...
- AUC-ROCv9See also: Machine learning terms AUC (Area Under the Curve), most often the area under the ROC curve (AUC-ROC), is a threshold-independent evaluation metric...
- SARSA (State-Action-Reward-State-Action)v3SARSA (State-Action-Reward-State-Action) is an on-policy temporal-difference (TD) control algorithm that learns the action-value function Q^pi(s, a) of the...
- Earth Mover's Distancev6Earth Mover's Distance (EMD), also known as the Wasserstein-1 distance, Kantorovich-Rubinstein metric, or Mallows's distance, is a measure of dissimilarity...
- Recurrent Neural Networkv10See also: Machine learning terms A recurrent neural network (RNN) is a class of artificial neural network designed to process sequential data by maintaining an...
- Statistical learning theoryv4Statistical learning theory (SLT) is the mathematical framework that explains when and why machine learning algorithms generalize from a finite training sample...
- Voltage Parkv3Voltage Park is a United States cloud computing company that operates a fleet of NVIDIA H100 graphics processing units for artificial intelligence training and...
- Open-Sorav3Open-Sora is an open-source text-to-video diffusion project initiated in March 2024 by Singapore-based startup HPC-AI Tech (the team behind the Colossal-AI...
- Loss Surfacev4The loss surface (also called the loss landscape, the error surface, or the objective function surface) is the geometric representation of a loss function as a...
- LLaMA-Factoryv3LLaMA-Factory is an open-source unified framework for the efficient fine-tuning of large language models (LLMs) and vision-language models (VLMs). It...
- GitHub Copilot Xv6GitHub Copilot X was the umbrella name GitHub used in 2023 for its vision of an artificial intelligence assistant that would reach across the entire developer...
- MiniCPM-Vv3MiniCPM-V is a family of open-weights multimodal large language models developed by the OpenBMB lab at Tsinghua University's Natural Language Processing group...
- GPU clusterv4A GPU cluster is a tightly coupled collection of servers, each populated with multiple graphics processing units, interconnected by high-bandwidth low-latency...
- Deepgram Nova-3v3Deepgram Nova-3 is the third-generation automatic speech recognition (ASR) model developed by Deepgram, a San Francisco-based voice AI company. The model...
- Blueprint for an AI Bill of Rightsv3The Blueprint for an AI Bill of Rights is a non-binding policy framework released by the White House Office of Science and Technology Policy (OSTP) on October...
- Agentic misalignmentv2Agentic misalignment is a term coined by Anthropic in a June 2025 research release for cases in which a goal-directed large language model (LLM), placed in an...
- Fully Connected Layerv5A fully connected layer (also called a dense layer or linear layer) is a layer in an artificial neural network in which every input value connects to every...
- Gaussian Processv3A Gaussian process (GP) is a probabilistic machine learning model defined as a collection of random variables, any finite number of which have a joint Gaussian...
- Importance samplingv4Importance sampling (often abbreviated IS) is a Monte Carlo method for estimating the expectation of a function under a target probability distribution by...
- Weight Decayv4Weight decay is a regularization technique used in training neural networks that shrinks every parameter toward zero by a small fraction on each update step,...
- DeepSpeedv9DeepSpeed is an open-source deep learning optimization library developed by Microsoft that makes distributed training and inference of large models efficient,...
- One-vs.-allv5See also: Machine learning terms One-vs.-all (OvA), also known as one-vs.-rest (OvR) or one-against-all, is a strategy for turning a multi-class classification...
- Decision Forestv5See also: Random Forest, Decision Tree, Ensemble Learning A decision forest is a family of ensemble learning methods in machine learning that combine many...
- Attribute samplingv4See also: Machine learning terms Attribute sampling is a randomization technique in which a decision tree considers only a small, randomly drawn subset of the...
- Activation Functionv7An activation function is a nonlinear mathematical function applied to the output of each neuron in an artificial neural network, and it is what gives the...
- Pudu FlashBot Armv5--- General information Pudu Robotics Country of origin 2025 Status Available (Q1 2026 onward) Website The Pudu FlashBot Arm (stylized as PUDU...
- MetaGPTv3MetaGPT is an open-source multi-agent framework that organizes large language model agents into a simulated software-development company, with role-specialized...
- LimX Oliv8--- General information LimX Dynamics Country of origin 2025 Status RMB 158,000 ($21,800 USD) Height 55 kg (121 lbs) Degrees of freedom 5 km/h...
