https://www.linearalgebraforprogrammers.com/ An interactive, open-source guide to linear algebra for programmers, with runnable Python and visuals you can interact with. https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/3_block_scaling_in_mxfp4_and_nvfp4 https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/3_block_scaling_in_mxfp4_and_nvfp4 How MXFP4 and NVFP4 multiply 4-bit E2M1 values by a shared block scale, and NVFP4 by a tensor scale too, to get the stored numbers. Thu, 05 Nov 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/4_adding_two_floats https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/4_adding_two_floats How two floats are added, step by step: align the exponents, add the bits, normalize, round, with a made-up 3-bit mantissa. Thu, 05 Nov 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/5_why_fp_addition_is_not_associative https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/5_why_fp_addition_is_not_associative Each floating point addition rounds its exact sum, so the order of additions can change the result. The three ways a sum loses bits, shown in FP8 E5M2. Thu, 05 Nov 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/6_exponentiation_by_squaring https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/6_exponentiation_by_squaring The binary digits of n say which squares of x multiply to x to the n, so x to the n takes about log2(n) multiplications instead of n − 1. Shown step by step, then used for Fibonacci numbers. Thu, 05 Nov 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/1_binary_representations_of_integers https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/1_binary_representations_of_integers uint, two's complement, one's complement and biased integers seen as one formula, a dot product of bits and weights plus a bias. Mon, 02 Nov 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/2_anatomy_of_floating_point https://www.linearalgebraforprogrammers.com/series/integer_and_floating_point_representations/2_anatomy_of_floating_point How a floating point value is calculated from its bits, how inf and NaN are represented, what normal and subnormal modes are, and why they matter. Mon, 02 Nov 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/continuous_normalizing_flow https://www.linearalgebraforprogrammers.com/blog/continuous_normalizing_flow A shorter proof of the instantaneous change of variables theorem from the Neural ODEs paper, and why it is the continuity equation in disguise. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/flag_decomposition https://www.linearalgebraforprogrammers.com/blog/flag_decomposition The key idea of the flag decomposition paper by Mankovich et al., a QR-like factorization that keeps the hierarchy of nested data subspaces. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/inverse_matrix_block https://www.linearalgebraforprogrammers.com/blog/inverse_matrix_block Given a matrix and its inverse, compute the inverse of a square block after removing k rows and columns, using the Schur complement. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/inverse_row_col_removed https://www.linearalgebraforprogrammers.com/blog/inverse_row_col_removed Update a known matrix inverse after removing one row and column, a trick from the Optimal Brain Compression paper, and when it beats re-inverting. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/jacobians_for_nf https://www.linearalgebraforprogrammers.com/blog/jacobians_for_nf Autoregressive, affine and channel-wise transformations used in normalizing flows, and why their Jacobian determinants are easy to compute. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/khatri_rao_product https://www.linearalgebraforprogrammers.com/blog/khatri_rao_product The Khatri-Rao product as a column-wise Kronecker product, its row-wise variant, and three properties with proofs, including the Hadamard Gram matrix. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/kronecker_product https://www.linearalgebraforprogrammers.com/blog/kronecker_product A visual guide to the Kronecker product, its lesser-known properties, its link to bilinear forms and products like CXBᵀ, and an unusual application. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/matmul_animations https://www.linearalgebraforprogrammers.com/blog/matmul_animations Animations of three ways to read a matrix product: weighted sums of columns or rows, dot products, and a sum of outer products. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/matrix_transpose_product https://www.linearalgebraforprogrammers.com/blog/matrix_transpose_product How the eigenvalues and eigenvectors of AAᵀ and AᵀA relate, derived from the SVD of A, with applications. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/minimize_norm_of_vector_sum https://www.linearalgebraforprogrammers.com/blog/minimize_norm_of_vector_sum Find the x that minimizes the squared norm of a + Mx, with a visual example and a step-by-step solution. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/new_proof_gptq https://www.linearalgebraforprogrammers.com/blog/new_proof_gptq A different, more geometric derivation of the GPTQ and Optimal Brain Compression weight update for post-training quantization, shown to be equivalent. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/permutation_matrices https://www.linearalgebraforprogrammers.com/blog/permutation_matrices How permutation matrices shuffle rows and columns, why their inverse is their transpose, and how they form a group, with animations. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/proof_fibonacci_sequence_in_stick_lengths https://www.linearalgebraforprogrammers.com/blog/proof_fibonacci_sequence_in_stick_lengths A linear algebra proof that the chance no three of n random sticks form a triangle is the product of 1/Fᵢ over the Fibonacci numbers. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/blog/sinq https://www.linearalgebraforprogrammers.com/blog/sinq The key idea in the SINQ paper, which normalizes rows and columns of a weight matrix in Sinkhorn-like iterations before quantization. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/0_intro https://www.linearalgebraforprogrammers.com/la/0_intro What this free eBook covers, who it is for, and how its runnable Python and interactive canvas help programmers learn linear algebra. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/1_weighted_means https://www.linearalgebraforprogrammers.com/la/1_weighted_means Linear algebra starts with weighted sums. See how scaling and adding numbers shows up in indexing, expected values, rotation and polynomials. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/2_spacewalk https://www.linearalgebraforprogrammers.com/la/2_spacewalk An array of numbers becomes a vector with a length and a direction. Scaling and adding vectors leads to matrix-vector multiplication. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/3_mat_vec_mul https://www.linearalgebraforprogrammers.com/la/3_mat_vec_mul See a matrix as a function that maps vectors to vectors, turning lines into lines and circles into ellipses, with interactive visuals. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/4_rank_and_reversibility https://www.linearalgebraforprogrammers.com/la/4_rank_and_reversibility What the rank of a matrix means, how it relates to linearly independent vectors, and when a matrix can be inverted, including the 2-by-2 inverse. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/5_barycentric_coordinates https://www.linearalgebraforprogrammers.com/la/5_barycentric_coordinates Use barycentric coordinates and a matrix inverse to check whether a point lies inside a triangle, then extend the idea to polygons. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/6_matrix_matrix_multiplication https://www.linearalgebraforprogrammers.com/la/6_matrix_matrix_multiplication How to multiply two matrices, why it is associative but usually not commutative, and how to see it as composing and factorizing transformations. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/7_linear_transformations https://www.linearalgebraforprogrammers.com/la/7_linear_transformations Reach the same point with two different sets of vectors, see matrices as encoders and decoders, and learn how a change of basis works. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/8_polynomial_curves https://www.linearalgebraforprogrammers.com/la/8_polynomial_curves A polynomial is defined by its coefficients, so it is a vector. See how adding and scaling polynomials looks as curves. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/9_bezier_curves https://www.linearalgebraforprogrammers.com/la/9_bezier_curves Build quadratic and cubic Bezier curves from polynomial vectors, and see why they stay inside the convex hull of their control points, in 2D and 3D. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/10_dot_product https://www.linearalgebraforprogrammers.com/la/10_dot_product The dot product of two vectors, how it encodes the angle between them, when it is zero, and its role in projections and matrix-vector multiplication. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/11_matrix_transpose https://www.linearalgebraforprogrammers.com/la/11_matrix_transpose What the transpose of a matrix is, how rows and columns each explain matrix multiplication, and how to visualize the row space and column space. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/12_rotations https://www.linearalgebraforprogrammers.com/la/12_rotations Derive the 2D rotation matrix, see why its columns are perpendicular unit vectors, and why the transpose of a rotation matrix is its inverse. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/13_normal_vectors https://www.linearalgebraforprogrammers.com/la/13_normal_vectors Why a normal vector stops being perpendicular when you transform it with the same matrix, and which matrix keeps it perpendicular. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/14_eigenvectors_eigenvalues https://www.linearalgebraforprogrammers.com/la/14_eigenvectors_eigenvalues What happens when a matrix maps a vector to the same direction. The algebra and geometry of Ax = λx, with interactive 2-by-2 examples. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/15_eigendecomposition https://www.linearalgebraforprogrammers.com/la/15_eigendecomposition How a matrix with n distinct eigenvectors factors as VDV⁻¹, what that means geometrically, and how it makes powers of a matrix easy. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/16_eigenvectors_symmetric_matrices https://www.linearalgebraforprogrammers.com/la/16_eigenvectors_symmetric_matrices Why a symmetric matrix has real eigenvalues and perpendicular eigenvectors, with a visual of eigenvectors as a matrix becomes symmetric. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/17_powers_of_a_matrix https://www.linearalgebraforprogrammers.com/la/17_powers_of_a_matrix Power iteration finds the largest eigenvalue and its eigenvector by applying a matrix again and again. See when and why it converges. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/18_fibonacci_numbers https://www.linearalgebraforprogrammers.com/la/18_fibonacci_numbers Write the Fibonacci recurrence as a matrix, derive Binet's closed-form formula from its eigenvalues, and generalize to linear recurrences. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/19_markov_matrices https://www.linearalgebraforprogrammers.com/la/19_markov_matrices Model a Markov process as a matrix, find its stationary distribution, and see what the eigenvalues and eigenvectors of a Markov matrix tell you. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/20_projection_matrix https://www.linearalgebraforprogrammers.com/la/20_projection_matrix Project a vector onto a line, a plane and any subspace, derive the projection matrix, and see why it is idempotent. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/21_symmetric_matrix_vector_product https://www.linearalgebraforprogrammers.com/la/21_symmetric_matrix_vector_product Multiplying a vector by a symmetric matrix is a weighted sum of projections onto its eigenvectors, shown with the eigendecomposition and a visual. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/22_psd_matrices https://www.linearalgebraforprogrammers.com/la/22_psd_matrices Symmetric positive semi-definite matrices have non-negative eigenvalues. Their properties, the meaning of xᵀMx, and uses in optimization. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/23_svd https://www.linearalgebraforprogrammers.com/la/23_svd SVD answers one question: can a matrix map perpendicular vectors to perpendicular vectors? Derive and visualize it for matrices of any shape. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/24_dimension_reduction https://www.linearalgebraforprogrammers.com/la/24_dimension_reduction Store n-dimensional data points with fewer dimensions by taking the SVD of the covariance matrix, worked through on a small dataset. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/la/25_what_next https://www.linearalgebraforprogrammers.com/la/25_what_next Topics this eBook did not cover, such as solving linear equations and LU decomposition, and directions to learn and build more. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/0_linear_operator https://www.linearalgebraforprogrammers.com/series/matrix_calculus/0_linear_operator Matrix calculus starts with a small change in output for a small change in input. Linear operators, the product and chain rules, and worked examples. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/1_gradient https://www.linearalgebraforprogrammers.com/series/matrix_calculus/1_gradient The gradient of a scalar-valued function of a vector or matrix, the Frobenius inner product, trace identities, and gradients of the norm and determinant. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/2_jacobian https://www.linearalgebraforprogrammers.com/series/matrix_calculus/2_jacobian The Jacobian of a vector-valued function, the Jacobian of a composition of functions, the Jacobian determinant and the Jacobian inverse. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/3_key_derivatives https://www.linearalgebraforprogrammers.com/series/matrix_calculus/3_key_derivatives Derivatives of singular values, elementwise functions, softmax and row-wise vector functions, each checked in Python. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/4_function_as_dag https://www.linearalgebraforprogrammers.com/series/matrix_calculus/4_function_as_dag View a function as a directed acyclic graph (DAG) with Jacobians on its edges, the idea behind how autodiff systems compute gradients. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/5_autodiff_on_dag https://www.linearalgebraforprogrammers.com/series/matrix_calculus/5_autodiff_on_dag The chain rule on a computation graph, forward and reverse ways to compute gradients, and why full Jacobians are overkill for autodiff. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/6_vjp_and_jvp https://www.linearalgebraforprogrammers.com/series/matrix_calculus/6_vjp_and_jvp Vector-Jacobian products (VJP) and Jacobian-vector products (JVP), directional derivatives, and forward-mode autodiff, with Python code. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/matrix_calculus/7_hessian https://www.linearalgebraforprogrammers.com/series/matrix_calculus/7_hessian Compute a full Jacobian matrix with batching, then the Hessian, why it is symmetric, its bilinear form, and how to calculate it in PyTorch. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/0_permutation_cycles https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/0_permutation_cycles An overview of the series from permutations to Sinkhorn distances, starting with how to decompose a permutation into cycles. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/1_permutation_spectra https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/1_permutation_spectra The eigenvalues and eigenvectors of a permutation matrix come from its cycles, with each cycle adding roots of unity. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/2_circulant_matrix https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/2_circulant_matrix Circulant matrices, their eigenvalues and eigenvectors, and their link to circular convolution and the convolution theorem. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/3_birkhoff_polytope https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/3_birkhoff_polytope The Birkhoff polytope is the convex hull of all n-by-n permutation matrices. Its edges, faces, and the points inside it. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/4_doubly_stochastic_matrices https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/4_doubly_stochastic_matrices Properties of doubly stochastic matrices, majorization, and Muirhead's inequality, with Python checks. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/5_birkhoff_von_neumann https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/5_birkhoff_von_neumann An algorithm that writes any doubly stochastic matrix as a convex sum of permutation matrices, based on counting its zeros, with an example. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/6_matrix_scaling https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/6_matrix_scaling Sinkhorn's theorem that any positive matrix scales to a doubly stochastic one, proof of existence and uniqueness, and Sinkhorn iteration in Python. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/7_sinkhorn https://www.linearalgebraforprogrammers.com/series/permutation_sinkhorn/7_sinkhorn A derivation of dual Sinkhorn distances for entropy-regularized optimal transport, using Lagrange multipliers. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/rotations/0_intro https://www.linearalgebraforprogrammers.com/series/rotations/0_intro Start of a series on rotations, from 2D rotation matrices to quaternions and Lie algebra. Why rotation is linear and preserves lengths and angles. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/rotations/2d_rotations https://www.linearalgebraforprogrammers.com/series/rotations/2d_rotations Describe a 2D rotation with a rotation matrix or with complex numbers and the complex exponential. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/rotations/3d_rotation_matrices https://www.linearalgebraforprogrammers.com/series/rotations/3d_rotation_matrices Why 3D rotations do not commute, intrinsic and extrinsic rotations, and Euler angles, with Python examples using scipy. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/rotations/geometric_algebra https://www.linearalgebraforprogrammers.com/series/rotations/geometric_algebra The geometric product and bivectors, and how geometric algebra describes reflections and rotations in 2D and 3D, with the Python clifford library. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/rotations/lie_algebra https://www.linearalgebraforprogrammers.com/series/rotations/lie_algebra Rotations form the Lie group SO(n). Derive its Lie algebra of skew-symmetric matrices for 2D and 3D rotations, with Python code. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/rotations/quaternions https://www.linearalgebraforprogrammers.com/series/rotations/quaternions Quaternion algebra, how to rotate a vector with a quaternion, visuals of quaternion rotations, and the matrix form of a quaternion. Sat, 24 Oct 2026 00:00:00 GMT https://www.linearalgebraforprogrammers.com/series/rotations/rotation_as_reflections https://www.linearalgebraforprogrammers.com/series/rotations/rotation_as_reflections Reflection matrices around lines and hyperplanes, and how any rotation is two reflections, shown algebraically and in 2D and 3D. Sat, 24 Oct 2026 00:00:00 GMT