The Math for Understanding AI

1.Vectors: Anything as a List of Numbers

AI handles text, photos, and even tastes by turning them into lists of numbers. Such a list of numbers is called a vector. If you express movie tastes as preferences for (action, romance, comedy), you can write Jisoo as [5, 1, 3], Minho as [4, 2, 3], and Seoyeon as [1, 5, 2]. A language model turns each word or word piece (token) into a vector of hundreds to thousands of numbers, but the principle is the same as in this three-number example.

Given two vectors, you can calculate "how similar they are." The distance between two points from Lesson 6 is one way; the ways used more often in AI are the dot product and cosine similarity.

2.Dot Products and Cosine Similarity

The dot product multiplies the numbers in the same positions of two vectors and adds them all up. The more two people like the same items, the more large numbers get multiplied together, and the bigger the dot product. However, the dot product automatically grows for vectors whose numbers are large overall, so cosine similarity divides by the vectors' lengths (magnitudes) to compare direction alone.

Cosine similarity takes values from −1 to 1. Close to 1 means the same direction (similar tastes), 0 means unrelated directions (at right angles), and −1 means exact opposites. In search and recommendations, "finding documents with similar meaning" usually means calculating this value and picking the largest ones.

3.Matrix Multiplication: Many Weighted Sums at Once

What a single neuron in a neural network does is simple. It multiplies each input by a weight and adds them up (a weighted sum), adds a bias, and then transforms the result once by a fixed rule. One of the most common rules is to turn negative values into 0 and leave positive values as they are. With inputs [2, 3], weights [0.5, −1], and a bias of 1, you get 0.5×2 + (−1)×3 + 1 = −1, which is negative, so the neuron outputs 0.

With several neurons, you have to compute several weighted sums. A matrix is the way to write them all at once. Each row of the matrix holds one neuron's weights, and multiplying a matrix by a vector means "taking the dot product of each row with the input vector." Most of the computation in a giant AI model is this kind of matrix multiplication, which is why specialized chips that do matrix multiplication fast have become so important.

4.Loss Functions: How Wrong, in a Number

For a model to learn, it first has to measure "how wrong it is" with a single number. The function that produces this number is called a loss function. A common one for models that predict numbers is mean squared error (MSE): the average of the squared differences (errors) between predictions and correct answers. Its form is almost the same as the variance from Lesson 8.

There are two reasons for squaring. One is to count errors as "wrong" whether they are positive or negative; the other is to penalize big mistakes more heavily. An error of 2 becomes 4, and an error of 4 becomes 16.

5.Slopes and Gradient Descent

Which way should the weights be adjusted to reduce the loss? Picture someone climbing down a foggy mountain to the valley. They can't see far, but they can feel the slope under their feet. They take a step downhill, feel the slope again, and take another step. "The slope at a single point" from Lesson 4 is exactly this slope underfoot, and descending this way is called gradient descent.

As the simplest example, take a loss of L(w) = (w − 3)². The loss is smallest, 0, when w = 3. The slope of this function is 2(w − 3). If w is less than 3, the slope is negative, so w should be increased; if it is greater, the slope is positive, so w should be decreased. So you update with "new w = w − learning rate × slope." The learning rate is the size of one step.

A real model may have billions of weights, but it does the same thing. For each weight, it calculates "how much the loss changes if this weight changes a little" (the slope, or gradient), and repeats small adjustments in the direction that reduces the loss countless times.