# Linear AlgebraDot Products

Consider a shop inventory which lists unit prices and quantities for each of the products they carry. For example, if the store has 32 small storage boxes at $4.99 each, 18 medium-sized boxes at$7.99 each, and 14 large boxes at $9.99 each, then the inventory's price vector and quantity vector are The total value of the boxes in stock is This operation—multiplying two vectors' entries in pairs and summing—arises often in applications of linear algebra and is also foundational in the theory of linear algebra. Definition The dot product of two vectors in is defined by Example If and then + + + = 100. One of the most algebraically useful features of the dot product is its linearity (which may be checked using the definition): The dot product also has two fundamental connections to geometry. The first is the identity for all vectors . Let's see how this identity can work in conjunction with linearity of the dot product. Exercise Show that for all vectors and in . Solution. Using linearity of the dot product, we get as required. The second connection between geometry and the dot product pertains to angle. If is the angle between two vectors and (when they are situated so that their tails coincide), then It follows that if and only if and meet at a angle. We say that two vectors and which satisfy are orthogonal. Exercise In natural language processing, one basic way to compare a finite number of text documents is to use vectorized word counts. Suppose the documents have a combined total of distinct words, which are arranged in some order. Each document is then associated with a vector of length whose$i$th entry indicates the number of times the th word occurs in the associated document. One way to measure similarity between two documents is to take the dot product of the associated unit vectors: If two documents and have associated vectors and respectively, their similarity is defined by By the dot product cosine formula, we have for any two documents and Documents with no words in common are associated with orthogonal vectors and thus have similarity. If two documents have similarity their associated vectors are scalar multiples of each other, meaning that they have the same words and that the words appear in the same proportions. The vectorized word count similarity between the sentences "The rain in Spain falls mainly in the plain" "The plain lane in Spain is mainly a pain" is . Solution. Listing the words in the order the, in, rain, Spain, falls, mainly, plain, lane, pain, is, a, the two vectorized word counts are and . Substituting into the definition of , we get a similarity of approximately 0.647. Exercise Let be a list of orthogonal non-zero vectors, that is, for all suppose that whenever . Show that this list is linearly independent. Solution. Suppose, for the sake of contradiction, that the vectors are linearly . Then one of the vectors can be written as a linear combination of the others. Suppose is such a vector. Then there exists a list of weights such that Then Since , this implies that is zero. Repeating this for all vectors we see that . Thus is also zero (since it's a linear combination of the other vectors, with all zero weights), and that contradicts the fact that . The same reasoning tells us that none of the vectors in the list can be equal to a linear combination of the others. Therefore the vectors must be linearly . The following exercise illustrates another way of calculating matrix products. We will call it the matrix product dot formula: Exercise Let and Consider the matrix whose$(i,j)$th entry is equal to the dot product of the$i$th row of and the$j\$th column of . Show that , and use this fact to work out the full product .

Solution. By the product column rule, the first column of is , where is the first of . Therefore, the first entry of that column is . This is the dot product of the first row of and the first column of . The same reasoning applies to the other entries.

Calculating all eight such dot products, we find that

## Block multiplication

A block matrix is a matrix defined using smaller matrices which are called blocks. For example, suppose that

Then the block matrix defined in terms of the blocks , , , and is

The advantage of writing a matrix in block form is that we can formally carry out the matrix multiplication dot formula, treating the blocks as matrix entries, and we get the correct result (in block form). For example,

if and are block matrices with blocks , , , , , , , blocks . We call this the block matrix product formula.

Exercise
Verify the matrix product block formula above with

Solution. We have

while

So the block matrix product formula checks out.

Exercise
Show that if is a matrix whose columns are and is a matrix whose rows are , then

Solution. This follows directly from the block matrix product formula by writing is a block matrix with its columns as blocks and with its rows as blocks.

Bruno