Inner Product

.

Inner Product (.) is a dyadic operator that produces a dyadic function when applied with two dyadic functions. It's a generalisation of the matrix product, allowing not just addition-multiplication, but any dyadic functions given as operands.

Description

For each rank-1 cell in the arguments, inner product applies ⍵⍵ between the two and then applies ⍺⍺⌿ to the results of those invocations. If the arguments themselves are simply vectors, there is only one rank-1 cell in each argument, so this results in the following application pattern:

      1 2 3 4 F.G 5 6 7 8

      (1 G 5) F (2 G 6) F (3 G 7) F (4 G 8)

The second line is an illustration of how the first will be evaluated. Note that this is precisely the vector dot product when used as +.×. (This simple invocation with vector arguments will be referred to as the "vector inner product" below, but it is just a simple case of the general inner product.)

For matrix arguments, there may be more than one rank-1 cell. Considering the case of rank-2 matrices as arguments, if there are N rows in ⍺ and M columns in ⍵, the result will be a matrix with N rows and M columns. Each row of the resulting matrix will correspond to the elements of that same row in ⍺ being paired up with elements in a column of ⍵. Likewise, each column of the resulting matrix will correspond to the elements of that same column of ⍵ being paired up with elements in a row of ⍺. Important: This means that the inner product will be applied for each row of ⍺ but each column of ⍵!

In practice, this means that the upper left element of the resulting matrix will correspond to performing the vector inner product on the first row of ⍺ and the first column of ⍵, the upper right element will correspond to performing the vector inner product on the first row of ⍺ and the last column of ⍵, the lower right element will correspond to the vector inner product on the last row of ⍺ and the last column of ⍵, and so on and so on. Pictorially, then, for a 2x2 result we can represent the resulting matrix as:

┌──────────────────────────────┬──────────────────────────────┐
│(Row 1 of ⍺) F.G (Col. 1 of ⍵)│(Row 1 of ⍺) F.G (Col. 2 of ⍵)│
├──────────────────────────────┼──────────────────────────────┤
│(Row 2 of ⍺) F.G (Col. 1 of ⍵)│(Row 2 of ⍺) F.G (Col. 2 of ⍵)│
└──────────────────────────────┴──────────────────────────────┘

Note how the columns of ⍵ align with the columns of the matrix, and the rows of ⍺ align with the rows of the matrix.

The concept readily generalizes to matrices of higher rank.

Examples

      x ← 1 2 3
      y ← 4 5 6
      x ,.(⊂,) y ⍝ visualizing of pairing
┌─────────────┐
│┌───┬───┬───┐│
││1 4│2 5│3 6││
│└───┴───┴───┘│
└─────────────┘
      x {⊂⍺,'+',⍵}.{⊂⍺,'×',⍵} y ⍝ visualizing function application in matrix multiplication
┌───────────────────────────┐
│┌─────────────────────────┐│
││┌─────┬─┬───────────────┐││
│││1 × 4│+│┌─────┬─┬─────┐│││
│││     │ ││2 × 5│+│3 × 6││││
│││     │ │└─────┴─┴─────┘│││
││└─────┴─┴───────────────┘││
│└─────────────────────────┘│
└───────────────────────────┘
      x+.×y ⍝ matrix multiplication
32

The shapes of the arguments must be compatible with each other: The last axis of the left argument must have the same length as the first axis of the right argument, or formally, for X f.g Y it must be that (¯1↑⍴X)≡(1↑⍴Y). Although this rule differs from conformability, the arguments may also be subject to scalar or singleton extension. The shape of the result is (¯1↓⍴X),(1↓⍴Y).

For example, when applying inner product on two matrices, the number of columns in the left array must match with number of rows in the right array, otherwise we will get an error.

      ⎕  ← x ← 2 3⍴⍳10
1 2 3
4 5 6
      ⎕ ← y ← 4 2⍴⍳10
1 2
3 4
5 6
7 8
      x+.×y 
LENGTH ERROR
      x+.×y
        ∧
      ⎕ ← y ← 3 2⍴⍳10 ⍝ reshape y to be compatible with x
      x+.×y
22 28
49 64

History

Inner product appeared in early Iverson Notation as $_{g}^{f}$ and applied even to non-scalar functions, like Compress, Iverson bringing:

${\begin{aligned}{\text{For example, if}}\\{\boldsymbol {A}}&={\begin{pmatrix}1&3&2&0\\2&1&0&1\\4&0&0&2\\\end{pmatrix}}\qquad {\text{and}}\qquad {\boldsymbol {B}}={\begin{pmatrix}4&1\\0&3\\0&2\\2&0\\\end{pmatrix}}\\{\text{then}}\qquad {\boldsymbol {A}}\;_{\times }^{+}\,{\boldsymbol {B}}&={\begin{pmatrix}4&14\\10&5\\20&4\\\end{pmatrix}},\quad {\boldsymbol {A}}\;_{=}^{\land }\,{\boldsymbol {B}}={\begin{pmatrix}0&1\\0&0\\1&0\\\end{pmatrix}}{\text{,}}\\{\boldsymbol {A}}\;_{\neq }^{\lor }\;{\boldsymbol {B}}&={\begin{pmatrix}1&0\\1&1\\0&1\\\end{pmatrix}},\qquad {\text{and}}\qquad ({\boldsymbol {A}}\neq 0)\;_{\,/}^{+}\,{\boldsymbol {B}}={\begin{pmatrix}4&6\\6&4\\6&1\\\end{pmatrix}}{\text{.}}\end{aligned}}$

When the inner product notation was linearised (made to fit on a single line of code) the glyph . was chosed to denote what was previously indicated by positioning the two operands vertically aligned. Thus, the above correspond to the following modern APL:

⍝ For example, if
      A←3 4⍴1 3 2 0 2 1 0 1 4 0 0 2
      B←4 2⍴4 1 0 3 0 2 2 0
⍝ then
      A +.× B
 4 14
10  5
20  4
      A ∧.= B
0 1
0 0
1 0
      A ∨.≠ B
1 0
1 1
0 1
      (A ≠ 0) +./ B
4 6
6 4
6 1

Note that some dialects implement Compress (/) as a monadic operator rather than as a function, which means it cannot be an operand in the inner product. Instead, a cover function is necessary:

∇z←a Compress b
 z←a/b
∇

Differences between dialects

Implementations differ on the exact behaviour of inner product when the right operand is not a scalar function. Page 121 of the ISO/IEC 13751:2001 standard specifies that X f.g Y is equivalent to

f/¨ (⊂[⍴⍴x]x)∘.g ⊂[1]y

(assuming index origin 1). This is indeed what APL2, APLX, APL+Win, and ngn/apl follow, while Dyalog APL, NARS2000 and GNU APL differ as described by Roger Hui:

The following dop models inner product in Dyalog APL, with caveats. If you find a case where f.g differs from f IP g, not covered by the caveats, I'd be interested.

IP←{                                
  assert((⊃⌽⍴⍺)≡≢⍵)∨(1=×/⍴⍺)∨1=×/⍴⍵:
  ⊃⍤0 ⊢ (↓⍺) ∘.(⍺⍺/⍵⍵¨) ↓(¯1⌽⍳⍴⍴⍵)⍉⍵    
}

assert←{⍺←'assertion failure' ⋄ 0∊⍵:⍺ ⎕SIGNAL 8 ⋄ shy←0}

(Explanation: What's with the ⊃⍤0 in IP? It's because ∘.f has an implicit each, applying ⊂ to each item of its result. But the ⍺⍺/ in (⍺⍺/⍵⍵¨) also has an implicit each. So the ⊃⍤0 gets rid of one of those encloses.)

Caveats:

• You can not use the hybrid / directly as an operand as it runs afoul of the parser in weird and wonderful ways. Instead, you have to use {⍺/⍵}. The same goes for \ and {⍺\⍵} I guess.

• It differs from ISO/IEC 13751:2001(E) in using ⍵⍵¨ instead of just ⍵⍵ in the central key expression (i.e. (⍺⍺/⍵⍵¨) instead of (⍺⍺/⍵⍵)). So does the primitive f.g.

• It differs from ISO/IEC 13751:2001(E) in doing full-blown single extension instead of just scalar and 1-element vector extension (as in APL2). So does the primitive f.g. e.g.

     (3 4⍴5)+.×1 1 1 1⍴6  ⍝ works in Dyalog, not in ISO or APL2

• It differs from NARS2000 or APL\360 in not permitting unit axis extension. So does the primitive f.g. e.g.

     (3 4⍴5)+.×1 5⍴6  ⍝ works in NARS2000 or APL\360, not in Dyalog APL

The

⊃⍤0⊢(↓⍺)∘.(⍺⍺/⍵⍵¨)↓(¯1⌽⍳⍴⍴⍵)⍉⍵

line of IP above can be rewritten as

⍺(⍺⍺⌿⍵⍵¨⍤¯1)⍤1 99⊢⍵

which uses the more efficient item-at-a-time algorithm (rather than row-by-column). The ISO/IEC 13751:2001(E) inner product, conversely, can only be calculated row-by-column, as computing the results one item (of the right argument) at a time relies on each application of the right operand being done between two scalars and producing a scalar result—that is, on the Each operator being applied to the right operand.

Some implementations extend the inner product by implementing Iverson's monadic variant, which takes a single argument and performs the operation of computing the alternant, as modelled by dfns.alt.

External links

Documentation

• Dyalog
• APLX
• J Dictionary, NuVoc

Publications

• Inner Product: An Old/New Problem by Roger Hui

Discussion of differences between dialects

• Dyalog / APL2000 discrepancy (Google Groups)
• multiple inner product (GNU APL mailing list)
• an other inner product ,., bug (GNU APL mailing list)