Dyalog APL
- "Dyalog" redirects here. For the company, see Dyalog Ltd.
Dyalog APL, or simply Dyalog, is a nested APL based on NARS and APL2, first released by British company Dyadic Systems Ltd. (now Dyalog Ltd.) in 1983 for the Zilog Z8000 processor. (The name Dyalog is a portmanteau of Dyadic and Zilog). Continuously developed since, Dyalog has added support for many programming paradigms including object-oriented programming based on .NET, Lisp-style lexically scoped anonymous functions (dfns), and leading axis and tacit programming support based on J. It supports several platforms and interoperability between them, and interfaces with other languages and runtimes including native shared libraries, .NET, the JVM, R, and Python.
Although it initially received very little commercial interest, Dyalog has steadily grown in prominence and in the 2010s has been the basis of several new APL dialects including ngn/apl, APL\iv, and dzaima/APL. Even in APLs not derived from Dyalog such as GNU APL and NARS2000, dfn-style function syntax has become common, and Dyalog has also popularised SHARP APL and J innovations such as the Rank operator and trains among nested APLs.
History
- See also: Dyalog Ltd.
Work on Dyalog was begun in 1981 by APL consulting company Dyadic Systems, which by that time had grown to support about 15 employees. In partnership with Zilog UK, Dyadic developed an interpreter using the C programming language for the Zilog Z8000's UNIX operating system—both obscure technologies at the time. Dyadic employees John Scholes and Geoff Streeter worked full-time on the implementation, while David Crossley managed its development as a part-time role. Initially aiming to produce something like SHARP APL, they eventually chose a nested model rather than adding boxes to the flat array model like SHARP, and drew most design decisions from STSC's experimental NARS dialect and the material available at the time regarding IBM's plans for APL2.[1]
Released at APL83, Dyalog sold very few copies due to the lack of interest in either Unix or the nested array model. Subsequent sales were also limited, with only a single licence sold through Zilog partly due to the Z8000's limited popularity. Supported by Dyadic's APL consulting and later by sales of Unix hardware, Scholes and Streeter continued work on Dyalog by porting it to a wide variety of Unix systems in response to requests from users; in 1995, Scholes was awarded the Iverson Award jointly with Peter Donnelly for his work on the Dyalog interpreter. Although it continued to run significant losses every year, Dyalog slowly acquired users during the 1980s, including current client SimCorp. In 1991, Dyadic hired John Daintree to begin work on the <source lang=apl inline>⎕WC</syntaxhighlight> graphical user interface for Microsoft Windows; Dyalog for Windows debuted at APL92 and quickly become Dyalog's main platform.[1]
In 1996, John Scholes introduced a new form of functional definition to Dyalog based on his studies of the functional programming language Scheme, which he called dfns, for "direct functions". Another major addition to the language began in 2000, when John Daintree was invited to participate in the design of Microsoft's .NET. Based on this work, and the namespaces which he had added to the language in 1994 to deliver a Windows GUI API and an interface to COM/OLE, Daintree developed an object model for Dyalog, based on concepts from C# and Visual Basic.
In 2005, Dyadic transferred management to CEO Gitte Christensen and CTO Morten Kromberg, and sold ownership of the company to a group of investors including their company Insight Systems. With this arrangement, the development team at Dyalog grew rapidly. Dfns were promoted as "mainstream" APL features, with the result that many newer APLs only support dfns, abandoning traditional defined functions completely. Daintree's work on objects resulted in the addition of full object oriented programming capabilities to Dyalog APL in 2006. The influence of the Microsoft's .NET Framework also led to a project to create a new language called APL#, prototypes of which were released in 2010. The project was abandoned in 2012 when Microsoft deprecated Silverlight.
In addition to functional and object oriented language features, Dyalog APL folded concepts and features from the SHARP APL and J side of the APL family tree back into what fundamentally remains a dialect of APL2. In 2011 the company hired Roger Hui, developer of J, and in 2016 it also hired J programmer and language implementer Marshall Lochbaum. Both developers improved performance of Dyalog's primitives on flat arrays, and brought concepts such as the Rank operator, trains, and composition operators, namely Atop and Over, from J to APL.
In the 2010s, Dyalog development began to focus on performance, which had been improved out of necessity in early releases but had not been a major focus. In 2010 Dyalog Ltd. hired Jay Foad, a compiler developer who initially created a bytecode compiler for APL and later improved performance in other ways, and served as CTO from 2016 until his departure in 2019. Hui and Lochbaum also improved performance of Dyalog's primitives on flat arrays.
The Dyalog interpreter has also incorporated significant components written in APL in the 2000s and 2010s. Dan Baronet, hired in 2006, introduced the SALT (Simple APL Library Toolkit) system to distribute APL code, and user commands based on it, in version 12.0 in 2008. Work on APL components of Dyalog has also been done by Brian Becker and Adám Brudzewsky. Brudzewsky, hired in 2015, has also driven the adoption of new functionality such as Nest and array notation in Dyalog.
Platforms
Besides for working out-of-the-box on Windows, macOS, and AIX, Dyalog APL runs on many Linuxes. However, to function, some require additional action beyond simple installation. As of 2019-05-15, the necessary actions for versions 16.0, 17.0, and 17.1 are:
Distribution | Action |
---|---|
Debian 8.2, 9.8, 9.9 | no pre-reqs |
Fedora 29, 30 | dnf install ncurses-compat-libs
|
Linux Mint 18.0, 19.0 | no pre-reqs |
openSUSE 15.0 | zypper install libncurses5
|
CentOS/RedHat 6 | not supported for 17.1 as glibc is at too low a level |
CentOS/RedHat 7 | no pre-reqs |
RedHat 8 | rpm -Uvh ncurses-compat-libs (note: RH8 would not install in a VM)
|
Ubuntu 16.04, 18.04 | no pre-reqs |
Ubuntu 19.04 | apt-get install libtinfo5
|
Versions
- Main article: Dyalog APL versions
Dyalog lists historical versions, along with release notes since 14.0, on its website. Its early history is recounted in more detail by Pete Donnelly in Dyalog APL: A Personal History (pdf).
Number | Year | Month | Features |
---|---|---|---|
1 | 1983 | April | (Zilog S8000 only) |
2 | 1984 | (Many more platforms) | |
3.0 | 1985 | (More platforms) Rectangular display of arrays | |
4.0 | 1986 | October | User-defined operators, Assignment for functions (including derived functions), <source lang=apl inline>⎕MONITOR</syntaxhighlight> |
5.0 | 1987 | April | Nested array editor |
5.1 | 1988 | April | (first version for DOS) User-defined input/output tables, <source lang=apl inline>⎕SM</syntaxhighlight> and <source lang=apl inline>⎕SR</syntaxhighlight>, windowed editor/tracer, interface to GSS/CGI |
5.2 | 1990 | January | Naked trace |
6.0 | 1990 | April | GUI IDE |
6.1 | 1990 | October | <source lang=apl inline>⎕ED</syntaxhighlight> |
6.2.1 | 1992 | July | (first version for Windows) <source lang=apl inline>⎕WC</syntaxhighlight>, <source lang=apl inline>⎕DQ</syntaxhighlight>, etc. |
6.3.1 | 1993 | April | <source lang=apl inline>⎕NA</syntaxhighlight>, graphical, clipboard and printer objects |
7.0.1 | 1994 | August | Namespaces, additional GUI objects |
7.1 | 1995 | May | <source lang=apl inline>⎕CS</syntaxhighlight>, GUI objects as namespaces, greater APL2 compatibility |
8.0 | 1996 | May | Keywords (<source lang=apl inline>:If</syntaxhighlight>/<source lang=apl inline>:Else</syntaxhighlight>, <source lang=apl inline>:Repeat</syntaxhighlight>/<source lang=apl inline>:Until</syntaxhighlight>, <source lang=apl inline>:Trap</syntaxhighlight>, and so on), <source lang=apl inline>⎕PATH</syntaxhighlight>, additional GUI objects, OLE |
8.1 | 1997 | March | dfns with lexical scope, syntax colouring, TCPSocket object, OLE client/server, automatic file tie numbers |
8.2 | 1999 | January | Windowed Reduction and scalar functions with axis (from APL2), Threading with Spawn (<source lang=apl inline>&</syntaxhighlight>), ActiveX, <source lang=apl inline>:With</syntaxhighlight>, additional GUI objects |
9.0 | 2000 | September | Namespace references (instead of string names) and dot syntax, context-sensitive help (F1), aditional GUI objects with animation |
9.0.1 | 2001 | January | (Windows CE) Pocket APL |
9.0.2 | 2002 | January | .NET support |
9.5 | 2002 | September | |
10.0 | 2003 | March | <source lang=apl inline>⎕NULL</syntaxhighlight>, <source lang=apl inline>⎕MAP</syntaxhighlight>, idiom recognition (mapped files), retained hash tables, .NET support built-in, run-time workspace as .exe, auto-completion, mapped |
10.1 | 2004 | July | Multiple arguments in tradfn headers, thread tokens, 64-bit component files, value tips |
11.0 | 2006 | October | Object-oriented programming (classes, objects, interfaces) modelled after C#, Index (<source lang=apl inline>⌷</syntaxhighlight>), Power operator (<source lang=apl inline>⍣</syntaxhighlight>), GCD (<source lang=apl inline>∨</syntaxhighlight>), LCM (<source lang=apl inline>∧</syntaxhighlight>) |
12.0 | 2008 | August | Unicode support (<source lang=apl inline>⎕AVU</syntaxhighlight>, <source lang=apl inline>⎕UCS</syntaxhighlight>), <source lang=apl inline>⎕FCOPY</syntaxhighlight>, <source lang=apl inline>⎕FPROPS</syntaxhighlight> |
12.1 | 2009 | November | I-beam (<source lang=apl inline>⌶</syntaxhighlight>), Table (<source lang=apl inline>⍪</syntaxhighlight>), <source lang=apl inline>⎕XML</syntaxhighlight>, <source lang=apl inline>⎕FCHK</syntaxhighlight>, User commands |
13.0 | 2011 | April | Left (<source lang=apl inline>⊣</syntaxhighlight>), Right (<source lang=apl inline>⊢</syntaxhighlight>), Variant (<source lang=apl inline>⍠</syntaxhighlight>), <source lang=apl inline>⎕OPT</syntaxhighlight>, <source lang=apl inline>⎕R</syntaxhighlight>, <source lang=apl inline>⎕S</syntaxhighlight>, <source lang=apl inline>⎕PROFILE</syntaxhighlight>, <source lang=apl inline>⎕RSI</syntaxhighlight>, complex number and decimal float support, short arguments for Take, Drop, and Index (<source lang=apl inline>↑</syntaxhighlight>, <source lang=apl inline>↓</syntaxhighlight>, <source lang=apl inline>⌷</syntaxhighlight>) |
13.1 | 2012 | April | <source lang=apl inline>⎕DMX</syntaxhighlight>, <source lang=apl inline>⎕FHIST</syntaxhighlight> |
13.2 | 2013 | January | Array Editor |
14.0 | 2014 | June | Trains, Tally (<source lang=apl inline>≢</syntaxhighlight>), Key (<source lang=apl inline>⌸</syntaxhighlight>), Rank operator (<source lang=apl inline>⍤</syntaxhighlight>), high-rank Index Of, multi-threading with futures and isolates |
14.1 | 2015 | June | <source lang=apl inline>:Disposable</syntaxhighlight> .NET objects and resources, gesture support, many new I-beams |
15.0 | 2016 | June | <source lang=apl inline>⎕MKDIR</syntaxhighlight>, <source lang=apl inline>⎕NDELETE</syntaxhighlight>, <source lang=apl inline>⎕NEXISTS</syntaxhighlight>, <source lang=apl inline>⎕NGET</syntaxhighlight>, <source lang=apl inline>⎕NINFO</syntaxhighlight>, <source lang=apl inline>⎕NPARTS</syntaxhighlight>, <source lang=apl inline>⎕NPUT</syntaxhighlight> |
16.0 | 2017 | June | At (<source lang=apl inline>@</syntaxhighlight>), Interval Index (<source lang=apl inline>⍸</syntaxhighlight>), Where (<source lang=apl inline>⍸</syntaxhighlight>), Nest (<source lang=apl inline>⊆</syntaxhighlight>), Partition (<source lang=apl inline>⊆</syntaxhighlight>), Stencil (<source lang=apl inline>⌺</syntaxhighlight>), <source lang=apl inline>⎕JSON</syntaxhighlight>, <source lang=apl inline>⎕CSV</syntaxhighlight> |
17.0 | 2018 | July | <source lang=apl inline>⎕NCOPY</syntaxhighlight>, <source lang=apl inline>⎕NMOVE</syntaxhighlight>, total array ordering, high-rank Unique |
17.1 | 2019 | October | Duplicates in Interval Index (<source lang=apl inline>⍸</syntaxhighlight>) look-up array |
18.0 | 2020 | June | Atop (<source lang=apl inline>⍤</syntaxhighlight>), Over (<source lang=apl inline>⍥</syntaxhighlight>), Constant (<source lang=apl inline>⍨</syntaxhighlight>), Unique Mask (<source lang=apl inline>≠</syntaxhighlight>), duplicates from Where (<source lang=apl inline>⍸</syntaxhighlight>), empty partitions from Partitioned Enclose (<source lang=apl inline>⊂</syntaxhighlight>), date-time conversion (<source lang=apl inline>⎕DT</syntaxhighlight>), case folding/mapping (<source lang=apl inline>⎕C</syntaxhighlight>), launching with text source file, .NET Core support |
18.2 | 2022 | March | <source lang=apl inline>⎕ATX</syntaxhighlight>, shell scripting |
Primitives
Functions
Glyph | Monadic | Dyadic |
---|---|---|
<source lang=apl inline>+</syntaxhighlight> | Conjugate | Plus |
<source lang=apl inline>-</syntaxhighlight> | Negate | Minus |
<source lang=apl inline>×</syntaxhighlight> | Direction | Times |
<source lang=apl inline>÷</syntaxhighlight> | Reciprocal | Divide |
</syntaxhighlight> | Magnitude | Residue |
<source lang=apl inline>⌊</syntaxhighlight> | Floor | Minimum |
<source lang=apl inline>⌈</syntaxhighlight> | Ceiling | Maximum |
<source lang=apl inline>*</syntaxhighlight> | Exponential | Power |
<source lang=apl inline>⍟</syntaxhighlight> | Natural Logarithm | Logarithm |
<source lang=apl inline>!</syntaxhighlight> | Factorial | Binomial |
<source lang=apl inline>○</syntaxhighlight> | Pi Times | Circular |
<source lang=apl inline>~</syntaxhighlight> | Not | Without |
<source lang=apl inline>?</syntaxhighlight> | Roll | Deal |
<source lang=apl inline>∧</syntaxhighlight> | And | |
<source lang=apl inline>∨</syntaxhighlight> | Or | |
<source lang=apl inline>⍲</syntaxhighlight> | Nand | |
<source lang=apl inline>⍱</syntaxhighlight> | Nor | |
<source lang=apl inline><</syntaxhighlight> | Less | |
<source lang=apl inline>≤</syntaxhighlight> | Less Or Equal | |
<source lang=apl inline>=</syntaxhighlight> | Equal | |
<source lang=apl inline>≥</syntaxhighlight> | Greater Or Equal | |
<source lang=apl inline>></syntaxhighlight> | Greater | |
<source lang=apl inline>≠</syntaxhighlight> | Unique Mask | Not Equal |
<source lang=apl inline>⍴</syntaxhighlight> | Shape | Reshape |
<source lang=apl inline>,</syntaxhighlight> | Ravel | Catenate |
<source lang=apl inline>⍪</syntaxhighlight> | Table | Catenate First |
<source lang=apl inline>⌽</syntaxhighlight> | Reverse | Rotate |
<source lang=apl inline>⊖</syntaxhighlight> | Reverse First | Rotate First |
<source lang=apl inline>⍉</syntaxhighlight> | Transpose | |
<source lang=apl inline>↑</syntaxhighlight> | Mix/First | Take |
<source lang=apl inline>↓</syntaxhighlight> | Split | Drop |
<source lang=apl inline>⊂</syntaxhighlight> | Enclose | Partitioned Enclose |
<source lang=apl inline>⊆</syntaxhighlight> | Nest | Partition |
<source lang=apl inline>∊</syntaxhighlight> | Enlist/Type | Membership |
<source lang=apl inline>⊃</syntaxhighlight> | First/Mix | Pick |
<source lang=apl inline>/</syntaxhighlight> | Replicate | |
<source lang=apl inline>⌿</syntaxhighlight> | Replicate First | |
<source lang=apl inline>\</syntaxhighlight> | Expand | |
<source lang=apl inline>⍀</syntaxhighlight> | Expand First | |
<source lang=apl inline>∩</syntaxhighlight> | Intersection | |
<source lang=apl inline>∪</syntaxhighlight> | Unique | Union |
<source lang=apl inline>⊣</syntaxhighlight> | Same | Left |
<source lang=apl inline>⊢</syntaxhighlight> | Same | Right |
<source lang=apl inline>⍳</syntaxhighlight> | Index Generator | Index Of |
<source lang=apl inline>⍸</syntaxhighlight> | Where | Interval Index |
<source lang=apl inline>⍒</syntaxhighlight> | Grade Down | |
<source lang=apl inline>⍋</syntaxhighlight> | Grade Up | |
<source lang=apl inline>⍷</syntaxhighlight> | Find | |
<source lang=apl inline>≡</syntaxhighlight> | Depth | Match |
<source lang=apl inline>≢</syntaxhighlight> | Tally | Not Match |
<source lang=apl inline>⍎</syntaxhighlight> | Execute | |
<source lang=apl inline>⍕</syntaxhighlight> | Format | |
<source lang=apl inline>⊥</syntaxhighlight> | Base | |
<source lang=apl inline>⊤</syntaxhighlight> | Represent | |
<source lang=apl inline>⌹</syntaxhighlight> | Matrix Inverse | Matrix Divide |
<source lang=apl inline>⌷</syntaxhighlight> | Materialise | Index |
Operators
Syntax | Monadic call | Dyadic call |
---|---|---|
<source lang=apl inline>f/</syntaxhighlight> | Reduction | Windowed Reduction |
<source lang=apl inline>f⌿</syntaxhighlight> | Reduction First | Windowed Reduction First |
<source lang=apl inline>f\</syntaxhighlight> | Scan | |
<source lang=apl inline>f⍀</syntaxhighlight> | Scan First | |
<source lang=apl inline>f¨</syntaxhighlight> | Each | |
<source lang=apl inline>f⍨</syntaxhighlight> | Commute | |
<source lang=apl inline>A⍨</syntaxhighlight> | Constant | |
<source lang=apl inline>f⍣v</syntaxhighlight> | Power | |
<source lang=apl inline>f.g</syntaxhighlight> | Inner Product | |
<source lang=apl inline>∘.f</syntaxhighlight> | Outer Product | |
<source lang=apl inline>A∘g</syntaxhighlight> | Bind | |
<source lang=apl inline>f∘B</syntaxhighlight> | ||
<source lang=apl inline>f∘g</syntaxhighlight> | Beside | |
<source lang=apl inline>f⍤B</syntaxhighlight> | Rank | |
<source lang=apl inline>f⍤g</syntaxhighlight> | Atop | |
<source lang=apl inline>f⍥g</syntaxhighlight> | Over | |
<source lang=apl inline>f@v</syntaxhighlight> | At | |
<source lang=apl inline>f⍠B</syntaxhighlight> | Variant | |
<source lang=apl inline>f⌸</syntaxhighlight> | Key | |
<source lang=apl inline>f⌺B</syntaxhighlight> | Stencil | |
<source lang=apl inline>A⌶</syntaxhighlight> | I-beam | |
<source lang=apl inline>f&</syntaxhighlight> | Spawn | |
<source lang=apl inline>f[B]</syntaxhighlight> | Axis |
Implementation
Dyalog APL is implemented primarily in C with some parts implemented in C++ in order to use templates. C intrinsics are used to access instruction set extensions. Some architecture-specific assembly, both compiled separately and inline from C, is used for functionality like exception flags which is not easily accessible in C. Prior to version 17.0, assembly was also used for vectorized arithmetic. In 17.0, this code was replaced by a new C++ implementation.
Internal types
Dyalog uses the following numeric types:
- 1-bit packed Boolean
- 1-byte integer
- 2-byte integer
- 4-byte integer
- 8-byte double
- 16-byte complex (one double for each component)
- 16-byte decimal float "decf" (BID or DPD)
Character encodings differ for classic and unicode interpreters: classic interpreters use a custom 1-byte encoding for all characters, and are limited to a 256-character set, while characters in unicode interpreters are 1-, 2-, or 4-byte unsigned unicode code point values.
Nested and mixed arrays (that is, pointer arrays) are always stored as arrays of pointers, while simple numeric or character arrays are always stored using one of the above types. For both numbers and characters, an array may be represented using any type that can contain all the values. The interpreter may reduce the type of an array to the minimum possible ("squeeze" the array) during execution.
Because there is no complex representation using decimal floats for the components, arrays containing both decimal floats and complex numbers have no common representation. Dyalog converts such arrays to complex numbers, resulting in a loss of precision for decf elements.
Instruction set usage
Dyalog makes heavy use of vector instructions on all platforms, as well as other special instruction sets primarily on x86. Instruction set availability is checked at runtime, so that the minimum required instruction set remains low:
- For 32-bit x86, only SSE2 is required.
- For x86_64, there is no minimum requirement as every processor supports SSE2. SSE4.1 is required on macOS as all x86 Apple machines support this instruction set.
- For ARM32, there is no minimum requirement.
- As of version 17.1, POWER7 and above are supported. Support for older systems is dropped because Dyalog compiles separate binaries for each POWER architecture.
In Dyalog 17.0, the code for vectorized scalar functions was unified and extended to allow Intel AVX2 and ARM NEON in addition to Intel SSE2 and SSE4.1, and AltiVec VMX for IBM POWER. This code is also used for operations involving the scalar dyadics Plus, Minus, Times, Divide, Maximum, Minimum, and comparison functions, as well as some functions derived from operators applied to these functions, such as the Outer Product and Inner Product.
Dyalog also uses many other x86 extensions:
- Since at least 12.1, SSE2 is used for scalar dyadics.
- Since 17.0, AVX2 is used for scalar dyadics if available.
- Since 14.1, SSE4.1 is used for Minimum and Maximum, and finding the range of an array. AVX2 can also be used for these purposes in 18.0.
- Since 17.0, SSSE3 is used primarily for the shuffle instruction for permuting arrays and searching small lookup tables.
- Since 14.0, SSE4.2 POPCNT is used to sum Boolean arrays.
- Since 14.0, SSE4.2 CRC32 is used to compute fast hash functions.
- Since 15.0, BMI2 is used for Boolean matrix transpose. Since 16.0, it is used for Boolean Compress and Expand, and several structural functions on Boolean arrays.
- In 18.0 only, CLMUL is used for xor reductions and scans.
- In 18.0 only, FMA3 is used to implement division by a singleton.
It also uses the POWER8 gather-bits-by-bytes instruction, which is equivalent to transposing an 8x8 bit matrix for Boolean Transpose since version 15.0 (expanded in applicability in 16.0) and the fused multiply-add instruction for division like x86 FMA3 in 18.0.
External links
References
- ↑ 1.0 1.1 Stephen Taylor. "How we got here". Vector journal Volume 23 special supplement "Dyalog at 25". 2008-09.
APL dialects [edit] | |
---|---|
Maintained | APL+Win ∙ APL2 ∙ APL64 ∙ APL\iv ∙ Aplette ∙ April ∙ Co-dfns ∙ Dyalog APL ∙ Dyalog APL Vision ∙ dzaima/APL ∙ GNU APL ∙ Kap ∙ NARS2000 ∙ Pometo ∙ TinyAPL |
Historical | A Programming Language ∙ A+ (A) ∙ APL# ∙ APL2C ∙ APL\360 ∙ APL/700 ∙ APL\1130 ∙ APL\3000 ∙ APL.68000 ∙ APL*PLUS ∙ APL.jl ∙ APL.SV ∙ APLX ∙ Extended Dyalog APL ∙ Iverson notation ∙ IVSYS/7090 ∙ NARS ∙ ngn/apl ∙ openAPL ∙ Operators and Functions ∙ PAT ∙ Rowan ∙ SAX ∙ SHARP APL ∙ Rationalized APL ∙ VisualAPL (APLNext) ∙ VS APL ∙ York APL |
Derivatives | AHPL ∙ BQN ∙ CoSy ∙ ELI ∙ Glee ∙ I ∙ Ivy ∙ J ∙ Jelly ∙ K (Goal, Klong, Q) ∙ KamilaLisp ∙ Lang5 ∙ Lil ∙ Nial ∙ RAD ∙ Uiua |
Overviews | Comparison of APL dialects ∙ Timeline of array languages ∙ Timeline of influential array languages ∙ Family tree of array languages |