Tooling

One binary, every command. No project file to scaffold, no plugin system for the toolchain itself.

$ tau help
Usage: tau COMMAND [OPTIONS] ARGS

Tau is a dynamically typed, interpreted programming language.

Commands:
  run       Run a tau file
  build     Compile tau files into '.tauc' bytecode
  bundle    Compile a tau file into a standalone executable
  test      Run the tests of the given files or directories
  fmt       Format tau source files
  doc       Show what a module exports
  repl      Start the interactive prompt
  get       Fetch a module and add it to tau.mod
  mod       Look after tau.mod
  version   Print version information
  help      Display help for a command

run

$ tau run main.tau
$ tau main.tau          # the same thing, run is the default
$ tau main.tau a b c    # arguments land in os.Args

run compiles the file to bytecode in memory and hands it to the VM. It also runs a .tauc file, so the compile step can happen ahead of time without changing how you launch the program.

A file with a shebang is an ordinary executable:

#!/usr/bin/env tau

println("Hello, world")

build

$ tau build main.tau            # writes main.tauc next to the source
$ tau build -o app.tauc main.tau
$ tau build *.tau

The output is not just bytecode. build walks the import and dlopen calls in the source and packs every module and shared object the program needs into the .tauc file. The result runs on a machine where neither your modules nor the standard library are installed:

$ tau build -o app.tauc main.tau
$ scp app.tauc server:/tmp
$ ssh server 'tau run /tmp/app.tauc'
hello, world
greeter v1.0
HELLO, TAU

One file to copy, nothing to install beside it. The bundle format is marked so that plain bytecode files keep working exactly as before.

bundle

$ tau bundle main.tau            # writes an executable named after the source
$ tau bundle -o app main.tau

Same contents as build, appended to a copy of tau-rt instead: the runtime a compiled program actually needs, which is the VM, the objects and the bytecode decoder, written in C, with no lexer, no parser, no compiler and no Go in it. The result runs on its own on a machine with no Tau installed, and is a few hundred kilobytes rather than the several megabytes the interpreter weighs.

$ tau bundle -o app main.tau
$ ls -lh app
-rwxr-xr-x 1 you you 431K app
$ ./app
hello, world

test

$ tau test                     # the current directory
$ tau test stdlib
$ tau test stdlib/strings_test.tau

test finds the *_test.tau files under the paths you give it and runs each one in its own process, so a test that crashes the VM takes down only itself. The convention is Go’s: a test file sits next to the code it tests, and the whole run exits non-zero as soon as one case fails.

$ tau test .
=== mathx_test.tau
--- PASS: Double (0ms)
--- PASS: Div (0ms)
ok      2 passed of 2 (0ms)

ok      1 test files passed

The testing module supplies the assertions — see the tutorial for the shape of a test file.

fmt

$ tau fmt main.tau      # print the formatted source
$ tau fmt -w main.tau   # rewrite the file
$ tau fmt -w .          # rewrite the tree
$ tau fmt -l .          # only list what would change

One canonical style, no options to argue about: tabs for indentation, one space around binary operators, comments left where you put them. Directories are walked recursively. -l is the form for a CI check — it prints the files whose formatting differs and nothing else.

doc

$ tau doc strings           # the module: its comment and every exported name
$ tau doc sync.Mutex        # one name: its whole comment and the names under it
$ tau doc sync.Mutex.Lock   # go as deep as the names go
$ tau doc -b encoding/json  # open it as a page in the browser

There is no separate documentation to keep in step with the code, because the documentation is the code: doc reads the comment written above each exported name and the names that name holds in turn, and lays them out. A module is looked up the way an import looks one up, so it works on your own modules exactly as it does on the standard library.

-b writes a self-contained page — the prose in a serif to be read, the names in a monospace to be typed, every name linking to the line it was read from — and opens it. The whole standard library is rendered that way in the reference: strings, sync, encoding/json and the rest are all linked from the standard library page.

repl

$ tau repl
$ tau            # no arguments does the same

The prompt is multiline: it keeps reading while a block is open, so functions and loops can be typed as they are written.

Tau v2.1.0 on Linux
>>> repeat = fn(n, func) {
...     for i = 0; i < n; ++i {
...         func(i)
...     }
... }
...
>>> repeat(3, fn(i) {
...     println("Hello #{i}")
... })
...
Hello #0
Hello #1
Hello #2
>>>

Modules

A module is one file, or the directory holding several. import("name") evaluates it once and gives back an object; the names that start with a capital letter are what it hands out, the rest stay private. The files of a directory share one scope, so a module splits across files without any of them having to export to the others, and a path with slashes reaches into a directory:

sha256 = import("crypto/sha256")   # a submodule of crypto
util   = import("./util")          # util.tau, or util/, next door

A local import is resolved in order against:

every directory listed in the TAUPATH environment variable,
the directory of the file doing the import,
~/.local/lib/tau, then /usr/local/lib/tau and /lib/tau, which is where make install puts the standard library.

So TAUPATH is how you point at a library that lives somewhere else, and a module next to the file that imports it is found without any setting at all:

$ TAUPATH=~/tau/stdlib tau run main.tau

Modules from somewhere else

An import path whose first element is a host — anything with a dot in it — names a module fetched from elsewhere rather than looked up locally. There is no registry: the path is already the address.

example = import("github.com/NicoNex/example")
util    = import("github.com/NicoNex/example/util")

A module says who it is in a tau.mod at its root, and lists what it requires:

module github.com/NicoNex/example

tau 2.1

require (
	github.com/x/y v1.4.0
)

$ tau mod init github.com/you/thing   # write the tau.mod
$ tau get github.com/x/[email protected]        # add a requirement and fetch it
$ tau mod tidy                         # require what the source imports, drop the rest

tau get without a version takes the highest tag of the form vX.Y.Z. Where two dependencies ask for different versions of a third, the build takes the highest of what was asked and never a version nobody asked for — minimum version selection, an answer that needs no solver. From v2 the major is part of the path, so github.com/you/thing and github.com/you/thing/v2 are two modules and a program may hold both.

Fetched modules land under $TAUHOME, or ~/.tau when that is unset, one directory per version, read-only once written. tau.sum holds the hash of every version the build reads, so a tag moved after the fact stops the build instead of running. Nothing is ever fetched while a program runs: tau get and tau mod tidy reach the network, a build reads what is already there, and a bundle carries it. Fetching goes through git, which is therefore needed to get a module and not to build or run one.

C libraries

C shared libraries load straight into a program, no binding layer to write:

$ gcc -shared -o mylib.so -fPIC mylib.c

mylib = dlopen("mylib.so")

mylib.hello()

dlopen(path) opens a shared object and the dot on the handle is dlsym, so mylib.hello is the symbol of that name. The index does dlsym too, and mylib["hello"] is the same lookup with the name worked out while the program runs. dlopen(null) is the handle of the program itself, which is where the C library it was linked against can be reached.

The name is the name of a file, and it is a different one on every system: libm.so.6 on a glibc machine, libm.dylib on macOS, and on glibc the unversioned libm.so is a linker script dlopen refuses. ffi.Lib("m") is the one to reach for when the program has to run on more than one machine — it tries the shapes of this system and says what it tried when none of them opens, while a name that already has a slash or an extension is opened as it stands.

tau build picks the shared object up and carries it inside the bundle along with the modules.

From there the C interface is two layers, and the second is built out of the first.

Layer 1: no declaration at all

A symbol can be called immediately, with nothing written down. Every argument goes as a 64 bit integer or a double, and the result comes back as the machine word the function left behind: int(x) reads it as a number, string(x) as a char *, bytes(x, n) as a buffer, and a pointer C handed over goes straight back into the next call.

# Layer 1: open a shared object and call a symbol with nothing declared.
# The name is the system's; this one is the C library of a glibc machine.
libc = dlopen("libc.so.6")

# Memory, the way C does it: a pointer C returns can be handed straight back
# to C, and read from tau with string() or bytes().
p = libc.malloc(32)
libc.memcpy(p, "hello", 6)
println(int(libc.strlen(p)), string(p), string(bytes(p, 5)))
libc.free(p)

# The dot is dlsym, and the call takes your word for the types: what comes
# back is the machine word the function left behind, decoded with int().
println(int(libc.strlen("hello"), 64))
println(int(libc.abs(-7), 32), int(libc.toupper(97), 32))

# The index does dlsym too, so a name can be worked out while the program
# runs rather than written in the source.
names = ["atoi", "strlen"]
for i = 0; i < len(names); ++i {
	println(names[i], int(libc[names[i]]("1234"), 32))
}

# A symbol that is not there is a value, not a crash.
println(failed(libc.nosuchfunction))

# The limit of the layer: a function returning a double answers in a register
# this call does not read, so the word is meaningless. That is where the
# signature starts earning its keep.
libm = dlopen("libm.so.6")
println(float(libm.sqrt(16.0), 64))

5 hello hello
5
7 65
atoi 1234
strlen 4
true
4.94065645841247e-324

Three lines to try a library, and it takes your word for the types: get them wrong and nothing complains, you just get a number that means nothing. The last line of the sample is that, in the open — sqrt answers in a floating-point register the untyped call never reads, so the word it hands back is not the result. That is the trade, and it is a fine one while you are finding out whether a library does what you want.

Layer 2: the declaration written down

ffi is the standard library module that gives a symbol its types. Same dlopen, same dot, one more step:

ffi = import("ffi")
pow = ffi.Func(libm.pow, "double pow(double, double)")

From then on the arguments travel as those types and the result comes back as a tau value, a float that prints 1024 rather than a word to decode.

ffi = import("ffi")

# Layer 2: the same dlopen and the same dot, with the C declaration written
# down. Nothing comes back as a machine word.
libm = dlopen("libm.so.6")

pow = ffi.Func(libm.pow, "double pow(double, double)")
println(pow(2.0, 10.0), type(pow(2.0, 10.0)))

# Bind takes a whole library at once, naming each function the way its
# signature names it.
sigs = [
	"double sqrt(double)",
	"double hypot(double x, double y)"
]
m = ffi.Bind(libm, sigs)
println(m.sqrt(2.0), m.hypot(3.0, 4.0))

# A char * travels as a tau string, any other pointer as an address, and a
# bytes value is the buffer C writes into.
libc = dlopen("libc.so.6")
snprintf = ffi.Func(libc.snprintf,
	"int snprintf(char *str, size_t size, const char *format, double x)")

buf = bytes(64)
n = snprintf(buf, 64, "pi is about %.3f", 3.14159)
println(string(slice(buf, 0, n)))

# The parse on its own: the result code, the argument codes, the name.
println(ffi.Sig("size_t strlen(const char *s)"))

# A signature that does not say enough is an error, not a surprise later.
println(failed(ffi.Func(libc.printf, "int printf(const char *fmt, ...)")))

1024 float
1.4142135623730951 5
pi is about 3.142
[9, [13], strlen]
true

ffi.Bind(lib, [signatures]) does a whole library at once, naming each function the way its signature names it, and takes the name of a library as well as an open handle. ffi.Sig(text) is the parse on its own, returning [result code, [argument codes], name] — the codes are ffi.Void … ffi.CString, and 9, [13] above is a uint64 result taking one C string. Underneath, ffi.Func hands those numbers to the cfunc builtin, which prepares the call once: reading a C declaration is tau, in stdlib/ffi.tau, and the builtin never sees text.

The other direction

ffi.Export turns a tau function into one C can call, which is what a library wants when it takes a handler, a comparator or a visitor — GTK is nothing but handlers, and until this existed a whole class of library was out of reach.

# The other direction: a tau function C can call. qsort takes a comparator,
# and the comparator is written here.
ffi = import("ffi")
ref = import("ref")

libc = ffi.Lib("c")
qsort = ffi.Func(libc.qsort, "void qsort(void *base, size_t n, size_t size, void *cmp)")

# The count goes in a ref because a closure captures by value: a plain number
# would be a copy of its own, and C would be writing into nothing.
calls = ref.New(0)

cmp = ffi.Export("int compare(const void *a, const void *b)", fn(a, b) {
		calls.v = calls.v + 1
		return bytes(a, 1)[0] - bytes(b, 1)[0]
	})

buf = bytes([5, 3, 9, 1, 7])
qsort(buf, 5, 1, cmp)

println(buf)
println("C called back into tau {calls.v} times")

[1, 3, 5, 7, 9]
C called back into tau 7 times

What comes back is an ordinary function pointer as far as C is concerned: libffi writes the trampoline, and the entry point converts what C passed into tau values, runs the function and writes the answer back where C is waiting.

Three things are worth knowing. The call is answered by the VM of the thread that entered C, so a handler called from the loop of a library works and one called from a thread the library made for itself finds no tau there and gets a zero. A function that fails inside a callback does not unwind through C: the failure stops there and the C side carries on, because a jump through a half-finished C frame leaves it holding locks and allocations nobody will free. And the exported function lives for as long as the program does — whoever holds the pointer never says when they are done with it.

Types across the boundary

The signature is a C declaration. The name of the function and the names of the arguments may be there or not, so a line copied out of a header works as it stands, and const, volatile and restrict are read and ignored. char, short, int, long, long long and size_t have the width they have on this machine rather than an assumed one; int8_t … uint64_t are exactly that many bits, and so are int8 … uint64, float32 and float64, for when tau’s own spelling reads better. A char * travels as a tau string, any other pointer as an address.

A variadic signature is refused: the call is prepared once, and ... says nothing about what will be passed, so write the types this call passes — int(char *, size_t, char *, double) rather than int(const char *, ...).

A bytes value is what a function writes into when it wants a buffer, and bytes(ptr, n) copies back whatever a returned pointer points at. A missing symbol, a bad signature or an argument C cannot take all come back as errors you check with failed, rather than taking the process down.

← Standard library Home →