Slide 59 of 74

Services across nodes

The same service, truly distributed
type Note {
	Say     { text: Str }
	HowMany
	Counted { seen: Int }
}

proc inbox(seen: mut Int, note: Note): Note {
	if note is Note.Say(text) {
		echo "heard: {text}"
		seen += 1
		return note
	} else if note is Note.HowMany {
		return Note.Counted(seen)
	} else if note is Note.Counted(_) {
		return note
	}
}

// Run this exact program on two machines: a node is identified by where it
// is, so nothing about the two processes differs but the addresses typed
// in below — 192.168.1.10:9100 and 192.168.1.11:9100, one each way round.
proc main(): void {
	echo "This node's own ip:port?"
	me := hive.term.read()
	echo "The other node's ip:port?"
	peer := hive.term.read()

	if hive.syslink.listen(me) is Result.Error(err) {
		panic err
	}

	if spawn(inbox, 0, #Inbox) is Result.Error(err) {
		panic err
	}

	mine := at(#Inbox)
	theirs := at(peer, #Inbox)

	async theirs(Note.Say("hello from {me}"))
	hive.task.sleep(500)
	echo "peers connected: {len(hive.syslink.peers())}"

	if theirs(Note.HowMany()) with timeout 4000 is Result.Ok(reply) {
		echo "the other node answered: {reply}"
	}
}

What hive.syslink keeps is the node itself. A node has no name: it is identified by the endpoint it can be dialled at, so a list of peers is ordinary runtime data.

  • listen(endpoint) opens this node to others and answers with the endpoint a peer should dial — a port of 0 asks the system to choose one, and the answer carries it. The host is advertised as written, since which address this machine is reached on is the program's to say, with hive.net.localAddress(). The port is bound on every interface, except that a loopback host binds only the loopback.
  • node() is this node's endpoint, "" before it listens, and peers() the nodes it is connected to right now.
  • A node has to listen before it can reach another: until then a request answers "NoListener".

Two nodes need nothing shared but a cluster key, written to ~/.hive/syslink.key on first run: copy that file to the other machine, set HIVE_SYSLINK_KEY the same on both, or have each program call hive.syslink.setKey(key) with a Secret — a key that is that program's alone, held in memory and never written to the file or the environment, and used for every connection made or accepted after the call. Every connection between two nodes is TLS 1.3, mutually authenticated. There is one persistent connection per pair of nodes, dialled on first use, so messages between two nodes arrive in the order they were sent. They cross as JSON, through the same codecs hive.json derives — so a message type's JSON as annotations shape the wire too, and a map cannot travel — with a digest of the message type beside each, so a node built from a different declaration fails loudly rather than reading another type's bytes. Delivery is best-effort: messages queued for a node declared down are dropped.