I have been dipping my feet into Distributed Systems and set up a cluster of Raspberry Pi Nodes recently.
The first thing I wanted to try was forming an Erlang cluster, And libcluster
makes this very easy through the Gossip strategy. Here is the code to form the
erlang cluster automatically (as long as it is on the same network).
Once, the clustering was set up I wanted to try sending messages through the
cluster and see how it performed, the simplest test I could think of was a baton
relay. Essentially, I spin up one GenServer per node and it relays a counter to
the next node, which sends it to the next node and so on like the picture below
(psa, psb, psc, and psd are the names of the nodes):
The code for this ended up being very straightforward. We create a GenServer and
make one of the nodes a main_node so that it can kick off the baton relay.
And, whenever we get a counter with a :pass message we increment the counter
and forward it to the next node. Here is the full code:
defmoduleHerd.Baton.ErlangProcess do use GenServer require Logger
@docfalse defstart_link(opts) do name = global_proc_name(Node.self()) Logger.info("starting a baton process", name: inspect(name)) GenServer.start_link(__MODULE__, opts, name: name) end
defpass(node, rest_nodes, counter) do GenServer.cast(global_proc_name(node), {:pass, rest_nodes, counter}) end
@impltrue definit(state) do send(self(), :init) {:ok, state} end
@impltrue defhandle_info(:init, state) do if main_node?() do if cluster_formed?() do pass(Node.self(), [], 1) else Process.send_after(self(), :init, 1000) end end
{:noreply, state} end
defpmain_node?() do Application.get_env(:herd, :main_node) == Node.self() end
defpcluster_formed?() do Node.list() != [] end
@impltrue defhandle_cast({:pass, nodes, counter}, state) do pass_the_baton(nodes, counter) {:noreply, state} end
defppass_the_baton([], counter), do: pass_the_baton(cluster_nodes(), counter) defppass_the_baton([next_node | rest_nodes], counter) do Datadog.gauge("baton", counter, tags: host_tags()) pass(next_node, rest_nodes, counter + 1) end
defphost_tagsdo tags(host: to_string(Node.self())) end
deftags(kwlist) do kwlist |> Enum.map(fn {k, v} -> "#{k}:#{v}"end) end
defpglobal_proc_name(node) do {:global, {node, __MODULE__}} end
defpcluster_nodesdo [Node.self() | Node.list()] |> Enum.shuffle() end end
Finally, here is the Datadog graph for the counter, The big thing to note is
that the 4 GenServers on a local lan were able to pass around 100M messages in 8
hours which amounts to about 3.5K messages per second which is impressive:
I am currently working on LiveForm which makes setting up contact forms on your website a breeze.