I decided to divide this article into phases but if you want to jump directly into the finished code you can go here.
First of all, let’s install Elixir (1.11.0), Erlang (23.1.1.) and Phoenix (you can find the guide here) and create the automatic generate project with mix phx.new liveMapApp --live
. This command will generate the necessary code to have a running elixir server.
Done that, we want to generate a new page called Dashboard
. It should be linked with the different application’s downloads. To do so we also create an app
component and its related database table. The new Phoenix version can do all these actions in one command:
mix phx.gen.live Dashboard App apps longitude:decimal latitude:deciaml app_id download_at:utc_date_time
After running it, we should just copy the new router generated into the router page as it is stated in the command’s output:
Add the live routes to your browser scope in lib/liveMapApp_web/router.ex:
live “/apps”, AppLive.Index, :indexlive “/apps/new”, AppLive.Index, :new
live “/apps/:id/edit”, AppLive.Index, :edit
live “/apps/:id”, AppLive.Show, :show
live “/apps/:id/show/edit”, AppLive.Show, :edit
Our basic application is setup, we can test it by creating the database and applying migration. This is done by:
mix ecto.createmix ecto.migratemix phx.server
Actually, if we forget to run the database creation and the migration, Phoenix remembers it for us and shows a nice error message with the option to launch it though a button.
At the route page of our application we now have a list of downloaded applications stored into a table and some buttons to add/show/delete them. Let’s integrate with Google Maps.
The once below is the suggested from the documentation:
We can change part of this code to be generated dynamically by our liveView:
<%= for download <- @downloaded_apps do %> new google.maps.Marker({ position: { lat: <%= download.latitude %>, lng: <%= download.longitude %> }, map, title: "<%= download.app_id %>", });<% end %>
Moreover, in order to see the map in the page we need to tweak Phoenix a little bit by adding the following piece to our map container: phx-update="ignore"
. This command will allow us to avoid the map deletion from Phoenix. Be also sure to have an id in the container as the map will keep refreshing every milliseconds. To understand better this phenomena you can read how LiveView lifecycle works here.
We can now add a new app using the button and we can see the changes in the table but not on the map. That is because our Google Maps object is not yet linked with any update actions from LiveView. We will take care of it in a minute.
Let’s first create some dashboards. We need at least to show the downloads by time of the day and by country. To do so we need to know from which country the download comes from. We could use GoogleAPI to know the country by looking at the coordinates and then group them by country. This would imply to do a GoogleAPI call for every row in the database every time we refresh the page. This is not scalable at all 🙉!! I decided to go with a cheaper and scalable option which is to store the country when the new download is registered. This implies only one API call for each download in the database.
To do so we have to add various things:
mix ecto.gen.migration add_country
and inserting this code into the generated file:defmodule LiveMapApp.Repo.Migrations.AddCountry do
use Ecto.Migrationdef change do alter table(:apps) do add :country, :string, default: "Unknown" end endend
lib/liveMapApp/dashboard/app.ex
mix ecto.migrate
to apply the new migrationdownloaded_app
. We can do that by adding a new controller into the application which will be in charge of validate the parameters, make the API call to gather the country, store the information into the dbNow we can setup our simple dashboards by adding some HTML and SCSS:
@for $i from 1 through 100 { .percentage-#{$i}:after { $value: ($i * 1%); width: $value; }}
<dl> <dt>Download by country</dt> <%= for {countryName, count} <-Enum.reduce(@downloaded_apps,
%{}, fn x, acc -> Map.update(acc, x.country, 1, &(&1 + 1)) end) do %> <dd class="percentagepercentage-<%= trunc((count/length(@downloaded_apps))*100) %>">
<span class="text"> <%= countryName %>:<%= Float.round((count/length(@downloaded_apps))*100,2) %>
</span> </dd> <% end %></dl>
This will generate 100 css classes that can be used by the HTML code. In the HTML we use Elixir to group the downloaded app list by country and calculate the percentage.
We can now add many histogram dashboards we want by slightly changing the HTML code.
Note: If you want to see the whole SCSS used have a look at the repository.
We now have a working application which needs to be reloaded every time we have a change. This means every time we insert a new row into the database.
LiveView takes care of most of the real time part, we just need to broadcast the changes into our GenServer wrapper which is called PubSub
. To do so we just need to add these lines of code where we want to generate the update request. In our case it will be in the module Dashboard
which handles the insertion into the database:
defp broadcast({:error, _reason}=error, _event), do: errordefp broadcast({:ok, app}, event) do Phoenix.PubSub.broadcast(LiveMapApp.PubSub, "apps", {event, app}) {:ok, app}end
As you can see above, we need to do pattern matching on the parameters as we will call this function just after the insert.
In our index page we then need to subscribe to the channel created and handle the event sent.
if connected?(socket), do: Dashboard.subscribe()
Before subscribing we need to check if the socket on the frontend exists and if it is connected to the backend. In the end we need to handle the broadcast event in order to update the view:
def handle_info({:download_added, app}, socket) do {:no_replay, update(socket, :downloaded_apps,fn apps -> [ apps]end)}
Now we are able to hit the app endpoint using PostMan or Curl and we can see immediately the changes in the page, the dashboard and the table change but Google Maps doesn’t. That is because we blocked Phoenix to update the component in order to be able to use it.
To solve this we can use a new feature in LiveView called Hook which works in a pretty similar way as the React hook. Some more details here . We need to add this code into our app.js
let Hooks = {}Hooks.MapMarkerHandler = {
mounted() {this.handleEvent("new_marker", ({ marker }) => {var markerPosition = { lat: parseFloat(marker.latitude), lng: parseFloat(marker.longitude) }const mapMarker = new google.maps.Marker({ position: markerPosition, animation: google.maps.Animation.DROP, title: marker.app_id }) mapMarker.setMap(window.map) }); }let csrfToken = document.querySelector("meta[name='csrf-token']").getAttribute("content")}
let liveSocket = new LiveSocket("/live", Socket, {hooks: Hooks, params: {_csrf_token: csrfToken}})
This will create an eventHandler on the event coming from the server. Let’s link our container by specifying phx-hook="MapMarkerHandler"
into it and firing a new event from the backend every time we need to add a new marker. To do so we need to change our broadcast function to react to the creation event in a different way, leaving the general one so that a new event will be still handled:
defp broadcast({:error, _reason}=error, _event), do: errordefp broadcast({:ok, app}, :download_added = event) do Phoenix.PubSub.broadcast(LiveMapApp.PubSub, "apps", {event, app}) Phoenix.PubSub.broadcast(LiveMapApp.PubSub, "apps", {:new_marker, app}) {:ok, app} end defp broadcast({:ok, app}, event) do Phoenix.PubSub.broadcast(LiveMapApp.PubSub, "apps", {event, app}) {:ok, app}end
This is needed as we cannot push more than one event to the client in a single handle_info
method in our index.ex
file. We will then need also
def handle_info({:new_marker, app}, socket) do {:noreply, push_event(socket, "new_marker", %{marker: %{ latitude: app.latitude, longitude: app.longitude, app_id: app.app_id} } )}end
To handle the new event in the page.
We now have a function application respecting all the initial requirements. Let’s move to the most important part of it: TESTS
I usually start from the tests and then write the code to have them working. In this case, as it was my first time using Phoenix LiveView, I decided to go with a code first methodology. That is because I didn’t know what the app would have looked like at the end. Now the app is functional and we can actually see which tests are already there and which ones we want to add.
If you don’t write unit tests first the best practise is to write them along the way because while writing the code you know better in which way your code could fail. If you do it at the end, especially on big projects you will most probably forget one method that contains a tiny typo that will wake you up in the middle of the night on a Saturday😅.
Phoenix and LiveView help you in the test creation by adding some standard tests that you could start from. In particular for this application we want to test:
a library for defining concurret mocks in Elixir
from the documentationService Tests consist of the scaffolding, dependencies, and actual tests necessary to isolate and assert a service can function.
Service level tests are really useful to test the whole application without worrying about the dependencies. In the context of the article I didn’t wrote them but is certainly something to add as future improvement.
Note: to run the tests we just need to run MIX_ENV=test mix test
Even though I’ve done all this article on my local machine I prefer it to be dockerized as it’s easier to run from other environments and also on kubernetes 😉. Let’s do it.
In order to run this into docker we need to create some folders and files:
version: '2.2'services: live-map-app: environment: - UID build: context: . dockerfile: Dockerfile.local args: hex_repo_key: ${HEX_REPO_KEY} uid: ${UID} command: bash scripts/dev.sh env_file: ./env/dev.env ports: - 4005:4000 volumes: - .:/home/app/service/ - elixir-artifacts:/home/app/elixir-artifacts depends_on: live-map-app-db: condition: service_healthy networks: - shared- default
live-map-app-db: environment: POSTGRES_USER: postgres POSTGRES_PASSWORD: "postgres" PSQL_TRUST_LOCALNET: 'true' ENCODING: UTF8 image: postgres:9.6 healthcheck: test: ["CMD", "pg_isready", "-d", "postgres", "-U", "postgres"] interval: 10s timeout: 3sretries: 10
networks: shared:volumes:external: true
elixir-artifacts: {}
This is the starting point from where we create a database and a container for our application. Then we can add the Dockerfile.local
which contains:
# Elixir 1.10 with Erlang/OTP 22
FROM elixir@sha256:ba981350b63eb016427d12d90dad643eea8e2bfed37e0f2e4f2bce5aa5303eaeLABEL maintainer="francesco.zanoli"ARG run_deps="inotify-tools"ARG mix_env="dev"ARG http_port="4000"ARG app_path="/home/app/service"USER rootENV TERM xtermARG uid="1000"
ENV HOME /home/appENV APP_PATH ${app_path}ENV HTTP_PORT ${http_port}ENV MIX_ENV ${mix_env}ENV REFRESHED_AT 2020-10-08RUN apt-get -q update && apt-get -qy --no-install-recommends install ${run_deps}ENV ERL_AFLAGS="-kernel shell_history enabled"
RUN curl -sL https://deb.nodesource.com/setup_14.x
RUN apt-get install -y nodejs npmRUN adduser --disabled-password --gecos '' app --uid ${uid}RUN mkdir -p /home/app/elixir-artifacts
RUN chown -R app:app /home/app/elixir-artifactsUSER app:appRUN /usr/local/bin/mix local.hex --force && \ /usr/local/bin/mix local.rebar --force && \RUN echo "PS1=\"\[$(tput setaf 3)$(tput bold)[\]\\u@\\h$:\\w]$ \"" >> /home/app/.bashrcCOPY --chown=app:app . ${APP_PATH}WORKDIR ${APP_PATH}EXPOSE ${HTTP_PORT}CMD ["sh", "script/start.sh"]/usr/local/bin/mix hex.info
The only thing to note here it’s that we pointed to a specific docker image, using the SHA256. This will avoid strange situations in which a future deploy will change the Elixir or OTP version and causing the crash of the application or even worst, some strange bugs.
Now we can create the .env
file, we will use it in the next section. Finally we have to add the `dev.sh` file which contains all the code to be run once the container is up:
#!/bin/bashset -excd assetsnpm installcd ..
mix deps.getmix compileecho "run docker exec -it ${HOSTNAME} sh in another console to jump into the container!"tail -f /dev/nullmix do ecto.create, ecto.migrate
#mix phx.server
I usually don’t start directly the service because this allows to run the test first, as I don’t usually run them directly in my local machine. To start it up we just need to follow the instructions to enter the container and launch mix phx.server
We have a functional, working, tested, dockerized application, now let’s make it pretty 🎉. We have different aspects to cover:
The project generated from CLI doesn’t take care of the environment variable. However there are some information that we don’t want to store in the code which should be kept secret. In this project the most important ones are the GoogleApi Key and the Phoenix encryption key. It’s also best practise to include the database information as this could be a sensitive data in a production scenario. Elixir has a folder to manage the configuration, config
, so we just need to change the files for the different environments. For the database we just need to change LiveMapApp.Repo
config in dev
and test
config :live_map_app, LiveMapApp.Repo, username: System.get_env("POSTGRES_USER"), password: System.get_env("POSTGRES_PASSWORD"), database: System.get_env("POSTGRES_DB"),hostname: System.get_env("POSTGRES_HOSTNAME")
For the keys we need to update the main configuration as they won’t change depending on the environment
config :live_map_app, ecto_repos: [LiveMapApp.Repo],api_token: System.get_env("API_TOKEN")
# Configures the endpointconfig :live_map_app, LiveMapAppWeb.Endpoint, url: [host: "localhost"],secret_key_base: System.get_env("PHOENIX_SECRET")
To access to them we could then do: Application.get_env(:live_map_app, :api_token)
Note: This usage will not work for the production environment as the configuration we changed are loaded at compile time. In order to postpone this loading for run time we need to use an Elixir feature called mix release
which is not discussed in this article.
After the environment variables we want to remove all the warning ⚠️ from the test and from the application, cleaning them up will help us for future development and to discover problems.
If you have followed this article until here you would have notice that the months in the dashboard as well as the days are shown by using numbers. To change this we can add a private function into our controller as follow:
defp get_day_name(day) do case day do 1 -> "Monday" 2 -> "Tuesday" 3 -> "Wednesday" 4 -> "Thursday" 5 -> "Friday" 6 -> "Saturday" 7 -> "Sunday" _ -> "Unknown" endend
defp get_month_name(month) do case month do 1 -> "Jan" 2 -> "Feb" 3 -> "Mar" 4 -> "Apr" 5 -> "May" 6 -> "Jun" 7 -> "Jul" 8 -> "Aug" 9 -> "Sep" 10 -> "Oct" 11 -> "Nov" 12 -> "Dec" _ -> "Unknown" endend
We will just need to update the code to call this function before passing the values to the dashboard component.
If you look at the app you will now notice that we are missing a dashboard from the requirements. I left this at the end in order to show how LiveView allows you to go fast in future improvements. As we have our component we just need to add a new line into the view in order to have the dashboard we are looking for:
<%= live_component(@socket, Dashboard, id: "ByDayTime", title: "Dashboard by time of the day",list: Enum.reduce(@downloaded_apps, %{},
fn x, acc -> Map.update(acc, get_day_time(x.download_at), 1, &(&1 + 1)) end),total: length(@downloaded_apps),
percentage: true)%>
The reduce function we are using this time calls a new function from the controller to get in which interval the download is in.
So the app is working but it’s really ugly, let’s add some filters in the dashboard in order to hide some of them. To do so we can use Css and a radio button using this HTML:
<input type="radio" id="filter" name="categories" value="filter">
<label class="filters" for="filter">Country</label>
an this CSS:
[value="filter"]:checked ~ .dashboard:not([data-category*="filter"]) { display: none;}
Then we just need to add data-category="filter"
and class="dashboard"
to the HTML component we want to show when the filter is clicked. In particular the css code will hide all the components which are part of the same parents and have the dashboard
class.
⚠️ ⚠️ Phoenix will not care about the state of the radio button during the update. We need to add phx-update="ignore"
to all the filters that we want to.
In this phase I also always double check the documentation in the code and remove unnecessary comments, as the one generated by Phoenix. This time I decided to leave all of them in order for you to understand better what is what and what was automatically generated.
Lastly we need to update the README.md to describe how to use, test, run the project.