Hackathon Insights: November Projects

METHODOLOGIES

By
Vika

November 27, 2017


Each month, we organize a two-day internal hackathon during which WATTx-ers dedicate their time to working on side projects and get a chance to explore new trends in design, tech, and data science. In November, we hacked on projects involving coffee, canteen food, meeting rooms, as well as continuous integration and delivery. Here’s the digest for what we learned.

Making Cophi great again

During this month’s hackathon, Cophi - the smart scale that shows how much coffee is left in a bialetti - went into the third iteration round (read more about the first two rounds here and here), as it still showed operating problems:

  1. The Cophi software wasn’t running smoothly, crashed often, and had to be manually restarted;
  2. sensor values showed a high level of noise; and
  3. the smart scale, which the project team connected to Slack, was sending too many messages, spamming the channel with constant updates as can be seen in the following screenshot:

This month’s project team, consisting of Data Scientist Christian and Software Developer Wen, tried to eliminate the fluctuation and noise level of sensor values. By using the median instead of the raw values, the team filtered out outliers in the data the Cophi software processed. This strategy resulted in much more reliable output, as can be seen in the following charts:

To limit the amount of messages sent within Slack, Christian wrote an algorithm that continuously compares last recorded values and sent messages. To do this, the algorithm calculates the time since the last message was shared, while also checking for changes in the bialetti.

And the results are promising: we now have a stable and reliable coffee channel in which we are updated each time coffee is ready as well as with the exact number of cups left in the bialetti. Additionally, we are now able to analyse the data in order to create statistics on how high the coffee consumption in our office is.

Mapping Berlin canteens

We at WATTx love eating in canteens. Especially our Engineer Pedro, who cannot resist a quick and cheap canteen lunch. After he found a complete list of Berlin canteens, he was driven to build a map that plots every place from this list, solving the issue of having to search for each one individually.

Excerpt from the list of Berlin canteens

Pedro first turned to Google Maps APIs. In order to convert addresses from the canteen list into coordinates, he used the Geocoding API. With the help of Places API Web Services he was able to display ratings, reviews, and opening times for most of the places. Finally, by using Distance Matrix API, he managed to calculate the distances and time needed from our WATTx office by means of walking, cycling, driving, or public transport.

Once the team, which included our Developers Tassilo and Omar, had the preprocessed data ready, they used the following architecture in order to place the coordinates on a map:

Tassilo built a Go backend which hooked into a Postgres database which was fed with the canteen data. In a next step, Omar built the React frontend using LeafletJS to handle the map visualization.

In the end, the team was able to create a map displaying all canteens in Berlin while also visualizing the distance (using a colour code from green (close) to red (far away)) to the WATTx office with the individual ratings (using a circle diameter; the bigger the circle the better the canteen). An example can be seen in the following screenshot:

What’s going on in our meeting rooms?

An issue we often encounter here at WATTx is that once the doors to our meeting rooms are shut, there is no way of knowing what is going on inside: Is there anyone in the room? Can I just walk in, or is there an important meeting going on? You may have encountered the issue in your own company.

The problem inspired our tech people Franzi and Blake to create a digital solution for room indicators that shows whether a meeting room is occupied and what is happening inside at a glance.

The technological options were vast, so as a starting point, the team was considering the use of LCD versus E-Ink displays in order to show information about meetings. While LCD displays offer the clear benefit of user interaction they still were quickly discarded for one main reason: the high energy consumption. E-Ink displays on the other hand are efficient enough to represent information while at the same time consuming energy only when a new picture has to be pushed to the screen, otherwise - when there is no change in the picture - the display consumes no energy.

The team tested two different displays: the Papirus E-Ink and the Waveshare E-Ink. As the first display unfortunately broke during setup, Franzi was forced to spontaneously order the Waveshare E-Ink display, which luckily arrived the second morning of the two-day hackathon. However, in the meantime, she and Blake also tested a tablet interface.

What the team did was firstly, writing a Go application that consumes the Google Calendar API, serves a web frontend for the tablet view, and exposes the relevant room information in json-format.; and secondly, writing a Python program that reads the exposed information and feeds the E-Ink display.

In the end, the team had two interfaces that showcased whether a meeting room was occupied, and, in case of an ongoing meeting, details on title or meeting content, duration, and the involved participants or organizers.

Ensuring bug-free code

We at WATTx consider testing a new service we’re developing highly important in order to keep it bug-free.

During this month’s hackathon, our Software Engineer Rafal aimed for improving the current process of testing newly written code by setting up an infrastructure tool for the continuous integration and delivery pipeline (CI/CD) of our services and micro services.

The first step Rafal undertook was to set up the tool Drone.io on the Google Cloud platform:

Drone is a Continuous Delivery system built on container technology. Drone uses a simple YAML configuration file, a superset of docker-compose, to define and execute Pipelines inside Docker containers.

When working on a service, developing features, or simply changing something; as soon as code is submitted to GitHub, a job on Drone.io is triggered that checks the code and runs tests (pipeline example: clone, test, compile, deploy). In a further step, Drone sends out feedback (if integrated to Slack for instance) on whether everything is correct or whether there are mistakes found in the code.

In the end, Rafal was able to migrate a few of our services from Travis CI to Drone.io and the results were promising: the code running on Drone was slightly faster and the migration process went smoothly.

Ad-hoc service deployments

Our Software Engineer Marcin had the idea to create a workflow for developers to deploy and publish their individual work independently.

Traditionally, the software development and testing process is done on a developer’s machine and only deployed to the staging or production environment in the very last step. This process is constraining especially when a system consists of multiple services developed independently.

Testing a development version of one service against a production-like system requires either deploying it to a staging environment (which might block the staging access to other developers), running all those services locally (which might be complex and resource intense), or mocking them out (which will hide potential issues with the integration).

Using Kubernetes, Helm, Docker and some bash scripting, Marcin created a small system that allowed multiple independent deployments:

  1. Kubernetes, a Google Container Engine, was used as a base for deploying, managing, and orchestrating the system;
  2. Helm, which works like a package manager on top of Kubernetes, further allows you to: a) describe the relations between different parts of an application (e.g. an app that consists of independently developed backend APIs, a database and a fronted application); b) dynamically configure runtime parameters for those parts; and c) manage the lifecycle and release cycle.
  3. Docker was used for encapsulating and distributing services; and finally,
  4. A little bit of bash glued things together.

In the end, Marcin created a system that allows developers to automatically build their version of a particular service and instantly integrate it with the rest of the system in a production-like environment that is completely separated. In case that a service exposes some kind of an interface, such as an web-UI or an API, it is possible to get a DNS endpoint that points to it.

If you’re not a developer and you’re reading this, you might wonder why this is important. The answer to that question would be that it allows you to create both testing and production environments very quickly, without any overlap between them. In the case of our ventures here at WATTx, that gives them the possibility to set up different versions of their software in minutes, and lets them develop those separate versions. So, if our venture is working with Company A and Company B, it is very easy for them to set up and show their software to clients as well as develop custom versions of their software for a client’s specific need.


The next hackathon is planned for December and we’re already looking forward to it! Feel free to reach out to us if you have any further questions on a specific topic, or if you simply want to grab a cup of coffee and discuss technologies, data science or design.