Leveraging COVID-19 APIs to build a global dashboard

With all of the focus on the COVID-19 global pandemic, this blog walks through the process of building a dashboard with real-time insights, not just with the numbers available, but also visually representing each country's flag and providing drill down data for India and Australia.

Enabling Insights from Open APIs

With all of the focus on the COVID-19 global pandemic, I was interested in building insights not just with the numbers available, but also representing each country's flag in the form of cards, and including individual insights for India and Australia.  This blog walks you through the process I went through from choosing the right APIs, visualizing the data, creating the nodes through to verifying the APIs - it shows how simple it is to leverage open APIs to quickly build your own dashboard.

Choosing the right API’s

To get started, I made use of various open-source APIs related to COVID-19 available on the internet:

Visualizing the Data

To visualize the data, I wanted to use a low-code tool that would let me build my dashboard application really quickly. There are many tools available on the market, this time I decided to try something new, and used Node-RED https://nodered.org/,

Creating the Nodes

The process of building the logic individually is called Nodes. The idea of bringing insights from the APIs comprises 3 key Nodes for each API and 2 additional Nodes for Logging.

  • Node that exposes the API
  • Node that queries the APIs to fetch data
  • Node that holds the HTML template as an HTTP response
  • Node that holds a text to log
  • Node that Writes a file into a specific location

 

COVID Report

 

Using HTTP-IN Node

This node is used to build a relative URI on top of the Node-RED server.

Create a URL /covid19/global/report and choose the Method as GET.

Security: All the relative URI’s developed can be secured with a certificate and basic authentication. As we intend to expose this URI to the public, authentication is disabled in the security file. User specific access and groups can be defined in the security file.

HTTP IN node

 

HTTP Request Node

This Node Sends HTTP requests and returns the response.

Choose the Method as GET, as we fetch Data from the open Source APIs.

Paste the URL in the URL Section.

HTTP Request Node

 

About the API: http://corona-api.com/countries

The API is open and generally uses a GET method to retrieve data, so didn’t require any major installation of certificates or tokens .

TEMPLATE Node

With the help of the template node, the JSON structure can be converted into a nice HTML dashboard. The template node is used for creating web pages and so generally contains HTML mark-up. It can be used for dynamically inserting data into a page using Mustache templates, which is the default configuration.

The example given is a template containing the following:

This is the payload: !

This will replace with the value of the message’s payload property.

By default, Mustache will replace certain characters with their HTML escape codes. To stop that happening, you can use triple braces: }.

Mustache supports simple loops on lists. For example, if msg.payload contains an array of names, such as: ["Nick", "Dave", "Claire"], the template will create an HTML list of the names:.The node will set the configured message or context property with the result of the template. If the template generates valid JSON or YAML content, it can be configured to parse the result to the corresponding JavaScript Object.

 

TEMPLATE node

The HTML code

The head section inside HTML contains two major parts:

  1. Importing a bootstrap template to make the site compatible to different screen sizes.
  2. Navigation bar to navigate between different URLs

The body section contains a div card that supports multiple devices. These class files are inherited from the bootstrap template imported in the Head Section

Country dashboard

 

Dynamic Card Generation

The data for a number of countries in the response message has a standard JSON structure which comes in an array, the mustache template can generate multiple <div> cards at runtime, depending on the array size.

Dynamic card generation syntax

Embedding the flags inside the card

The header section of the card contains the country name and I wanted to display the flag under the name. A quick search on the internet, and I found some APIs that provide an image as a response, with just the country code as a parameter in the relative URI itself.

APIs with flag image response based on country code

 

 

 

 

 

 

 

 

 

 

 

 

Link the URLs into Navigation Tab

Create an individual template for each API that we need to expose and refer them in the NAV button tab inside HTML.

Navigation tab

 

 

 

 

 

 

Wiring up the Nodes

Now the Nodes have to be wired such that our exposed URLs can fetch data and render an HTML doc as a response in the browser.

Next, we create a similar set of Nodes with different URLs. For the rest of the APIs we have selected to visualize India, Australia, and India Emergency Contacts.

Wiring up the nodes

 

Deploying the Flows to Cloud

A set of Nodes fulfilling a set of functionality is called a Flow.

Built on top of a Node JS Server, Node-RED has built-in functionality to deploy flows with just a single-click, into its own server. This is great for building and testing the Flows in a local environment, as we test it multiple times to bring up the shape of UI. Once the Flows are developed, the code should be transferred to a Cloud environment, AWS, to view the reports publicly.

One of Node-REDs most loved features is that the flows can be easily exported in the form of JSON with just a click and can be imported into any node-red environment.

Bringing up the AWS instance

Create an AWS EC2 instance with the desired port opened to the public via security groups.

Install Node-RED along with plugins required from the Plugin Manager.

Export the flows from the dropdown menu .

The wizard brings up several options to export, as shown below.

 

Export nodes

 

Copy the Flow to the clipboard.

Login to Node-RED admin dashboard and from the dropdown menu on top right corner, import the nodes into the environment. Once all the flows are imported, click on deploy to run them.

Copy flow

 

Verifying the APIs

Once deployed, the URIs that were created earlier, can be accessed via the hostname and port that the Node-RED is running on.

EX: http://localhost:xxx/covid19/global/report

Global dashboard

 

Dashboard URL after deploying to AWS

http://ec2-18-221-178-197.us-east-2.compute.amazonaws.com:1981/covid19/global/report

Enabling metrics

Google analytics can be embedded into the HTML to track the URIs. As it might consume bandwidth on a free tier AWS account, I have enabled a file write node which logs the number of hits into a file.

Future Improvements

  • A common URI with parameters enabled can be used to minimize the number of URLs
  • A common template can be used for all nodes instead of using multiple templates for each URI exposed.
  • A light weight SQLite database could be used to store cached data, in the case where API’s cannot be accessed

Summary

With my work focussed primarily on integrating systems, I have always wanted to build a realtime web app and so I have used this opportunity to access public data via open APIs and leverage a low code platform to quickly build out a COVID-19 dashboard.

Building up the URIs and connecting to the APIs took no time at all. I spent most of my time building up the template in such a way it is compatible across multiple devices.

One of my key learnings was through the use of the Flags API. API’s are not just for data, APIs are for everything - be it an image, a function, or a website. API’s are key to rapidly surface, share and visualize much more than just data.