With all of the focus on the COVID-19 global pandemic, this blog walks through the process of building a dashboard with real-time insights, not just with the numbers available, but also visually representing each country's flag and providing drill down data for India and Australia.
Enabling Insights from Open APIs
With all of the focus on the COVID-19 global pandemic, I was interested in building insights not just with the numbers available, but also representing each country's flag in the form of cards, and including individual insights for India and Australia. This blog walks you through the process I went through from choosing the right APIs, visualizing the data, creating the nodes through to verifying the APIs - it shows how simple it is to leverage open APIs to quickly build your own dashboard.
- Choosing the right APIs
- Visualizing the Data
- Creating the Nodes
- Using HTTP-IN Node
- HTTP Request Node
- TEMPLATE Node
- The HTML Code
- Link the URLs into Navigation Tab
- Wiring up the Nodes
- Deploying the Flows to Cloud
- Verifying the APIs
- Future Improvements
Choosing the right API’s
To get started, I made use of various open-source APIs related to COVID-19 available on the internet:
- http://corona-api.com/countries to fetch the global data of all countries available.
- https://api.rootnet.in/covid19-in/stats/latest to fetch the latest data from Indian States
- https://api.rootnet.in/covid19-in/contacts to fetch emergency contacts of each state in India
- https://api.infotorch.org/api/covid19/totals/ to fetch the latest data from the Australia States
Visualizing the Data
To visualize the data, I wanted to use a low-code tool that would let me build my dashboard application really quickly. There are many tools available on the market, this time I decided to try something new, and used Node-RED https://nodered.org/,
Creating the Nodes
The process of building the logic individually is called Nodes. The idea of bringing insights from the APIs comprises 3 key Nodes for each API and 2 additional Nodes for Logging.
- Node that exposes the API
- Node that queries the APIs to fetch data
- Node that holds the HTML template as an HTTP response
- Node that holds a text to log
- Node that Writes a file into a specific location
Using HTTP-IN Node
This node is used to build a relative URI on top of the Node-RED server.
Create a URL /covid19/global/report and choose the Method as GET.
Security: All the relative URI’s developed can be secured with a certificate and basic authentication. As we intend to expose this URI to the public, authentication is disabled in the security file. User specific access and groups can be defined in the security file.
HTTP Request Node
This Node Sends HTTP requests and returns the response.
Choose the Method as GET, as we fetch Data from the open Source APIs.
Paste the URL in the URL Section.
About the API: http://corona-api.com/countries
The API is open and generally uses a GET method to retrieve data, so didn’t require any major installation of certificates or tokens .
With the help of the template node, the JSON structure can be converted into a nice HTML dashboard. The template node is used for creating web pages and so generally contains HTML mark-up. It can be used for dynamically inserting data into a page using Mustache templates, which is the default configuration.
The example given is a template containing the following:
This is the payload: !
This will replace with the value of the message’s payload property.
By default, Mustache will replace certain characters with their HTML escape codes. To stop that happening, you can use triple braces: }.
The HTML code
The head section inside HTML contains two major parts:
- Importing a bootstrap template to make the site compatible to different screen sizes.
- Navigation bar to navigate between different URLs
The body section contains a div card that supports multiple devices. These class files are inherited from the bootstrap template imported in the Head Section
Dynamic Card Generation
The data for a number of countries in the response message has a standard JSON structure which comes in an array, the mustache template can generate multiple <div> cards at runtime, depending on the array size.
Embedding the flags inside the card
The header section of the card contains the country name and I wanted to display the flag under the name. A quick search on the internet, and I found some APIs that provide an image as a response, with just the country code as a parameter in the relative URI itself.
Link the URLs into Navigation Tab
Create an individual template for each API that we need to expose and refer them in the NAV button tab inside HTML.
Wiring up the Nodes
Now the Nodes have to be wired such that our exposed URLs can fetch data and render an HTML doc as a response in the browser.
Next, we create a similar set of Nodes with different URLs. For the rest of the APIs we have selected to visualize India, Australia, and India Emergency Contacts.
Deploying the Flows to Cloud
A set of Nodes fulfilling a set of functionality is called a Flow.
Built on top of a Node JS Server, Node-RED has built-in functionality to deploy flows with just a single-click, into its own server. This is great for building and testing the Flows in a local environment, as we test it multiple times to bring up the shape of UI. Once the Flows are developed, the code should be transferred to a Cloud environment, AWS, to view the reports publicly.
One of Node-REDs most loved features is that the flows can be easily exported in the form of JSON with just a click and can be imported into any node-red environment.
Bringing up the AWS instance
Create an AWS EC2 instance with the desired port opened to the public via security groups.
Install Node-RED along with plugins required from the Plugin Manager.
Export the flows from the dropdown menu .
The wizard brings up several options to export, as shown below.
Copy the Flow to the clipboard.
Login to Node-RED admin dashboard and from the dropdown menu on top right corner, import the nodes into the environment. Once all the flows are imported, click on deploy to run them.
Verifying the APIs
Once deployed, the URIs that were created earlier, can be accessed via the hostname and port that the Node-RED is running on.
Dashboard URL after deploying to AWS
Google analytics can be embedded into the HTML to track the URIs. As it might consume bandwidth on a free tier AWS account, I have enabled a file write node which logs the number of hits into a file.
- A common URI with parameters enabled can be used to minimize the number of URLs
- A common template can be used for all nodes instead of using multiple templates for each URI exposed.
- A light weight SQLite database could be used to store cached data, in the case where API’s cannot be accessed
With my work focussed primarily on integrating systems, I have always wanted to build a realtime web app and so I have used this opportunity to access public data via open APIs and leverage a low code platform to quickly build out a COVID-19 dashboard.
Building up the URIs and connecting to the APIs took no time at all. I spent most of my time building up the template in such a way it is compatible across multiple devices.
One of my key learnings was through the use of the Flags API. API’s are not just for data, APIs are for everything - be it an image, a function, or a website. API’s are key to rapidly surface, share and visualize much more than just data.