Grafana Dashboards for Prometheus Metrics

This is a short, last article to complete the installation of a comprehensive Monitoring and Alerting system for cloud-native applications with Prometheus. In the previous articles we looked into the concepts and the steps to setup and integrate Prometheus server, alert-manager, Prom2teams and a dead man’s switch. So now we are going to setup our Dashboards for the monitoring system using Grafana to give a graphical representation for system’s alert rules.

Image for post
Image for post
Grafana

Why Grafana?

Grafana is an opensource, cloud-native, highly available, fast, Software as a Service(SaaS) metrics platform. If can work with several time-series data storage systems and fully compatible with Prometheus matrices. Grafana is primarily designed for analyzing and visualizing metrics such as system CPU, disk, memory and I/O utilization. But we can make use of it to create dashboards for any of our custom metrics available in Prometheus.

  • opensource
  • can create graphs and dashboards easily
  • fast, client-side rendering even over long ranges of time
  • work with multiple time-series data stores (eg: Graphite, Prometheus, Elasticsearch, InfluxDB, MySQL etc)

Grafana has it’s built-in alerting engine that allows users to create conditional rules to dashboard panels to triggered alerts to a notification endpoint of your choice (e.g. email, PagerDuty, Slack, custom webhooks etc). But in our setup we used Prometheus alert manager for alerting which is a better reliable tool specifically designed for this particular functionality with more alerting features such as alert grouping, repetitions etc.

Integration of Grafana with Prometheus

You can easily startup a Grafana server by downloading the binaries for your operating system and running it even without installing.

Once the Grafana server is up and running, the first step is to connect Prometheus server as a Grafana data source.

Image for post
Image for post
Grafana data sources

Grafana support integration with multiple data stores. In our case, we will be using Prometheus data source as the time series database.

Image for post
Image for post
Prometheus data source configuration

Provide your Prometheus server URL for the data source and test the connectivity using the test button provided in the same page. If you get a successful connection, then let’s create a new dashboard.

You can create dashboards,

  • manually by defining queries for each graph
  • importing an existing template

Below is how to create a dashboard manually by defining each and every data required to be displayed in the graph.

Image for post
Image for post
Create dashboards

You can directly use Prometheus queries in Grafana. So that is so easy to create these dashboards as long as you have collected all the required matrices in Prometheus server.

You can create as may as dashboards you need to cover every aspect of your application system. It provides several visualization plugins as Pie charts, graphs, tables, clocks and many more. You will get extra plugins for Grafana with the Enterprise version.

Image for post
Image for post
Sample Grafana Dashboard

Alerting from Grafana

You can also make use the the alerting feature provided with Grafana if your system doesn’t have an explicit alerting component. Alert rules are configured per visualization basis which mean we can configure alerts for each and every graph we create in a dashboard. But the Prometheus alert manager which we configured in the previous article is a far better alert manager with more features like alert grouping, re-alerting for unresolved etc.

Image for post
Image for post
Alert Configuration

The above is a sample alert trigger for average value of the query A created above in the image ‘Create dashboards’ exceeding a limit of 5 counts for a duration of 5 minutes. Similarly you can create any number of alerts for the Grafana queries created in the given visualization.

You can create more tech blogs here.

Thanks!

Senior Software Engineer | BSc (Hons) Engineering | CIMA | Autodidact | Knowledge-Seeker

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store