What is Graphana?
Grafana is an open source visualizing tool which can be used with different time series databases.

What is Elasticsearch ?
ElasticSearch is NOSQL database or a search engine based on Lucene

Why Elasticsearch with Graphana ?

In my current Dev cluster , Elasticsearch is used as a repository for storing the logs and kibana is used for visualizing the logs and monitoring Elasticsearch. However Kibana doesn’t provide rich graphical interface, when it comes to metrics. Therefore , the focus here is to check if Graphana monitoring capabilities can be leveraged with the data( In this context, its Dev cluster logs ) present in Elasticsearch.

Why Graphana ?
Grpahana is already part of our infrastructure , with its rich graphical interface dashboards , its our choice of monitoring the components present in Dev cluster.

How Graphana interacts with Elasticsearch ?

There are two ways that Graphana can interact with Elastic search

1) Prometheus Datasource : [ Elasticsearch -> Prometheus Exporter -> Prometheus -> Graphana ]

Using Prometheus exporters, Elasticsearch data can be scraped by Prometheus and then Prometheus datasource can be used by Graphana dashboards .

2) Elasticsearch Datasource :[ Elasticsearch -> Graphana ]

Elasticsearch can be attached as a Datasource with Graphana dashboards.

POC: ( Elasticsearch DataSource )

This POC is to explore the capabilities of Graphana having elastic search as datasource with some sample data.Graphana provides different configurations for different datasources, as per the datasource capabilities.

In the case of Elasticsearch datasource, Graphana provides three importants fields

Query – Can have any Lucene query
Metric – Aggregate functions.
GroupBy – For Grouping .

Multiple queries and metrics can be defined.

Configuring Datasource: ( Attached Screenshot for reference )

1) Add DataSource:
Settings-> Datasource ( Select Elasticsearch)
2) Mandatory Configuration Parameters
Elasticsearch server URL, Access Mode, Index Pattern , TImeField Name

With the above two steps, Elasticsearch is ready to be used as Datasource in Graphana.

Configuring Dashboard: ( Attached Screenshot for reference )

Once datasource is created , next step is to create the dashboard and attach the datasource created with the above steps.

1) Select new Dashboard.
1) Select required Panel by going to Dashboards.
( Eg : Bar Graph, Pie Chart etc )

2) Right click on the panel header and click edit.
3) Select Metrics Tab and then select the datasource you are interested in. ( Select the Datasource
created in the above step. )
4) Finally select the required metrics and Groupby Attributes.

5) Updated Dashboard displayed.

Conclusion
To make use of elasticsearch as datasource with Graphana, We need to figure out Elasticsearch index pattern we are interested in. Followed by that , we need to identify the required fields/attributes in those patterns and configure the dashboard with the required queries/metrics on the fields identified .