Varnish Monitoring Structure, Setup and Dashboard Interpretation

6 min readSep 19, 2022

Greetings to everyone 🖐 Today, I will talk about the tracking structure of Varnish, a cache mechanism that is frequently used in more than one e-commerce site and preferred more than other tools with its integration and ease of use.

In this article, what is Varnish ? what advantages will it bring us? how to do varnish monitoring? I will try to explain step by step about these. If you’re ready, let’s start 😊.

What is Varnish ?

Varnish is an HTTP reverse proxy used to speed up websites. A reverse proxy is a proxy that appears to clients as the principal server. Varnish keeps the web pages in cache (ram or file) and when the client makes a request, if the answer has been requested before, it returns the same answer and responds to the user without going to the actual server. In this way, situations that tire the server such as database operations are reduced.

The Advantages of Varnish Monitoring

There are multiple advantages that Varnish monitoring brings to us. First of all, let’s say that we will make the monitoring structure using Prometheus and Grafana tools. Now let’s take a look at what advantages the monitoring structure gives us.

1 - We will now be able to view the Hit, Miss values related to Varnish on the Dashboard.
2 - We will be able to update varnish by examining the values on the chart.
3 - We will be able to examine the effects on the site in more detail.
4 - On the code prepared by the developer, we can see in more detail which requests are kept on Varnish and we can intervene.

Now let’s move on to the subject that everyone is curious about. How to integrate Varnish monitoring and which metrics should be examined after dashboarding?

Setup of the Varnish Structure

In the first step, it is necessary to get varnish-related metrics to create a varnish dashboard. Prometheus Exporter, Fluentd can be used for this. Then we can dashboard the metrics we get with different monitoring tools. I tried to get the metrics with Prometheus Exporter and then dashboard in Grafana and got positive results for the first step. Now I will share with you the steps I followed.

Pulling Prometheus Metrics

For this, we can follow the attached Prometheus Exporter Github endpoint. https://github.com/jonnenauha/prometheus_varnish_exporter Here we can download the latest Release in the Releases step.

I am also attaching the link for the download:
https://github.com/jonnenauha/prometheus_varnish_exporter/releases/download/1.6/prometheus_varnish_exporter-1.6.linux-amd64.tar.gz

Then we have to do the attached steps on the K8S in the example Varnish pod:

1 - apt-get update
2 - cd
3 - curl -LO https://github.com/jonnenauha/prometheus_varnish_exporter/releases/download/1.6/prometheus_varnish_exporter-1.6.linux-amd64.tar.gz
4 - tar -xzvf prometheus_varnish_exporter-1.6.linux-amd64.tar.gz
5 - cd prometheus_varnish_exporter-1.6.linux-amd64
6 - ./prometheus_varnish_exporter

We need to add the attached addition to the Varnish service in k8s cluster:

ports: 
- name: metrics
  port: 9131
  protocol: TCP
  targetPort: 9131

When we do the above steps, we will now be getting the Varnish metrics. In the next step, we need to send these metrics to Prometheus. For this, we need to redirect to varnish-svc specific to the brand using the local network. For this, it is necessary to add the attached ‘job’ to the Prometheus configmap.

job_name: 'varnish-svc
metrics_path: metrics
scheme: http  
static_configs:
    - targets: ['varnish-svc.<web_site_name>.svc.cluster.local:9131']

Then we can go to Prometheus and view the logs coming.

Dashboarding Metrics in Grafana

To dashboard Grafana metrics, we can import metric dashboard number 9903 from Grafana’s own site. Afterwards, we will send a request to the site and see that the metrics related to Varnish have been added to the dashboard.

You can view the outputs in Grafana in the various photos attached.

We can interpret the Varnish on the Dashboard here.

Using Grafana Dashboards

Cache Hit Rate (cache hit/cache miss):

Treats as cache_hit / cache_miss in Varnish. The higher the Cache Rate, the more effective the varnish. For example, if the Cache Hit is above 80%, it means that the majority of requests are handled by an object already in varnish and no backend requests are made.

Frontend Request: Number of requests sent to the frontend (per second)
Backend Request: Number of backend requests (per second)
Sessions: Opened session (per second)
Loaded VMODs: Background integrations (extensions) for Varnish

Requests: Frontend ve Backend’e atılan istek sayısını göstermektedir.

Network: Shows Network information passed by Frontend and Backend on Brand. (Kbit)

Expired: Shows the number of objects that have expired due to TTL.
LRU nuked: Shows the number of objects that Varnish has removed from the cache due to lack of space.
LRU moved: Returns the update information of the LRU list. Indicates the number of transaction transactions made in the LRU list.
Mem used: Displays the memory usage values.

Session herd: It is the counter of how many sessions are in progress.
Thread: Shows the number of connections opened in the background and information on each new request.
Banning: Blocks requests sent on the frontend. This operation is used when you want to remove many objects at once.
Ban lurker: It provides us a faster usage by caching some operations in the back that extend the TTL time that is overlooked when loading any object. At the same time, it provides support to us in deleting long-running unnecessary processes in background requests.

Threads, Bans, Size of Ban Lists and Ban Lurker

Today we have seen step by step with you what the Varnish structure is. We examined what advantages it provides for us and of course, we commented on the dashboards we obtained by installing it.

3I hope this is an article that you will love and enjoy reading. Happy reading everyone. See you in my next articles… 😊