Collecting WebP Cloud Account Data with Prometheus

As a paying user of WebP Cloud, we greatly appreciate your support! (What, you’re not a paying user yet? That means your usage isn’t high enough for our free plan to meet your needs!)

Users might have some concerns:

• How much quota did I use today?
• Can I integrate WebP Cloud data directly into my monitoring dashboard?
• How much traffic did my proxy output daily, and what is the distribution of requests?

WebP Cloud, as a SaaS service, provides detailed access data to users of all plans:

Yes, users of all plans can see detailed data, unlike some providers where you need to buy the Pro version to see… sampled data?

However, some users might find this data not visually appealing enough or may want to integrate it into their own dashboard. Previously, you could directly obtain all the data through our API, process it slightly, and write it into your own database.

But that’s still not convenient enough! Every user with their own dashboard needs to manually write the import logic and set up scheduled tasks, leading to significant resource/time waste. Why can’t WebP Cloud provide a universal interface to deliver the data directly?

/metrics

Here comes the /metrics endpoint. Currently, we have launched two endpoints:

• /v1/user/metrics
• /v1/proxy/<proxy_uuid>/metrics

These endpoints respectively display some information about the current user and basic information about the corresponding proxy. For detailed parameters, please refer to the documentation:

• https://docs.webp.se/webp-cloud/api/#get-user-metrics
• https://docs.webp.se/webp-cloud/api/#get-proxy-metrics

Prometheus

Let’s see how to use these two endpoints to quickly import data from WebP Cloud into your own dashboard!

The /metrics endpoint conforms to the Prometheus Exposition Format specification. Since this endpoint is part of our external API, you need to pass an API Key for authentication. We found that Prometheus natively supports the Authorization header (our API supports both token and api-key headers), so we additionally supported the Authorization header. Assuming your local monitoring stack is deployed as follows:

version: '3'

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - ./prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:9.5.20
    ports:
      - "127.0.0.1:3000:3000"
    volumes:
      - ./grafana_data:/var/lib/grafana

Your prometheus.yml file would look like this (in this example, we monitor the remaining quota information of our account and the information of the proxy with ProxyID afc8fd63-5ce6-48d2-a80b-edbf2ee8b6dc. Assuming your WebP Cloud API Key is dd25xxxx-xxxx-xxxx-xxxx-xxxxxxxx96a3):

The API Key can be found in the upper right corner of the dashboard

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'webp_cloud_user'
    metrics_path: /v1/user/metrics
    static_configs:
      - targets:
        - webppt.webp.se
    authorization:
      type: ""
      credentials: "dd25xxxx-xxxx-xxxx-xxxx-xxxxxxxx96a3"
  - job_name: 'webp_cloud_proxy'
    metrics_path: /v1/proxy/afc8fd63-5ce6-48d2-a80b-edbf2ee8b6dc/metrics
    static_configs:
      - targets:
        - webppt.webp.se
    authorization:
      type: ""
      credentials: "dd25xxxx-xxxx-xxxx-xxxx-xxxxxxxx96a3"

With this setup, we can see the data we want to scrape in Prometheus.

Next, you can integrate it into any dashboard, such as Grafana:

Enjoy~