Resource Metrics

As of CrateDB Cloud API 0.40.0, metrics for customer resources, such as the number of database queries in a CrateDB Cloud cluster are retrieved from Prometheus as documented in CrateDB Cloud Telemetry.

Due to the hard dependency on Prometheus, testing this feature on a given layer is not trivial. This guide aims at making the metrics for a CrateDB Cloud cluster running in one of the cratedb-dev.cloud regions available to a local setup.

First, pick any cluster running in one of our development regions. At the time of writing these are bregenz.a1 and eastus.azure. We’ll refer back to that region as theRegion later. The only required item you need is its ID. We’ll refer back to it as theClusterID.

You can find any live cluster by running:

$ kubectl --context k8s-dev.theRegion get pods --all-namespaces -l 'component_type=cratedb'

What you’re looking for is the “cryptic” ID in the NAME column. Format it as a UUID version 4 (8-4-4-4-12 characters): turn 273d9588bef84f46a3526384c4e7b6f7 into 273d9588-bef8-4f46-a352-6384c4e7b6f7.

Next, ensure you have a project in your local CrateDB Cloud that is accessible to your user. When you log in to CrateDB Cloud locally (either running in Docker with docker-compose or ‘natively’) you should be able to see it in the project list. You need that project’s primary key. We’ll refer back to it as theProjectID.

Next, you need to create a cluster with theClusterID in the project with theProjectID locally. Since the cluster primary key is auto-generated, you’ll need to do that through the database directly.

From the crate/cloud repository run these commands:

$ docker-compose up -d brain
Starting cloud_brain_1 ... done
$ docker-compose exec brain crash

You’ll be presented with a SQL shell on your local brain setup.

It’s time to insert the new cluster into the database. Please remember to replace the values for theClusterID and theProjectID with the corresponding object’s ID.

INSERT INTO "core"."clusters" (
   "id", "project_id", "name", "username", "num_nodes", "hardware_specs", "crate_version", "product_name", "product_tier", "product_unit", "channel", "origin"
) VALUES (
   'theClusterID', 'theProjectID', 'my-metrics-cluster', 'defaultuser',  '3', {"memory_per_node"='7Gi', "memory_per_node_bytes"=7516192768, "cpus_per_node"=2, "disk_size_per_node"='256Gi', "heap_size"='3500M', "heap_size_bytes"=3670016000, "disk_size_per_node_bytes"=274877906944}, '4.1.2', 'dev', 'basic', 0, 'stable', 'cloud'
);

If all has gone well, you should now be able to see that cluster in your project overview.

As a last step, you will need to port-forward the Prometheus instance running in theRegion to your computer. If you’re running CrateDB Cloud API locally, that is, outside of Docker, binding Prometheus to localhost:9090 or 127.0.0.1:9090 will be sufficient. Remember to adjust the value for theRegion.

$ kubectl --context k8s-dev.theRegion -n monitoring port-forward cluster-monitoring-prometheus-0 9090 --address 127.0.0.1
Forwarding from 127.0.0.1:9090 -> 9090

However, if you run CrateDB Cloud API inside Docker with docker-compose (docker-compose up -d cloud-api), you will need to bind the Prometheus to the network interface that your Docker setup is using. Oftentimes, especially on macOS, that could be IP address in your local network: for example 192.168.1.123 or the docker-compose bridge or sub-network. The command to do so is basically the same:

$ kubectl --context k8s-dev.theRegion -n monitoring port-forward cluster-monitoring-prometheus-0 9090 --address 192.168.1.123
Forwarding from 192.168.1.123:9090 -> 9090

Depending on how you do the port-forwarding, you’ll need to update the Prometheus URL in the database for the regions you’re using:

# When you run Cloud API locally, like this:
$ docker-compose exec brain crash -c "UPDATE core.regions SET prometheus_url='http://127.0.0.1:9090';"
# When you run Cloud API inside Docker, like this:
$ docker-compose exec brain crash -c "UPDATE core.regions SET prometheus_url='http://192.168.1.123:9090';"

Once that port-forward is set up, you should be able to navigate to the cluster’s metrics API endpoint. Again, remember to replace theClusterID accordingly: