Log Export

Cube Cloud allows exporting logs to third-party services, which allows users to leverage their existing log management tools. Cube Cloud uses Vector under the hood, which supports a wide range of destinations. To make use of Log Export, customers define a Vector sink configuration file and commit it to their repository.

Configuration

In the example below, we'll set up Cube Cloud to forward logs to a hosted Loki instance on Grafana Cloud.

Settings

To enable log export, first go to the deployment's settings page and turn it on:

Secrets

First, go to your deployment's environment variables, then add the authentication credentials for the Grafana Cloud-hosted Loki instance. We can define as many environment variables as we need, but we must prefix them with CUBE_CLOUD_LOG_EXPORT_ so that we can use them in our Vector configuration file later.

Vector

Now, enter Development Mode and add a file called vector.toml to your Cube project. Open the newly-created file, and then add any relevant sink configuration. We'll also want to use the environment variables we defined earlier:

[sinks.loki]
  type = "loki"
  inputs = [ "cubejs-server", "redis", "refresh-scheduler", "ext-db" ]
  endpoint = "https://logs-prod-us-central1.grafana.net"
[sinks.loki.encoding]
  codec = "json"
[sinks.loki.labels]
  app = "cube-cloud"
  env = "production"
[sinks.loki.auth]
  strategy = "basic"
  user = "$CUBE_CLOUD_LOG_EXPORT_LOKI_USER"
  password = "$CUBE_CLOUD_LOG_EXPORT_LOKI_PASS"

[sinks.loki_cubestore]
  type = "loki"
  inputs = [ "cubestore" ]
  endpoint = "https://logs-prod-us-central1.grafana.net"
[sinks.loki_cubestore.encoding]
  codec = "json"
[sinks.loki_cubestore.labels]
  app = "cube-store"
  env = "production"
[sinks.loki_cubestore.auth]
  strategy = "basic"
  user = "$CUBE_CLOUD_LOG_EXPORT_LOKI_USER"
  password = "$CUBE_CLOUD_LOG_EXPORT_LOKI_PASS"
[sinks.loki_cubestore.cubestore]
  levels = [ "trace", "info", "debug", "error" ]

Once done, save the file and commit it to your repository. Once you're back on the Overview page, you should now see a new "Log Export" resource:

Notes

Valid inputs

Valid values for input are

• cubejs-server
• redis
• refresh-scheduler
• ext-db
• warmup-job
• cubestore

Cube Store

The cubestore input supports a levels property, which allows filtering logs by type. Valid values are:

• trace
• info
• debug
• error

If levels is not specified, then only logs of type info and error will be sent.