Log Export

This feature is only available on an Enterprise plan. Contact us for more details.

Cube Cloud allows exporting logs to third-party services, which allows users to leverage their existing log management tools. Cube Cloud uses Vector under the hood, which supports a wide range of destinations. To make use of Log Export, customers define a Vector sink configuration file and commit it to their repository.

In the example below, we'll set up Cube Cloud to forward logs to a hosted Loki instance on Grafana Cloud.

Settings

To enable log export, first go to the deployment's settings page and turn it on:

Cube Cloud Settings Screen showing Log Export toggle

Secrets

Only environment variables prefixed with CUBE_CLOUD_LOG_EXPORT_ can be used in vector.yml.

First, go to your deployment's environment variables, then add the authentication credentials for the Grafana Cloud-hosted Loki instance. We can define as many environment variables as we need, but we must prefix them with CUBE_CLOUD_LOG_EXPORT_ so that we can use them in our Vector configuration file later.

Cube Cloud Settings Screen showing Log Export environment variables

Vector

Now, enter Development Mode and add a file called vector.toml to your Cube project. Open the newly-created file, and then add any relevant sink configuration. We'll also want to use the environment variables we defined earlier:

[sinks.loki]
  type = "loki"
  inputs = [ "cubejs-server", "redis", "refresh-scheduler", "ext-db" ]
  endpoint = "https://logs-prod-us-central1.grafana.net"
[sinks.loki.encoding]
  codec = "json"
[sinks.loki.labels]
  app = "cube-cloud"
  env = "production"
[sinks.loki.auth]
  strategy = "basic"
  user = "$CUBE_CLOUD_LOG_EXPORT_LOKI_USER"
  password = "$CUBE_CLOUD_LOG_EXPORT_LOKI_PASS"

[sinks.loki_cubestore]
  type = "loki"
  inputs = [ "cubestore" ]
  endpoint = "https://logs-prod-us-central1.grafana.net"
[sinks.loki_cubestore.encoding]
  codec = "json"
[sinks.loki_cubestore.labels]
  app = "cube-store"
  env = "production"
[sinks.loki_cubestore.auth]
  strategy = "basic"
  user = "$CUBE_CLOUD_LOG_EXPORT_LOKI_USER"
  password = "$CUBE_CLOUD_LOG_EXPORT_LOKI_PASS"
[sinks.loki_cubestore.cubestore]
  levels = [ "trace", "info", "debug", "error" ]

Once done, save the file and commit it to your repository. Once you're back on the Overview page, you should now see a new "Log Export" resource:

Cube Cloud Overview screen showing Log Export resource

Valid inputs

Valid values for input are

  • cubejs-server
  • redis
  • refresh-scheduler
  • ext-db
  • warmup-job
  • cubestore

The cubestore input supports a levels property, which allows filtering logs by type. Valid values are:

  • trace
  • info
  • debug
  • error

If levels is not specified, then only logs of type info and error will be sent.

Did you find this page useful?