Recently, Cube Core version 0.36 was released. It includes long-awaited features and numerous enhancements to Cube Core, such as:

Also, some previously deprecated features were removed, so there are a few breaking changes to review before upgrading, such as:

Finally, the documentation has notably expanded as well.

New in the UI

Playground 2.0 and Chart Prototyping

Releasing Playground 2.0 and Chart Prototyping is by far the biggest change to Cube Core ever, at least visually.

Playground now provides a refreshed BI-like experience and UI for running queries and validating the data model while you're building it. Chart Prototyping enables you to go from a query in Playground to a minimal yet complete application source code that you can download and modify to build an embedded analytics experience. These features were both part of our public roadmap.

Here's how they work:

And here's what some of Cube community members think about them:

Read more in the blog post and check the documentation: Playground, Chart Prototyping.

New in data modeling

Custom time dimension granularities

Now Cube Core provides the first-class support for calculations involving fiscal and custom time dimension granularities. With this feature, you're able to define granularities such as a fiscal year, a fiscal quarter, or a week starting on Sunday for any time dimension, query them via all supported APIs, and have these queries accelerated by pre-aggregations. This feature was part of our public roadmap.

Here's an example of data model with a few custom granularities:

cubes:
  - name: custom_granularities
    sql: >
      SELECT '2024-01-15'::TIMESTAMP AS timestamp UNION ALL
      SELECT '2024-02-15'::TIMESTAMP AS timestamp UNION ALL
      SELECT '2024-03-15'::TIMESTAMP AS timestamp UNION ALL
      SELECT '2024-04-15'::TIMESTAMP AS timestamp UNION ALL
      SELECT '2024-05-15'::TIMESTAMP AS timestamp


    dimensions:
      - name: timestamp
        sql: timestamp
        type: time


        granularities:
          - name: sunday_week
            interval: 1 week
            offset: -1 day


          - name: fiscal_year
            interval: 1 year
            offset: -3 months


      - name: timestamp__sunday_week
        sql: "{timestamp.sunday_week}"
        type: time


      - name: timestamp__fiscal_year
        sql: "{timestamp.fiscal_year}"
        type: time

Read more in the blog post and check the documentation: time dimension concept, new granularities parameter, proxy dimensions with granularities. Also, check the new recipe on implementing custom time dimension granularities.

Improvements to the data model generation

Generating the data model based on the database schema is extremely useful for bootstrapping Cube Core applications, so we've made a few improvements to it.

Now Cube Core will do its best to detect primary and foreign keys in the database schema, generate composite primary keys in the data model as well as detect boolean columns and generate dimensions for them.

New in the SQL API

Query pushdown

We have introduced the query pushdown in the SQL API in Cube Core v0.35.40 and have been polishing it ever since. Query pushdown drastically improves the support for various BI tools and takes it to the next level.

With this feature, Cube now can seamlessly transpile an incoming SQL query, in Postgres dialect, generated by your BI tool, to the SQL dialect of a supported upstream data source (e.g., Snowflake, Databricks, BigQuery, or Redshift). Thanks to the e-graph technology, all possible execution branches are evaluated in parallel, so that Cube can choose the most optimal query plan:

sql-api-diagram

Query pushdown has long been in use by lots of Cube Core users and Cube Cloud customers, and it does wonders with regards to the BI support. You can hear from one of the early adopters:

Query pushdown in the SQL API enabled us to create calculated dimensions and measures on the fly in Preset (using the custom SQL functionality). This enabled prototyping of new dashboards without first having to update the data model in Cube.

Guy DawsonGuy DawsonEngineer at Permutive

Read more in the blog post and check the documentation: SQL API fundamentals, SQL API query format, SQL API reference.

Streaming mode

SQL API can now leverage an optimization, also known as the streaming mode, that would stream the result set from the data source rather than load it in batches.

With this feature, the SQL API provides a way to override the maximum row limit in the result set, so that your SQL API query can potentially return an unlimited number of rows, e.g., a million rows.

Read more in the documentation: streaming mode in the SQL API, default row limit, CUBEJS_DB_QUERY_LIMIT environment variable.

Default ordering opt-in

By default, Cube applies an ordering to any query that does not set it explicitly, using a set of heuristics that work well for most data visualization purposes. However, this behavior can have performance implications.

Since Cube Core v0.36.0, this behavior does not apply to the SQL API queries without an ORDER BY clause; they do not get the default ordering. However, you can set the CUBESQL_SQL_NO_IMPLICIT_ORDER environment variable to false to opt back in for the default ordering for the SQL API queries.

Read more in the documentation: default ordering.

Max session limit

Each concurrent connection to the SQL API consumes some resources and attempting to establish too many connections at once can lead to an out-of-memory crash.

Now you can use the CUBEJS_MAX_SESSIONS environment variable to adjust the session limit. By default, it's set to 1024 connections.

New in the REST API

Default ordering opt-out

By default, Cube applies an ordering to any query that does not set it explicitly, using a set of heuristics that work well for most data visualization purposes. However, this behavior can have performance implications.

Now you can opt out from default ordering for the REST API queries by passing an empty array ([]) object as the order parameter of your query.

Read more in the documentation: default ordering.

New in Cube Store

Performance optimizations

Cube Store is the purpose-built query queue, in-memory cache, and pre-aggregation storage for Cube, and it is instrumental to the performance of Cube deployments.

The latest version of Cube Core implements a few substantial performance optimizations to Cube Store, including the usage of the jemalloc allocator for RocksDB, an embeddable key-value storage used internally in Cube Store. As a result, Cube Store now has a visibly reduced memory consumption.

This has already been noted in the Cube community:

New in data source support

Improvements to Databricks support

Databricks is widely used with Cube and it's important that it has the first-class support. A few recent improvements to it include:

  • count_distinct_approx measures can be used with Databricks.
  • HyperLogLog sketches can be loaded from Databricks to enable the usage of count_distinct_approx measures with pre-aggregations.
  • Pre-aggregations can be built even if Cube only has read-only access to Databricks.

Read more in the documentation: Databricks support, count_distinct_approx measures.

Improvements to ClickHouse support

ClickHouse is also widely used with Cube.

count_distinct_approx measures can now be used with ClickHouse as well, powered by the efficient uniq function.

Read more in the documentation: ClickHouse support, count_distinct_approx measures.

Support for RisingWave

Streaming data sources are among many ones supported by Cube, and we're always glad to expand that support.

Cube now works with RisingWave, an innovative distributed streaming database that enables processing and management of real-time data.

Read more in the blog post and check the documentation: RisingWave support.

BREAKING: Changes to Docker images

Version upgrades

Even though Cube Core is distributed as Docker images, Cube uses Python and Node.js internally as runtime environments for the code of its configuration, dynamic data models, and dependencies. So, it is important to make sure that your code and dependencies are compatible with the versions in the runtime:

  • Python has been upgraded to v3.11.
  • Node.js has been upgraded to v20.17. Also, support for Node.js v16 has been removed; support for Node.js v18 has been deprecated.
  • Java Development Kit, used by some database drivers, has been upgraded to v17.
  • OpenSSL has been upgraded to v3.
  • Finally, the operating system of the Docker images has been upgraded to Debian v12 (bookworm).

Also, some work has been done to reduce the size of Docker images.

BREAKING: Removed features

Using Redis for in-memory cache and queue

In previous versions, Redis has been used as the storage for in-memory cache and queue. However, it has been deprecated since Cube Core v0.32.

Now, support for Redis has been removed. It is recommended to remove Redis from your deployments and use Cube Store since it's the purpose-built query queue, in-memory cache, and pre-aggregation storage for Cube.

Read more and follow the migration instructions in this blog post.

/v1/run-scheduled-refresh endpoint

In previous versions, Cube supported working in serverless environments. However, it has been deprecated since Cube Core v0.31.64. Also, the /v1/run-scheduled-refresh API endpoint, that was only useful for serverless deployments, has been deprecated since v0.35.

Now, the /v1/run-scheduled-refresh endpoint has been removed. It is recommended to use the Orchestration API and the /v1/pre-aggregations/jobs endpoint instead.

Learn more about the Orchestration API in the blog post and check the documentation.

USER_CONTEXT context variable

You can use a number of context variables when defining your data model. The USER_CONTEXT variable has been deprecated since v0.26. Note that its alias, the SECURITY_CONTEXT variable, has also been deprecated since v0.33.

Now, the USER_CONTEXT variable has been removed. It is recommended to use the COMPILE_CONTEXT variable instead of USER_CONTEXT or SECURITY_CONTEXT.

Read more in the documentation: COMPILE_CONTEXT variable.

checkAuthMiddleware configuration option

Before Cube has started to be distributed as Docker images, the checkAuthMiddleware configuration option could be used to control the authentication process. However, it has been deprecated since Cube Core v0.26.

Now, the checkAuthMiddleware configuration option has been removed. It is recommended to use the checkAuth option instead.

New in the documentation

With the evolution of Cube Core, its documentation evolves as well, so there are a few useful changes the documentation to know:

Finally, links to community-maintained resources has been added:

What's next?

We are sure that Cube Core v0.36 is the best Cube Core release ever, and we wholeheartedly encourage you to upgrade. You can also give it a try for free in Cube Cloud where this release is available as fully-managed Cube deployments.

As always, please share your feedback and thoughts with us via the Slack community. It really helps us drive the Cube development further.