Adnan Rahic

GraphQL

Building extremely fast, low-latency GraphQL APIs with Apollo and Cube.

GraphQL federation example with Apollo Federation and Apollo GraphOS

Use Hasura Remote Schemas to build a low-latency data application.

GraphQL federation with Hasura GraphQL Engine

We often hear from our customers that they want the ability to create data models in languages other than JavaScript—specifically YAML and Python. Today, we’re excited to release a public preview for YAML data modeling in Cube. This feature is available in both Cube Cloud and Cube Core.

Our goal is to support all the core features of Cube’s data modeling, as well as caching and multi-tenancy configurations, within YAML, and later, Python. 

YAML and Python are universally adopted in the modern data stack: data engineers setup Airflow DAGs in Python and author dbt models in YAML  while data analysts perform data exploration in Python-based notebooks. In order for data engineers to not have to shift between YAML and JS as they work with so many different tools in their stacks, we want to bring support for these languages to Cube.

> “Cube’s YAML support makes my life easier because it streamlines the compatibility between the tools in my stack and my data modeling. Not needing to toggle between different languages saves me time and effort. I’m excited about this feature—and about what Cube has in store for future updates!”

– Olivier Dupuis, Analytics Engineering Consultant, Rittman Analytics

## Getting started with YAML

To get started with YAML data modeling in Cube, you’ll need to first ensure that you’re using Cube version 0.31.0 or higher.

Then,  create a `schema/orders.yml` file with the following content: 

```yaml
cubes:
  - name: orders
    sql: SELECT * FROM public.orders

    measures:
      - name: count
        type: count
      - name: total_revenue
        sql: amount
        type: sum

    dimensions: 
      - name: id
        sql: id
        type: number
        primary_key: true  
      - name: status
        sql: status
        type: string
```

You can learn more about YAML models in our [newly-released documentation](https://cube.dev/docs/schema/getting-started/yaml). 

## The future of YAML (and Python) in Cube

This release is the first step in our journey to bring both YAML and Python support to Cube. As we continuously implement and bolster support, there are some things we’ll continue to enhance, specifically around developer experience.

The following are the major areas on which we’re working.

### Dynamic data models

Many of Cube’s customers leverage [dynamic data models](https://cube.dev/docs/schema/advanced/dynamic-schema-creation) for multi-tenant scenarios. While it is possible to use `COMPILE_CONTEXT` already, dynamic data model generation is not yet available for YAML models.

We’re planning to let users write Python to dynamically generate models the very same way they can write JavaScript to generate them today. And, if you’re interested in this feature, we’d love to hear your thoughts—so, please [post your thoughts](https://github.com/cube-js/cube.js/issues/5045) on how this feature should be implemented.

### YAML configuration

While most of the Cube configuration can be done via environment variables—the way we recommend doing it—users lean on the JS-based `cube.js` configuration file for advanced configuration. We’re planning to introduce YAML and Python support for advanced configuration scenarios as well.

### Documentation and best practices
We’re releasing initial documentation for YAML modeling today. In the near future, we’ll update Cube documentation to fully integrate YAML models reference and examples. We are also updating our guides, tutorials, and best practices to showcase YAML examples alongside JS ones.

### Cloud IDE support
As part of our Cube Cloud vision, we’re aiming to build a first-class development environment for building data models in Cube, which now includes YAML models. Our Cloud IDE will include data model validation, in-line tips, pre-commit testing, and many more useful features focused on developer productivity and ergonomics. 

## Give us feedback

You can help us prioritize the features we work on and engage with our open-source software via [Slack](https://slack.cube.dev/), [GitHub](https://github.com/cube-js/cube.js), or [our contact page](https://cube.dev/contact). 

And if you have any questions about getting started with Cube’s YAML support (or any other questions, for that matter) find us on our Slack! As always, we’d love to hear from you. 

Artyom Keydunov

Data Modeling

You asked—we listened. Now, you can do your Cube data modeling in YAML. 

Introducing YAML Data Modeling in Cube

Igor Lukanin

Chart.js

A step-by-step guide into the new release of the most popular charting library.

Chart.js tutorial: Getting started with the all-new Chart.js v4

At Cube, we're on a mission to provide developers with an analytical data access layer to help them build modern applications. We call this data access layer a [’headless BI’ platform](https://cube.dev/blog/headless-bi).

The goal of headless BI is to facilitate consistent, performant, and secure access to all of your data. Of course, this mission only works if we—that is, [Cube, the headless BI platform](https://cube.dev/)—can simultaneously support all of your data sources and make those data sources compatible with the broadest array of downstream tools possible. 

So today, we’re proud to launch our new [integrations page](https://cube.dev/integrations)—so that you can find and read all about how your favorite tools integrate with Cube (and in that, the rest of your stack.)

## The inspiration: complexity

Cube was borne out of the sheer complexity of the modern data stack—one that exists not only because of the fractal-like explosion of data sources (cloud data warehouses, transactional databases, streams, document databases, query engines, etc.) but also in the types of tools with which users consume data.

Yes, for better or worse, gone are the days of an organization simply choosing a single business intelligence tool and connecting it to a transactional database. 

Now, users expect to gain insights from their data with both visualization and business intelligence tools, and with notebooks, and by building their own applications with low-code tools, powerful front-end frameworks, and charting libraries. And naturally, this expectation and the remarkable number of options available result in incredible building ability—but potentially mind-boggling connectivity complexity.

![](https://ucarecdn.com/60b3a40f-79aa-40d6-803a-928de8d7bfcc/)

## The goal: universal compatibility 

Universal compatibility—meaning the ability of every data source to connect to every downstream use case—is a tall order. The good news? We have an exceptionally well-known, durable, and scalable language on our side: SQL. [Cube’s Postgres-compliant SQL API](https://cube.dev/docs/backend/sql) is the magic translation layer by which we’re able to make universal compatibility happen. 

So, perhaps your business intelligence tool doesn’t have a native driver for your fancy new analytical query engine. Well, now there’s a solution for that: if you connect your query engine to Cube, Cube’s SQL API can expose it to whichever downstream, SQL-speaking tools you choose with a simple Postgres connection. 

Or, let’s take another scenario: what if you were building a [customer-facing analytics application](https://cube.dev/use-cases/embedded-analytics)? In that case, maybe you’d prefer to expose connections to your data sources as a [REST](https://cube.dev/docs/rest-api) or a [GraphQL API](https://cube.dev/docs/backend/graphql)? Cube could do that too. A rigid out-of-the-box business intelligence tool just can’t offer the same flexibility as Cube, with which you could then use any number of front-end frameworks or low-code tool builders.

![](https://ucarecdn.com/86bdb379-b989-4e68-bb71-df784c819bd9/)

## The future: new data sources, new tools, and much (much) more. 

We believe that you should be able to curate your stack with any data source and downstream tools you want. 

And we believe in open standards and in the power of a broad ecosystem.

These tightly held values mean that if you’ve come across a fantastic tool for which we haven’t yet listed an integration, [we’d love to hear from you](https://cube.dev/contact). Because really—we built this tool with a dedication to the data community and are determined to hone it further for the community (you can also find us on [Slack](https://slack.cube.dev/) and [Github](https://github.com/cube-js/cube.js).)

Even with all of the integrations we’ve built out already, this is just the beginning of our plans for the Cube ecosystem. So looking ahead, we’ll continue adding more data sources, more downstream tools, and more categories within the modern data stack. 

We’re on an exciting journey, here—and we hope you’ll join us for the ride :) 


Brian Bickell

News & Product Updates

So you can easily find your favorite data source or tool in Cube's integrations. 

Introducing Our Integrations Gallery

Today, we're happily announcing that Cube now works with [Trino](https://trino.io), the fast distributed SQL query engine for big data analytics, formerly known as PrestoSQL.

> [Meet the Cube team at Trino Summit](https://cube.dev/events/cube-at-trino-summit) on November 10, 2022. We'll be happy to chat and explore how Cube can augment your Trino experience.

## What is Trino?

Trino is an [open-source](https://github.com/trinodb/trino) query engine designed to work with big data.

It's fast, scalable, SQL-compliant, and has almost universal connectivity to all kinds of data sources and business intelligence tools. Let's explore these in more detail:

* **Scale.** Trino is capable of querying exabyte-scale data lakes and massive data warehouses, as confirmed by [Trino installations](https://trino.io/users.html) at companies such as LinkedIn, Netflix, and Shopify.
* **Performance.** Trino is a highly parallel and distributed query engine with two types of [cluster nodes](https://trino.io/docs/current/overview/concepts.html#server-types): coordinator nodes, responsible for query planning and execution, and worker nodes, responsible for fetching and processing data. You can fine-tune your Trino cluster to allow for efficient, low latency analytics.
* **Connectivity.** Trino can natively query data from a plethora of [data sources](https://trino.io/docs/current/connector.html) without the need to extract and load the data into a data warehouse, even if you'd like to do a cross-database join. Also, Trino is an ANSI SQL-compliant query engine that works with BI tools such as Tableau, Power BI, and Superset as well as headless BI tools such as Cube.

Most common use cases for Trino include ad-hoc analytics at interactive speeds, massive multi-hour batch queries, and high volume apps that perform sub-second queries.

Trino was originally [designed and developed at Facebook](https://trino.io/blog/2020/12/27/announcing-trino.html) in 2013 and beared the name *Presto* at that time. In 2019, Presto development forked and two query engines emerged: PrestoDB maintained by Facebook and PrestoSQL maintained by the Presto Software Foundation and the original creators. Later in 2020, PrestoSQL was renamed to Trino.

If you're not a Trino user yet, consider [self-hosting](https://trino.io/docs/current/installation.html) or using a managed platform provided by [Starburst](https://www.starburst.io) or AWS (as [Amazon Athena](https://aws.amazon.com/athena/), serverless interactive query service). 

## What is Cube?

Cube is the [headless BI](https://cube.dev/blog/headless-bi) platform for accessing data from modern data stores (including Trino), organizing it into consistent metrics definitions, and delivering them to downstream applications.

![Cube schema](https://ucarecdn.com/5f6287b5-fd33-47c8-95be-2dffd84ccfe2/)

Cube is designed to take the central part in the data pipeline, delivering consistent data to all downstream teams and data consumers. It serves as a source of truth for the metrics definitions, access control rules, and caching settings. Regardless of how many data consumers you have (e.g., front-end applications with embedded analytics or BI tools), Cube will deliver consistent data to all of them with its [REST API](https://cube.dev/docs/rest-api), [GraphQL API](https://cube.dev/docs/backend/graphql), or [SQL API](https://cube.dev/docs/backend/sql).

## How Cube works with Trino

In a typical data pipeline, Trino is placed upstream of Cube, providing unrestricted access to all data sources as well as data federation capabilities. You can use Cube to create an additional [semantic layer](https://cube.dev/blog/open-source-data-modeling) or a last-mile [caching layer](https://cube.dev/docs/caching) on top of Trino.

More importantly, you can use the set of APIs that Cube provides, including REST API and GraphQL API, to deliver the data directly to custom-built front-end applications, retaining low latency and high concurrency.

![Cube and Trino architecture](https://ucarecdn.com/cc996b6f-5f7d-4efb-83e6-960fc63546cd/)

Also, you can leverage Cube's [multitenancy](https://cube.dev/docs/multitenancy-setup) and [access control](https://cube.dev/docs/security) features to fully control and customize how every user, role, or data consumer accesses the data in Trino and underlying data sources.

## Try Cube with Trino

Cube is [open source](https://github.com/cube-js/cube.js), so you can feel free to [self-host and deploy](https://cube.dev/docs/deployment/overview) it. You can also run fully managed Cube in [Cube Cloud](https://cube.dev/cloud) which is the fastest and most convenient way to use Cube.

Start by signing up for a free [Cube Cloud account](https://cubecloud.dev/auth/signup). You'll be prompted to select the cloud provider (AWS, GCP, or Azure) and a region for your deployment:

![Cube Cloud: 1st step](https://ucarecdn.com/7ae7b4ae-57a1-4fb9-9e3d-bfa3fc011a67/)

Then, pick Trino from the data source options:

![Cube Cloud: 2nd step](https://ucarecdn.com/9dcf732a-eb10-4be9-bb8b-685e9060c5f0/)

Lastly, provide the credentials for the Trino connection:

![Cube Cloud: 3rd step](https://ucarecdn.com/aa796b76-ee85-4aae-ad66-c2a7e8ced05a/)

In a few seconds, you'll get your Cube Cloud deployment up and running, ready to query your Trino installation and deliver data to downstream applications.

## What's next?

Would you like to learn more about Cube and what it brings to the table? [Check the docs](https://cube.dev/docs/) and create a free [Cube Cloud account](https://cubecloud.dev/auth/signup) today.

Also, please feel free to [join our community](https://slack.cube.dev/) of almost 7000 data practitioners on Slack, give Cube a [star on GitHub](https://github.com/cube-js/cube.js), or [schedule a time](https://cube.dev/contact) to chat 1:1.

Trino

Perform interactive big data analytics with a fast and distributed query engine.

Announcing the Cube integration with Trino, the SQL query engine for big data 

With [Metabase](https://www.metabase.com/) as the business intelligence platform, your team doesn’t need to speak SQL. It’s the easy, open-source way to help everyone in your company work with data like an analyst. Pairing it with [Cube](https://cube.dev/), the headless BI platform, will provide a consistent data model making it easy to build self-service analytics, shared across multiple teams.

Here’s what your Metabase dashboard will look like after we integrate it with Cube. Check out the [live demo](http://metabase.examplescube.dev/public/dashboard/947a9fa9-0c79-47e8-9cca-55cee7eafbcf) as well! If stuck, you can always refer to the [source code on GitHub](https://github.com/cube-js/cube.js/tree/master/examples/metabase).

![demo](https://ucarecdn.com/4d1d1527-a396-4b3a-8289-33c8812aaee6/)

> In early December, we're hosting a webinar on replacing Looker with Metabase and Cube. [RSVP and save the seat.](https://cube.dev/events/replacing-looker-with-cube-and-metabase)

## What is Metabase?

[Metabase](https://www.metabase.com/) is an [open-source](https://github.com/metabase/metabase) business intelligence platform with a low entry barrier, smooth visual interface, and generally pleasant user experience. It provides an easy way for everyone in your company to work with data like an analyst.

You can use Metabase to ask questions about your data, build self-service analytics, or even embed Metabase in your front-end data application to let your customers explore data on their own. You can save questions, group them into dashboards, and share both questions and dashboards with the your team.

Metabase enables three main scenarios:

* **Query and visualize your data.** Use the query builder to filter and summarize data. Choose between either the *Notebook editor* to ask sophisticated questions with joins, filters, and aggregations, or *Native queries* to compose questions in the database’s native query languages. Awesome if you're proficient with SQL!

* **Embed charts and share your results.** You can add questions to a dashboard and include filters and connect them to fields on questions to narrow the results. You can share the dashboard by sending the URL to your co-workers, or by embedding them using iframes. You can even embed the full Metabase app, which allows you to do things like delivering multi-tenant, self-service analytics.

* **Find things and stay organized.** Collections organize questions, models, dashboards, and other collections. They work like folders on a file system. Or, use the search bar to find data, metrics, segments, dashboards, models, and questions. But, [X-ray](https://www.metabase.com/docs/latest/exploration-and-organization/x-rays) is by far the most amazing feature in Metabase. X-raying a table will antomagically generate a bunch of questions that slice the table’s records in different ways and save it as a dashboard.

## What is Cube?

[Cube](https://cube.dev/) is the [headless BI](http://cube.dev/blog/headless-bi) platform for accessing data from [modern data stores](https://cube.dev/docs/config/databases), organizing it into consistent metrics definitions, and delivering them to downstream applications.

![headless BI](https://ucarecdn.com/dfb299b9-7e27-47f0-a362-0a14c643c740/)

Why would you want to have Cube in your data pipeline? **Data consistency.**

Cube will deliver consistent data to all teams using Metabase. Serving as a source of truth for the metrics definitions, access control rules, and caching settings, it solves the *many-to-many problem* many data teams have. Regardless of how many data sources (like Snowflake or Redshift) or data consumers (like Metabase) you have, Cube allows to get rid of siloed data pipelines.

![https://ucarecdn.com/728192a0-d30f-41c9-a8c9-f865361691e5/](https://ucarecdn.com/728192a0-d30f-41c9-a8c9-f865361691e5/)

Conceptually, Cube has four layers:

* **Data modeling.** With dozens of [supported data sources](https://cube.dev/docs/config/databases) and a [centralized data model](https://cube.dev/docs/schema/getting-started) for defining metrics definitions, you can make sure every team has access to exactly the same data, in Metabase and beyond.
* **Access control.** Cube’s [security configuration](https://cube.dev/docs/security) lets you manage who is able to access data and which data it is. As a part of that, Cube provides [row-level security](https://cube.dev/docs/security/context#security-context) and [multi-tenancy](https://cube.dev/docs/multitenancy-setup) for Metabase users.
* **Performance.** Cube provides a [centralized caching layer](https://cube.dev/docs/caching) which ensures [consistent performance](https://cube.dev/docs/caching/pre-aggregations/getting-started) and data freshness across use cases or teams. As a result, your Metabase dashboards will load in under a second.
* **APIs**. Cube exposes three APIs: REST API, GraphQL API, and SQL API. Cube's [SQL API](https://cube.dev/docs/backend/sql) is fully Postgres-compliant, enabling connectivity to Metabase and, actually, any downstream tool that works with Postgres.

So, with Cube, you can use native, syntactically correct SQL queries, similar to how you’d query a data source directly:

```sql
SELECT
  DATE_TRUNC('day', createdAt) AS created_at_per_day,
  SUM(count) AS orders
FROM Orders
GROUP BY 1
ORDER BY 1 ASC
```

[Cube is open source](https://github.com/cube-js/cube.js), and you can self-host it in a cloud of your choice. Also, you can run Cube in [Cube Cloud](https://cube.dev/cloud) which provides fully managed experience with a [free tier](https://cube.dev/pricing).

## Integrate Metabase and Cube

Take a look a short video showcasing how Cube and Metabase work together:

<div style={{ position: 'relative', paddingBottom: '54.84375%', height: 0 }}><iframe src="https://www.loom.com/embed/38752bb23e5244d1a43675795c470a53" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen style={{ position: 'absolute', top: 0, left: 0, width: '100%', height: '100%' }}></iframe></div>

You can also follow more elaborate instruction below:

**Running Cube.** You can run Cube in two ways: either self-host it on your infrastructure or use the fully managed version in Cube Cloud.

For the sake of simplicity, we'll run [Cube Cloud](https://cube.dev/cloud). There’s a generous free tier for you to try. Go ahead, [sign up](https://cubecloud.dev/auth/signup) and follow the [instructions](https://cube.dev/docs/cloud/getting-started/create) on connecting to your data source and generating an initial data model.

Use these credentials to connect a sample e-commerce database to your Cube instance.

```bash
Hostname:  demo-db-examples.cube.dev
Port:      5432
Database:  ecom
Username:  cube
Password:  12345
```

This is what your deployment in Cube Cloud will look like:

![https://ucarecdn.com/4d1cf863-426c-4d40-829c-e3237a8e07d7/](https://ucarecdn.com/4d1cf863-426c-4d40-829c-e3237a8e07d7/)

Working with the data model. To see the data model you created above, navigate to the <Btn>Schema</Btn> tab. You will see files named `LineItems.js`, `Orders.js`, `Users.js`, etc. under the `schema` folder which defines the cubes in your data model.

![https://ucarecdn.com/b9d18c13-0d79-4487-9ed5-0a7c7c130db3/](https://ucarecdn.com/b9d18c13-0d79-4487-9ed5-0a7c7c130db3/)

Key ideas on what we’re seeing above:

- A cube is mapped to a table in your database to categorize measures (quantitative data) and dimensions (qualitative data) into [metrics definitions](https://cube.dev/docs/schema/reference/cube).
- This cube is mapped to the `public.orders` table.
- [Measures](https://cube.dev/docs/schema/reference/measures) are defined as aggregations (e.g. count, sum, etc.) over columns.
- [Dimensions](https://cube.dev/docs/schema/reference/dimensions) are mapped to columns.
- You can define complex measures and dimensions with custom SQL statements and references to other measures and dimensions.

**Customizing the data model.** You can add custom measures and dimensions, and edit them with filters or segments. In this example let’s update the schema by adding the `completedCount` measure to the `Orders` cube.

```javascript
completedCount: {
  type: `count`,
  filters: [{
    sql: `${CUBE}.status = 'completed'`
  }],
}
```

To do this, click the <Btn>Enter Development Mode</Btn> button in Cube Cloud to unlock the schema files for edits. This creates a development environment on a separate branch from your `main` branch. Here you can make changes to the data model before committing them to the main branch.

Here’s the `Orders.js` schema file, but with the added `completedOrders` measure. Paste this snippet into your `Orders.js` file.

```javascript
cube(`Orders`, {
  sql: `SELECT * FROM public.orders`,

  preAggregations: {
    // Pre-Aggregations definitions go here
    // Learn more here: https://cube.dev/docs/caching/pre-aggregations/getting-started  
  },

  joins: {
    Users: {
      sql: `${CUBE}.user_id = ${Users}.id`,
      relationship: `belongsTo`
    },

    Products: {
      sql: `${CUBE}.product_id = ${Products}.id`,
      relationship: `belongsTo`
    }
  },

  measures: {
    count: {
      type: `count`,
      drillMembers: [id, createdAt]
    },

    number: {
      sql: `number`,
      type: `sum`
    },

    completedCount: {
      type: `count`,
      filters: [{
        sql: `${CUBE}.status = 'completed'`
      }],
    }
  },

  dimensions: {
    id: {
      sql: `id`,
      type: `number`,
      primaryKey: true
    },

    status: {
      sql: `status`,
      type: `string`
    },

    createdAt: {
      sql: `created_at`,
      type: `time`
    },

    completedAt: {
      sql: `completed_at`,
      type: `time`
    }
  }
});
```

Once you commit and push this change, jump over to the Playground and run a query to get a response for the custom measure.

![https://ucarecdn.com/30dbcd19-a6c3-4d75-a532-5d4eeaea83f2/](https://ucarecdn.com/30dbcd19-a6c3-4d75-a532-5d4eeaea83f2/)

**Adding pre-aggregations.** Pre-aggregations are intermediate representation of the data, defined declaratively, computed in advance, persisted as tables, and used to accelerate queries. Cube has the ability to analyze queries against a defined set of pre-aggregations in order to serve the query from cache and not hit the data source directly.

Pre-aggregations physically reduce the size of the data. Fewer bytes to read, less time to return a response. This will dramatically improve query performance and provide a higher concurrency.

Let’s add a pre-aggregation definition to speed up the query above even more. Enter development mode once again, and edit the `Orders.js` file by pasting this piece of code in the pre-aggregations section.

```jsx
preAggregations: {
  completedOrdersByDay: {
    measures: [
      Orders.completedCount
    ],
    timeDimension: Orders.createdAt,
    granularity: `day`
  }
},
```

Here’s what the schema will look like.

![https://ucarecdn.com/3625d23f-91fc-4579-9f76-5d7638db32eb/](https://ucarecdn.com/3625d23f-91fc-4579-9f76-5d7638db32eb/)

Once you commit and push this change, jump over to the Playground and run the same query as you did above. This time you’ll see it return a response even faster:

![https://ucarecdn.com/b7e1d0ed-b9a9-4396-87fe-e335872382af/](https://ucarecdn.com/b7e1d0ed-b9a9-4396-87fe-e335872382af/)

**Integrating Metabase and building a dashboard.** You can run Metabase either in your infrastructure with the [open-source](https://github.com/metabase/metabase) version or use [Metabase Cloud](https://www.metabase.com/start/). For the sake of simplicity, let's [sign up](https://store.metabase.com/checkout/starter/login-details) for a Metabase free trial. Or, if you rather prefer using Docker, you can [run a Metabase instance yourself with Docker](https://www.metabase.com/docs/latest/installation-and-operation/running-metabase-on-docker).

Once you have signed in to your Metabase instance, go ahead and add a PostgreSQL data source. Cube’s SQL API speaks the PostgreSQL dialect of SQL which makes it compatible with all kinds of data tools, Metabase included. Copy the credentials from the SQL API section on the Overview page of Cube Cloud:

![https://ucarecdn.com/56fad92a-8a97-4e41-a602-4a36947be4e7/](https://ucarecdn.com/56fad92a-8a97-4e41-a602-4a36947be4e7/)

Or, you can use these sample credentials:

```bash
Host:      tan-ptarmigan.sql.gcp-us-central1.cubecloudapp.dev
Port:      5432
Database:  tan-ptarmigan
User:      cube
Password:  544056feb07f485e96913fc66b0404b4
```

Then, enter the credentials as a new PostgreSQL data source in Metabase.

![https://ucarecdn.com/3e4c319f-4708-493e-bd87-4a2a21d2f2ed/](https://ucarecdn.com/3e4c319f-4708-493e-bd87-4a2a21d2f2ed/)

Make sure to enable the "Use a secure connection (SSL)" toggle.

Now you can run queries to Cube from Metabase. By either using the Notebook editor to ask questions or Native queries, we can easily build a dashboard.

## Self-service analytics with Metabase

By far the most awesome feature in Metabase is the ability to X-ray a table and autogenerate a dashboard. Click the <Btn>⚡</Btn> icon to trigger the X-ray.

![https://ucarecdn.com/4db82cca-fae9-41ee-977e-261c5462a2fc/](https://ucarecdn.com/4db82cca-fae9-41ee-977e-261c5462a2fc/)

This will take you to an autogenerated dashboard. Here’s a [live preview](http://metabase.examplescube.dev/public/dashboard/947a9fa9-0c79-47e8-9cca-55cee7eafbcf) of what this dashboard looks like. 

![https://ucarecdn.com/e4b89682-d46a-460d-bcec-7e1a382a7446/](https://ucarecdn.com/e4b89682-d46a-460d-bcec-7e1a382a7446/)

Running an X-ray is an awesome way of hitting the ground running. But, let’s move on to more complex topics, building a custom dashboard.

Let's use the Notebook editor to ask a question. Select the `Orders` table. Summarize the `Sum of CompletedCount` by `CreatedAt: Day`.

![https://ucarecdn.com/0dd1a3d0-7e17-4e1f-85e4-374250e4a392/](https://ucarecdn.com/0dd1a3d0-7e17-4e1f-85e4-374250e4a392/)

This will visualize the same chart as we did before in the Cube Cloud Playground.

![https://ucarecdn.com/0f24c9ad-dbad-4508-8b79-054e1e5083d4/](https://ucarecdn.com/0f24c9ad-dbad-4508-8b79-054e1e5083d4/)

Using the Notebook editor is an awesome way of executing SQL without writing SQL. It’s great for team members that don’t speak SQL.

However, what about the ones that do? Let’s write a native SQL query next. Views in Cube can be queried the same way as cubes. This snippet below will return the same data as the Notebook query from above.

```sql
SELECT
  DATE_TRUNC('day', createdAt) AS created_at_per_day,
  SUM(completedCount) AS completed_orders
FROM Orders
GROUP BY 1
ORDER BY 1 ASC
LIMIT 10000
```

![https://ucarecdn.com/8d2a7839-e14a-40cf-879d-cb1aa56dce57/](https://ucarecdn.com/8d2a7839-e14a-40cf-879d-cb1aa56dce57/)

Once you’re happy with the questions, save them, and add them to a dashboard.

![https://ucarecdn.com/07c02752-3dc1-4a49-8fe3-204d1fa4a99c/](https://ucarecdn.com/07c02752-3dc1-4a49-8fe3-204d1fa4a99c/)

## Sharing and embedding Metabase dashboards

Metabase gives you multiple options to share dashboards. You can embed them by using iframes. You can share the dashboard by sending the URL to your co-workers. You can even embed the full Metabase app, which allows you to do things like delivering multi-tenant, self-service analytics.

In the Admin settings, click <Btn>Public Sharing</Btn> and toggle `Enable Public Sharing` to `Enabled`. This will allow you to create public URLs to share dashboards.

![https://ucarecdn.com/85d24d95-3eb8-4a67-b84c-eea5006a6319/](https://ucarecdn.com/85d24d95-3eb8-4a67-b84c-eea5006a6319/)

To embed dashboards jump below to the <Btn>Embedding</Btn> tab. Toggle it to <Btn>Enabled</Btn>, and you’re good to go.

![https://ucarecdn.com/3959935d-b3c0-4000-b40a-2076999f7885/](https://ucarecdn.com/3959935d-b3c0-4000-b40a-2076999f7885/)

To select a specific dashboard you want to share, go to the dashboard and click the <Btn>Sharing</Btn> button in the top right.

![https://ucarecdn.com/5fd254e2-566e-4bbe-be0e-0739541da643/](https://ucarecdn.com/5fd254e2-566e-4bbe-be0e-0739541da643/)

This will open up a prompt to select how to share the dashboard.

![https://ucarecdn.com/e84de9a9-680d-47bc-b147-ac6e7e960d72/](https://ucarecdn.com/e84de9a9-680d-47bc-b147-ac6e7e960d72/)

We’ve prepared a [live demo](http://metabase.examplescube.dev/public/dashboard/947a9fa9-0c79-47e8-9cca-55cee7eafbcf) for you to check out. Once you enable public sharing, you’ll have the option to send a link to the dashboard to whoever you want!

If you would rather want to embed the dashboard, using an iframe will look like this.

<iframe
src="https://metabase.examplescube.dev/public/dashboard/947a9fa9-0c79-47e8-9cca-55cee7eafbcf"
frameborder="0"
width="800"
height="600"
allowtransparency
> </iframe>

Check out the [Metabase docs](https://www.metabase.com/docs/latest/embedding/start) to learn more about embedding and sharing dashboards.

## Wrapping up

Using Metabase with Cube as a Headless BI layer will ensure data uniformity and consistent metrics definitions for your self-service analytics.

Would you like to explore more about Cube? [Check the docs](https://cube.dev/docs/) and create a free [Cube Cloud account](https://cubecloud.dev/auth/signup) today.

Also, please feel free to [join our community](https://slack.cube.dev/) of almost 7000 data practitioners on Slack, [drop us a line](https://cube.dev/contact), or give Cube a [star on GitHub](https://github.com/cube-js/cube.js). See you there!

Metabase

Business intelligence for everyone, with consistency and without the SQL barrier. 

Self-service analytics with Metabase and Cube

## The Background

[Cloud Academy][1] is a SaaS startup based in San Francisco, CA. Their e-learning platform aims to give tech workers a personalized upskilling experience in technical subjects such as cloud technology, DevOps, and software development. 

“Personalized” is somewhat of an understatement: Cloud Academy’s training resources and assessments are customized precisely to the user’s trajectory, role, and skills. In addition, the analytics and personalization granularity of the platform offers organizations robust insights into their teams’ proficiency. With these insights, companies can boost employee retention and accelerate company operations.

## The Challenge

Cloud Academy was searching for a modern analytics platform to improve its existing enterprise analytics offering.

[Alessandro Lollo][2], Senior Data Engineer, explains that the team needed to deliver a seamless, highly available embedded analytics experience based on CI/CD best practices that also allowed planned outages for required infrastructure maintenance. They also needed to leverage their existing data warehouse and maintain high flexibility in their data modeling to serve both internal and external end users. 

Before Cube, Cloud Academy used a major BI platform on which their enterprise embedded analytics UI was based; internal stakeholders used the BI platform directly. However, the team needed a solution that would allow for greater flexibility when it came to effective data modeling, security context orchestration, and caching.

### The Requirements

Cloud Academy was looking for a solution that could serve as the basis of their embedded analytics while also offering:

 - Compatibility with existing data warehouse and datasets
 - Highly flexible data modeling
 - Highly flexible security context orchestration and data visibility configuration
 - Fully managed platform with zero downtime and multi-deployment
 - Easy integration with existing CI/CD process
 - Highly flexible, powerful, and fully managed caching
 - SQL API

> “With Cube, we’ve been able to speed up time to release a new data model to production by 5x and decrease analytics downtime by 90%. And, since the Cube team is so responsive, collaborative, and fast to deliver, we benefit from new features very frequently.” 
> —Alessandro Lollo, Senior Data Engineer

## The Solution

Lollo’s team found Cube in mid-2021 and chose it for its highly flexible [caching layer][3] and fully managed, [Git-integrated][4], and zero-downtime deployments.  

It took ten engineers about a month to onboard and integrate Cube with Cloud Academy’s existing CI/CD platform. After these initial processes, they delivered their first Cube-based analytics app to enterprise customers in less than three weeks.

Cube’s flexible data modeling framework made it easy for the team to use their existing data warehouse as a single source of truth, allowing them to serve consistent data models to both enterprise customers and internal stakeholders.

In addition, [Cube Store][5]’s fully managed pre-aggregation capabilities allowed them to boost the response times of complex data models that serve both internal and external use cases. 

Lastly, Cube Cloud’s isolated deployment and Git integration enabled the team to implement true and efficient SDLC, while its [RBAC framework][6] has served as a component in security context orchestration. 

__With Cube, Cloud Academy was able to speed up releases of new data models in production by 5x while decreasing their platform’s analytics downtime by 90%.__

Another surprising benefit was Cube’s [Alerts][7] feature, which provided the team with a simple yet effective mechanism to monitor deployments’ API health and pre-aggregation build failures.  

The cherry on top? “Since the Cube team is responsive, collaborative, and fast to deliver, we benefit from new features very frequently,” says Lollo.

## The Future

As they progress in their Cube implementation, Lollo’s team is planning to expand into incremental pre-aggregations and experiments using Cube’s [SQL API joins][8]. 

Looking ahead, they’re most excited for future Cube releases including [metrics layer][9], [Cube Store’s alternative to Redis][10], and [YAML support][11]. 

As for advice for others considering Cube Cloud, Lollo says, “When we first found Cube, we considered hosting it internally. However, we realized doing so would require quite the overhead from our DevOps team. If you plan on using Cube but don’t want to (or can’t) increase your infrastructure OPEX, then go for Cube Cloud—you won’t regret it.”

_Have an exciting Cube case study and want to be featured? Drop us a line: [hello@cube.dev][12]_

_Interested in managed [Cube Cloud][13] for your data modeling, access control, and caching needs? [Request a 1:1 here][14], or find us on [Slack][15] and [GitHub][16]._

[1]: https://cloudacademy.com/
[2]: https://www.linkedin.com/in/alessandrolollo/?originalSubdomain=it
[3]: https://cube.dev/docs/caching#overview
[4]: https://cube.dev/docs/cloud/release-notes/12-2021#git-integration
[5]: https://cube.dev/docs/deployment/overview#cube-store
[6]: https://cube.dev/docs/recipes/role-based-access#enforcing-role-based-access
[7]: https://cube.dev/docs/cloud/dev-tools/alerts
[8]: https://cube.dev/docs/backend/sql#sql-api
[9]: https://github.com/cube-js/cube.js/issues/5044
[10]: https://cube.dev/blog/replacing-redis-with-cube-store
[11]: https://github.com/cube-js/cube.js/issues/5045
[12]: mailto:hello@cube.dev
[13]: https://cube.dev/cloud
[14]: https://cubecloud.dev/auth/signup
[15]: https://slack.cube.dev/
[16]: https://github.com/cube-js/cube.js

Tamar Barlev

User Stories

How Cloud Academy sped up data modeling by 5x and decreased analytics downtime by 90%

Since we open-sourced Cube in 2019, we’ve added quite a few new features, such as our [new SQL API](https://cube.dev/blog/expanded-bi-support), and integrations, such as our recently-announced [support for streaming databases](https://cube.dev/blog/headless-bi-with-streaming-data). However, we haven’t changed our core [data modeling layer](https://cube.dev/blog/open-source-data-modeling)—until now.

> Today, we’re announcing a new feature in Cube’s data model: views.

Designed to sit on top of the data graph of cubes, views create a façade of your whole data model with which data consumers can interact. They’re the place to define metrics, manage governance and data access, and control ambiguous join paths.

> You can watch the recording of our [webinar](https://cube.dev/events/using-cube-views-for-metrics-management) with an in-depth conversation about views.

## Why views? Why now?

As the [headless BI](https://cube.dev/blog/headless-bi) platform, Cube helps define and maintain metrics definitions. The fundamental constraint is that metrics should make sense for every downstream application (solved by Cube's SQL API) and every business user at the same time. They should represent substantial entities in a user's business domain and not be obscured by any tool in the data stack. However, we have to admit there's an [open debate](https://benn.substack.com/p/ghosts-in-the-data-stack) whether metrics should only describe tangible, real-world entities like *users* or *orders* or they're also allowed to reflect more ephemeral concepts like *sales funnel*, *recurring revenue*, or *storage quota usage*.

We're introducing views so Cube's users are well-equipped to build fully semantic layers, i.e., expose metrics that make sense in the business domain at all times and don't include anything obscuring or unnecessary. Using an analogy from software engineering, views allow to define the "public API" on the surface of your semantic layer. 

![Views Schema](https://ucarecdn.com/06ef6be9-14a3-40ec-a05c-88151781ee83/)

Over the past years, we’ve accumulated a lot of feedback across many use cases that has boiled down to these three questions:
1. How should single-measure metrics be defined?
2. How should you manage the visibility of cubes to, e.g., restrict access for specific data consumers?
3. How can you control the join path within the data graph?

We’ve designed views to tackle these and many other use cases. Now, let’s take an in-depth look at them. (It's worth noting that, if you're familiar with Looker and LookML, you can find some similarities between views and Looker's [explores](https://cloud.google.com/looker/docs/reference/param-explore-explore).)

## 1. Defining metrics with views

Views with a single measure can be used to define metrics.

Here’s a simple example that demonstrates the relationship between a view and its underlying cubes. To define a metric with a view, pick a measure from your data model together with the dimensions:

![Views explained schema](https://ucarecdn.com/90c5e207-9bdc-48a6-b8d7-f58a760ed2c4/)

You can use the [includes](https://cube.dev/docs/schema/reference/view#parameters-includes) property to bulk add measures or dimensions to a view or define specific member as a reference in case you need more control, e.g., to rename the dimension.

<CodeTabs>
```JavaScript
view(`ActiveUsers`, {
  description: `14 days rolling count of active users`,

  includes: [
    // Measure
    Users.rollingCount,
    // Dimensions
    Users.isPaying,
    Users.signupDate
  ],

  dimensions: {
    companyName: {
      sql: `${Company.name}`,
      type: `string`
    }
  }
});
```
```YAML
views:
  - name: active_users
    description: "14 days rolling count of active users"
    includes:
      # Measure
      - users.rolling_count
      # Dimensions
      - users.is_paying
      - users.signup_date
    dimensions:
      - name: company_name
        sql: "{company.name}"
        type: string
```
</CodeTabs>

Within a view, you can use expressions to define a new metric that’s derived from calculations of one or more measures:

<CodeTabs>
```JavaScript
view(`OrdersMargin`, {
  description: `Total margin of orders`,

  measures: {
    margin: {
      sql: `${Orders.revenue} - ${Orders.costOfGoodsSold}`,
      type: `number`
    }
  },

  includes: [
    Orders.date,
    ProductCategory.categoryName
  ]
});
```
```YAML
views:
  - name: orders_margin
    description: "Total margin of orders"
    measures:
      - name: margin
        sql: "{orders.revenue} - {orders.cost_of_goods_sold}"
        type: number
    includes:
      - orders.date
      - product_category.category_name
```
</CodeTabs>

You can query these metrics views via REST or SQL API the same way you query cubes. In Cube’s SQL API, views are exposed as tables in which measures and dimensions are columns.

```sql
SELECT
  margin,
  categoryName
FROM OrdersMargin
GROUP BY 2
```

> Curious about YAML? We have seen a lot of requests to support YAML for data modeling, and we’re planning a release later this year. Please stay tuned for updates and [reach out to us](https://forms.gle/4CZRRKK8nD4PvtxZA) if you’d like to sign up for early access to YAML support.

## 2. Managing visibility and governance

Views can help you manage visibility of your data model through the [shown](https://cube.dev/docs/schema/reference/view#parameters-shown) property. Some data may be available only to specific roles within an organization or to specific customers.

In the example below, we’re making the `ARR` view only visible to users with finance permissions:

<CodeTabs>
```JavaScript
view(`ARR`, {
  description: `Annual Recurring Revenue`,
  shown: COMPILE_CONTEXT.permissions.finance,
  includes: [
    Revenue.arr,
    Revenue.date,
    Customers.plan
  ]
});
```
```YAML
views:
  - name: ARR
    description: "Annual Recurring Revenue"
    shown: COMPILE_CONTEXT.permissions.finance
    includes:
      - Revenue.arr
      - Revenue.date
      - Customers.plan
```
</CodeTabs>

You can also use the `shown` property to manage visibility of the specific measures and dimensions based on the users’ permissions. 

## 3. Controlling join paths 

Sometimes your data model may feature more than one way to join two cubes. The schema below illustrates a simplified example of that case, where the `Countries` cube can be accessed from `Orders` either directly or via the `Users` cube.

The join path is a part of your semantic model since it affects the meaning of calculations. In the example below, we can look either at an order’s warehouse country or the country of the user who placed the order. 

![Join in Cube data model](https://ucarecdn.com/c9f27f2c-2185-4016-b06a-bdedd3546bcc/)

Use views to control the join path. In the example below, we are creating two dimensions with the same `Countries` cube and name dimension, but with different semantic meaning because of the join path.

<CodeTabs>
```JavaScript
view(`CompletedOrders`, {
  description: 'Count of orders',

  includes: [
    Orders.count
  ],

  dimensions: {
    orderWarehouseCountry: {
      sql: `${Orders.Countries.name}`,
      type: `string`
    },
    userCountry: {
      sql: `${Orders.Users.Countries.name}`,
      type: `string`
    }
  }
});
```
```YAML
views:
  - name: completed_orders
    description: "Count of orders"
    includes:
      - orders.count
    dimensions:
      - name: order_warehouse_country
        sql: "{orders.countries.name}"
        type: string
      - name: user_country
        sql: "{orders.users.countries.name}"
        type: string
```
</CodeTabs>

## Available now

Views are available in both [Cube Core](https://github.com/cube-js/cube.js) and [Cube Cloud](https://cube.dev/cloud) starting from [version 0.31](https://github.com/cube-js/cube.js/releases/tag/v0.31.0) ([see docs for details](https://cube.dev/docs/schema/reference/view)). However, they’re optional: if you don’t need them, don’t use them.

Views can be very helpful to develop and maintain large data models or provide governance for many data consumers, but they may be overkill for smaller projects.

In addition to the three cases detailed here, we believe views will enable many additional opportunities, so we’re eager to see how you use them. Please [get in touch](https://slack.cube.dev) with your feedback, questions, and examples.

Our biggest update to Cube’s data modeling layer.

Introducing Views for metrics management

We are happy to announce our official integration with Cube. Now you can enhance your modern data stack with Cube and CrateDB as a starting point for building data-intensive applications.

## Cube: API-First Business Intelligence

[Cube’s](https://cube.dev/) headless business intelligence platform makes lakehouse data consistent, performant, and accessible to every downstream application. Cube connects to dozens of warehouses, databases, or query engines, and allows you to quickly build data applications or analyze your data in BI tools. 

Cube’s advanced caching and [pre-aggregation](https://cube.dev/docs/schema/reference/pre-aggregations) layer can condense frequently-used queries to optimize response times. While the data modeling provides a central location for your metrics definitions.

Cube provides this data to applications with GraphQL, REST, and SQL APIs so that every tool—and every data consumer—can work with the same data.

![alt_text](https://ucarecdn.com/00ad48b5-c916-417d-8d5a-53f0a76767df/)

## CrateDB: Distributed database for real-time analytics

[CrateDB](https://crate.io/docs/crate/reference/en/5.0/) is an open-source distributed database management system that makes it simple to store and analyze massive amounts of data in real time. It is designed to handle various data types, including time-series, full-text search, geospatial, and structured and semi-structured data. In addition, the CrateDB engine is optimized for fast queries and aggregations thanks to in-memory columnar indexes and data compression. 

Furthermore, CrateDB offers NoSQL flexibility with an SQL interface. The SQL compatibility makes the integration of CrateDB with other data engineering tools easy and efficient.

In this article, we will demonstrate how to set up a CrateDB cluster and how to use it as a data source in Cube Cloud for further data analysis.

## Deploy a cluster on CrateDB Cloud

To deploy a new cluster on CrateDB Cloud, please sign up for a [CrateDB Cloud account](https://console.cratedb.cloud/). When creating a new organization, you are entitled to a $200 free credit to spend on cluster deployment, scaling, and other operations as you see fit.

After the signup process is completed, you will arrive at the CrateDB Cloud Console. 

![alt_text](https://ucarecdn.com/b5feea0b-947a-475a-b3c3-3b3794c82ae6/)

At this point click on the _Deploy new cluster_ button. This will open the cluster deployment screen that consists of three steps: region selection, cluster configuration, and account settings. 

As a first step, we will select a region where the cluster will be deployed. 

![alt_text](https://ucarecdn.com/0fcf81b6-bb5a-4773-b96f-7867777c862d/)

As a second step, we will choose [a configuration plan](https://crate.io/docs/cloud/tutorials/en/latest/cluster-deployment/stripe.html). For each plan, you can see the CPU, RAM, and (minimum) storage values per node. 

![alt_text](https://ucarecdn.com/d6c50105-a5ce-45b2-8d65-0ce1f8fea9a7/)

In our case, we will choose the CR0 plan which refers to a shared resource cluster. This type of cluster shares compute and storage resources with other clusters in this category and that allows better utilization of resources. 

Finally, at the bottom of the screen, you can configure your account settings. In our example we choose the following credentials:

* Project name: `crate-cube`
* Cluster name: `sample-cluster`
* Username: `admin`
* Password: _auto-generated_

![alt_text](https://ucarecdn.com/813d734e-45fe-46e3-95a6-a9e1783f69e6/)

The next step requires the setup of your[ billing information](https://crate.io/docs/cloud/tutorials/en/latest/cluster-deployment/stripe.html). You can use existing AWS or Azure subscriptions or credit card information. Once this is completed, you can see the overview of your cluster with instructions on how to connect to the cluster, how to import sample data, and usage metrics.

![alt_text](https://ucarecdn.com/630b8e8e-57de-4ae7-a7f7-bfbaeb13f568/)

To start creating tables and running queries, click on _Open Admin UI_. This opens a web administration user interface (or Admin UI) for your running cluster.

![alt_text](https://ucarecdn.com/f67cf867-631b-4831-9a4e-6d9d0dc5b295/)

Now, let’s import some data. To do so, go back to the cluster overview page and click the _Learn how to import data_ link. This will open a list of statements you need to execute to load 

NYC taxi data:

![alt_text](https://ucarecdn.com/0ce12208-9cec-475f-adce-9230426bc67f/)

Great! Now that you have the first table in your CrateDB cluster, let’s see how to add this dataset to your Cube Cloud instance.

## Connect CrateDB to Cube

To start with Cube, create a new Cube instance in [Cube Cloud](https://cube.dev/cloud). You can sign up for a Cube Cloud account with a free trier.

Once you create a new Cube instance, you will need to create the new deployment. For our _cube-crate_ deployment we choose the cloud provider and the region:

![alt_text](https://ucarecdn.com/17870922-8a59-4846-b55e-1cecc6362ffe/)

On the next page, we click on the _Create_ tab to set up our CrateDB connection:

![alt_text](https://ucarecdn.com/d0a00c79-08cc-4406-b163-cc255b7c2b07/)

To connect CrateDB with Cube we need to specify the database connection of our CrateDB cluster:

* Hostname: `sample-cluster.aks1.westeurope.azure.cratedb.net`
* Port: `5432` (default)
* Database: `crate` 
* Username: `admin`
* Password: _the password you created during cluster creation_
* SSL checkbox: _CrateDB Cloud does not allow non-encrypted connections_

![alt_text](https://ucarecdn.com/72ddb77d-17b3-45b0-abcf-408816333135/)

Click on _Apply_ and in the next view select all tables for which you want Cube to generate the initial schema. For instance, we can generate the schema for our *nyc_taxi* table.

![alt_text](https://ucarecdn.com/938e0131-83fc-4c3a-8330-421a28e44e1c/)

Click on _Generate_ button, wait for a couple of seconds and you will see the newly deployed instance that is connected to our CrateDB database.

From this point, you can start exploring generated schema and defining new data models for your applications! 

## Cube as a headless BI layer

Let's have a look at the autogenerated schema. Switch to the Schema tab and click the `NycTaxi.js` file.

![alt_text](https://ucarecdn.com/240f7630-e7c5-4da2-8901-fcbc507e969e/)

From here it's easy to add more complex measures and dimensions to create your own custom metrics layer.

Go ahead and add a custom measure.

```javascript
  totalAmountSum: {
    sql: `total_amount`,
    type: `sum`
  }
```

![alt_text](https://ucarecdn.com/1ba7ba29-bbb5-40f1-8db4-1756ed395fb7/)

After you save and commit this change, open up the playground, and let’s run a sample query.

![alt_text](https://ucarecdn.com/88e8676a-a416-4754-b535-55d0de9d9da2/)

You can connect Cube to your front-end app of choice. Choose from using React, Vue, Angular, Vanilla JavaScript for the library integration, with the option to use BI tools with Cube’s SQL API.

![alt_text](https://ucarecdn.com/08f20be2-a069-4e9a-903f-7bbe2a105097/)

This means you can continue using all the BI tools you’re already using and are connected to your CrateDB instances. Cube only acts as the middleware layer between your BI tools and CrateDB. You still use SQL to access the data for your dashboards! Pretty neat.

![alt_text](https://ucarecdn.com/4ebe6af9-0312-4924-82f8-27faa55d5448/)

You now know how to configure Cube as a metrics layer!

## Next steps

By using Cube with CrateDB in your modern data stack you get an excellent starting point for further building out your data-intensive applications.

To learn more about Cube please check our [documentation](https://cube.dev/docs/backend/sql) and [supported tools](https://cube.dev/docs/config/downstream), and give it a try by registering a free [Cube Cloud account](https://cube.dev/) today. 

If you want to try out CrateDB in a few clicks and enjoy all of the CrateDB features, sign up for the [CrateDB Cloud](https://console.cratedb.cloud/?utm_campaign=2022-Q2-WS-Free-Trial&utm_source=website&utm_medium=free-trial-overhaul&hsCtaTracking=a7e2a487-cfb9-4a50-8e75-3029b9e176fb%7C7863166c-05e4-4334-9dd5-58dfdd6e78c1) trial. 

For updates, features, and other questions you might have join our [Cube](https://slack.cube.dev/) and [CrateDB](https://community.crate.io/) communities. 


Marija Selakovic

Transactional Databases (OLTP)

In this article, you’ll learn how to configure your own modern data stack to build data-intensive apps.

Enabling data analysis at scale with Cube and CrateDB

When I was reading Pedram’s [deep dive on metrics layers](https://pedram.substack.com/p/what-is-the-metrics-layer) a few weeks ago, I couldn’t help but question the tool selection. One can obviously blame my employee-employer affiliation which is hard to deny. However, the real reason for that was this [summary statement](https://pedram.substack.com/p/what-is-the-metrics-layer):

> All three tools have different trade-offs, and their strengths and weaknesses tell of the challenges a metrics layer faces.
> 

While this is a true statement, it dismisses the fact that there’s an ergonomic, feature-complete, and accessible solution on the market that doesn’t suffer from highlighted pitfalls. With that, I’m putting Cube in the spotlight and kindly requesting your ~~blowback~~ [feedback on social media](https://twitter.com/thecubejs/status/1578049949811589123).

## In previous episodes

If you haven’t read the original [deep dive on semantic layers](https://pedram.substack.com/p/what-is-the-metrics-layer), please do. I promise it’s gonna be a rewarding experience.

There, Pedram mentions some prior thought leadership on the topic and introduces a challenge for three tools: Looker, dbt Metrics, and Lightdash. They have to model and provide access to the *Activation Metric*, a crucial characteristic of an imaginary B2B SaaS product with users and workspaces. A workspace is considered active if users perform some activation event, so we want to report on the activation rate daily, weekly, or monthly and how it changes over time. Later, Pedram shows how Looker, dbt Metrics, and Lightdash solve the data modeling and elaborates on their strengths and pitfalls.

Here’s the `workspace_details` table matching the imaginary dataset that was used by all tools to calculate the metrics. I’ve generated 20 million rows and put them into Postgres, so we have something to work with:

![dataset](https://ucarecdn.com/1f7f0a20-b331-4afa-9e63-7313a7e96c05/)

Now, let’s talk about Cube.

## Cube, the headless BI

Cube is the [headless business intelligence](https://cube.dev/blog/headless-bi) platform built to make data consistent and accessible. It solves the many-to-many problem many data teams have by serving as a source of truth for metrics definitions, access control rules, and caching settings.

![Cube in the pipeline](https://ucarecdn.com/dfb299b9-7e27-47f0-a362-0a14c643c740/)

Cube connects to numerous [data sources](https://cube.dev/docs/config/databases), from cloud data warehouses to [streaming platforms](https://cube.dev/blog/headless-bi-with-streaming-data), and delivers data to all kinds of [data consumers](https://cube.dev/docs/config/downstream) via multiple APIs, including the [SQL API](https://cube.dev/blog/expanded-bi-support). It is deliberately visualization-agnostic and delegates data presentation and visualization to BI tools, data notebooks, front-end apps, etc.

Cube is open source and there's also [Cube Cloud](https://cube.dev/cloud) that provides fully managed experience and additional tooling on Free, Standard, and Enterprise tiers (see [pricing](https://cube.dev/pricing)).

## Data modeling

In Cube, we define metrics in [data model](https://cube.dev/blog/open-source-data-modeling) files that seem to be heavily inspired by LookML.

At the top level, everything is grouped in “cubes” that are usually modeled over tables or views in the data warehouse but can also be modeled over pretty complex queries. Cubes include definitions for measures (quantitative features), dimensions (qualitative features), segments (reusable filters), and pre-aggregations (declarative definitions of rollups that drastically speed up queries, more on them later).

Here’s the data model for our activation metric:

```jsx
cube(`WorkspaceActivation`, {
  sql: `SELECT * FROM public.active_workspace_details`,

  measures: {
    count: {
      type: `count`
    },

    n_workspaces: {
      sql: `workspace_id`,
      type: `countDistinct`,
    },

    n_active_workspaces: {
      sql: `workspace_id`,
      type: `countDistinct`,
      filters: [ {
        sql: `${is_active_workspace} IS TRUE`
      } ],
    },

    activation_rate: {
      sql: `ROUND(100 * (1.0 * ${n_active_workspaces} / ${n_workspaces}), 2)`,
      type: `number`
    }
  },

  dimensions: {
    workspace_id: {
      sql: `workspace_id`,
      type: `number`,
      primaryKey: true
    },

    reporting_day: {
      sql: `reporting_day`,
      type: `time`
    },

    is_active_workspace: {
      sql: `is_active`,
      type: `boolean`
    }
  },
});
```

A few observations here:

- Similarly to Looker, we define measures as formulas, not as fully-formed SQL tables.
- While we can use “formatting” functions like `ROUND` in measure definitions, formatting options are deliberately limited. Cube puts the burden of data presentation on downstream tools.
- By default, we don’t have to specify how our reporting date should be broken down. We’ll be able to provide desired time granularity at query time.
- Needless to say, we can define other cubes and [joins](https://cube.dev/docs/schema/reference/joins#joins) between them.

## Query syntax

Cube provides a set of APIs to fetch data: [SQL API](https://cube.dev/docs/backend/sql) (mostly for BI tools and data notebooks), [REST API](https://cube.dev/docs/rest-api), and [GraphQL API](https://cube.dev/docs/backend/graphql) (mostly for front-end applications). A simple query to the SQL API:

```jsx
SELECT reporting_day, activation_rate
FROM WorkspaceActivation;
```

Of course, we can run a more complex query, too:

```jsx
SELECT
  DATE_TRUNC('WEEK', reporting_day) AS reporting_week,
  activation_rate
FROM WorkspaceActivation
ORDER BY 1 DESC;
```

Queries would turn into JSON when expressed in the REST API syntax. First one:

```json
{
  "measures": [
    "WorkspaceActivation.activation_rate"
  ],
  "timeDimensions": [ {
    "dimension": "WorkspaceActivation.reporting_day"
  } ]
}
```

Second one:

```json
{
  "measures": [
    "WorkspaceActivation.activation_rate"
  ],
  "timeDimensions": [ {
    "dimension": "WorkspaceActivation.reporting_day",
    "granularity": "week"
  } ],
  "order": {
    "WorkspaceActivation.reporting_day": "desc"
  }
}
```

Since we haven’t connected a downstream tool yet, we can run queries and explore the metrics in Cube’s [Playground](https://cube.dev/docs/dev-tools/dev-playground) with drop-down inputs for measures and dimensions:

![Playground](https://ucarecdn.com/0a0e01d2-cb3a-4cbe-8f2f-a87b257324bb/)

We can instantly get the JSON queries for the REST API here:

![JSON query](https://ucarecdn.com/51228e1d-33b6-46d7-8bf6-637e04261e2c/)

More conveniently, we can also look up the SQL statement that would be generated by Cube and run against the data warehouse when the query is executed. No surprise here: Cube uses the provided query and data model definitions to compose the final query: 

![Generated SQL query](https://ucarecdn.com/742514b5-ee4e-43dd-8c1b-d03b3284dbcc/)

## Query acceleration

Obviously, queries can be expensive: costly to run, too complex to wait for, or both.

In that case, we can define a pre-aggregation (actually, the recommendation is always to use pre-aggregations in production). Here’s what we need to append to the data model:

```jsx
  preAggregations: {
    main: {
      measures: [ activation_rate ],
      timeDimension: reporting_day,
      granularity: `week`,
      refreshKey: {
        every: `1 day`
      }
    }
  }
```

It instructs Cube to build a rollup and use it to serve queries that contain any subset of provided measures and dimensions (namely, `activation_rate` and `reporting_day` with weekly granularity). Usually, a single pre-aggregation speeds up multiple queries. We can also configure freshness: a pre-aggregation update can be triggered on schedule (e.g., every day) or when an arbitrary SQL statement yields a new result. 

Now, the last query would return the same result, but a completely different query would be executed against the data warehouse:

![Accelerated SQL query](https://ucarecdn.com/786f7cc1-791d-4122-b13a-b3660405f439/)

Truth be told, it wouldn’t be executed against the data warehouse at all. The `prod_pre_aggregations.workspace_activation_main` table is transparently created and maintained in [Cube Store](https://cube.dev/blog/introducing-cubestore), a custom-built massively distributed columnar storage for pre-aggregation data.

There’s an option to skip using Cube Store and store pre-aggregations in a data warehouse, but that’s not the most reasonable thing to do. With pre-aggregations in Cube Store, Cube can serve practically any query with sub-second latency (expect something between 50 ms and 500 ms) and allow for a decent concurrency (say, 100 QPS or maybe 1000 QPS).

Okay, but what would Cube do if we need to query for other granularity? Since it’s aware that queried measures are defined with distinct counts, it won’t be using the rollup and will transparently fall back to a query to the data warehouse (there’s an [option to disallow that](https://cube.dev/docs/caching/using-pre-aggregations#rollup-only-mode) at all times, if we need to).

![Non-accelerated SQL query](https://ucarecdn.com/a0d3bd13-889d-4a92-925d-984ee2dd9a4d/)

## Data consumers

Now, we’re ready to connect downstream tools. We can pick anything that supports Cube’s SQL API, and that’s practically any tool. Cube’s SQL API uses Postgres dialect and wire protocol, so if something works with Postgres as an upstream source, it works with Cube: 

![Supported data sources](https://ucarecdn.com/ba240ef2-11fc-477d-843d-39b0b9fe2649/)

We have numerous options here. Out of respect for the first tech company ever where the [first triple-name-tie is Claire](https://twitter.com/clairebcarroll/status/1570194384980971522), let’s connect Cube to [Hex](https://hex.tech):

![Connection to Hex](https://ucarecdn.com/7c89eb8a-d7dd-41b9-bcdb-c1889b70eba5/)

Indeed, [Hex works with Cube](https://cube.dev/blog/hex-integration-collaborative-data-notebooks):

![Hex notebook](https://ucarecdn.com/3761be9c-722c-461e-a1ba-7dcb01e1441b/)

In case we’re building a front-end app, we can directly `fetch` from the REST API endpoint or use the provided JavaScript client library:

```jsx
import cubejs from '@cubejs-client/core';

const cubejsApi = cubejs.CubejsApi(
  process.env.CUBE_TOKEN,
  { apiUrl: process.env.CUBE_API_URL }
);

const query = {
  measures: [ 'WorkspaceActivation.activation_rate' ],
  timeDimensions: [ {
    dimension: 'WorkspaceActivation.reporting_day',
    granularity: 'week'
  } ],
  order: {
    'WorkspaceActivation.reporting_day': 'desc'
  }
};

const resultSet = await cubejsApi.load(query);

console.log(resultSet.tablePivot());
```

Now you can plot the data with a [charting library](https://awesome.cube.dev/?tools=charts) of your choice:

![Data in the terminal](https://ucarecdn.com/dee1b9f9-74ff-4e06-8522-335922f2afd1/)

## Wrapping up

Hopefully, this write-up provides some insight into how Cube works and helps compare it to Looker, dbt Metrics, and Lightdash from the original [deep dive](https://pedram.substack.com/p/what-is-the-metrics-layer).

Also, if you’re using dbt Metrics, please keep in mind that you can leverage Cube’s [integration with dbt](https://cube.dev/blog/dbt-metrics-meet-cube) that transparently reads metrics definitions from dbt and automagically creates a data model from them.

Currently, Cube uses JSON-like markup for data model definitions and Looker-like metrics organization into cubes. Later in 2022, Cube is likely to [introduce YAML markup](https://forms.gle/4CZRRKK8nD4PvtxZA) for data models and an alternative approach to metrics.

With that, I do have an answer to the final question from Pedram’s article:

> Can a metrics layer be universal enough to gain applicability across the data stack yet still be designed in such a way to be relevant to BI tools?
> 

Yes. Try Cube.

The easiest way to get started with Cube is [Cube Cloud](https://cube.dev/cloud). It provides managed hosting and convenient tools for Cube, including the Playground, the data model editor with syntax highlighting, and—more importantly—the ability to instantly fork data models and query private versions of APIs while the production version stays intact and serves queries.

[Sign up for Cube Cloud](https://cubecloud.dev/auth/signup) on the free tier today and experience that yourself. Feel free to connect your production dataset or tinker with the Postgres instance used in this blog post:

```
Host:      demo-db-examples.cube.dev
Database:  ecom
User:      cube
Password:  12345
```

Deep Dives

Cube in the spotlight alongside Looker, dbt Metrics, and Lightdash.

Deep dive: What the heck is the Headless BI

The ongoing story of the digital age is still about [taking products apart](https://hbr.org/2014/06/how-to-succeed-in-business-by-bundling-and-unbundling). The wave of [bundling](https://towardsdatascience.com/the-great-data-debate-unbundling-or-bundling-7d7721ee8514), [unbundling](https://blog.fal.ai/the-unbundling-of-airflow-2/), and [rebundling](https://dagster.io/blog/rebundling-the-data-platform) happening in the data space has subsided, but the point remains. Generations of tools in the modern data stack, whatever it is, succeed each other primarily for two reasons:

- Emerging tools provide more flexibility and a lower entry barrier for practitioners.
- Alternatively, teams escape into a safe harbor of a well-known integrated solution that promises big returns on even bigger investments. Indeed, has anyone ever got fired for buying Looker?

Well, one can only so much joke about Looker. Things like Looker’s $2.6B acquisition by Google in 2019, Snowflake’s $3.4B IPO in 2020, or dbt Labs’ [four consecutive venture rounds](https://pedram.substack.com/i/54634871/the-devils-bargain-of-venture-funding) in 2020–2022 leading to a $4.2B valuation fuel the data industry and help put more tools into a data practitioner’s toolbox.

Today, we’re going to talk about Cube and Apache Superset—a bundle of tools representing a finer alternative to Looker in many use cases.

> We've had a webinar on replacing Looker with Cube and Superset. [Check the recording and slides.](https://cube.dev/events/replacing-looker-with-cube-and-superset)

## Free as in “open source”

[Cube](https://cube.dev) and [Superset](https://superset.apache.org) belong to different categories in the data stack. The former is the [headless business intelligence](https://cube.dev/blog/headless-bi) platform, and the latter serves as the data visualization and exploration platform. However, being open source and standing on the shoulders of giants are among many things that they have in common.

Both tools are licensed under Apache License 2.0 and actively developed on GitHub. Over the last 3 years, Cube’s [repository](https://github.com/cube-js/cube.js) has acquired almost 14,000 stars (an unmatched [leader in the category](https://twitter.com/sspaeti/status/1570044606338990081)). Written mostly in Rust, Cube’s data processing and storage are based on the [Arrow DataFusion](https://github.com/apache/arrow-datafusion) query execution framework which uses [Apache Arrow](https://arrow.apache.org/) as its in-memory format. Superset has a longer history (born at Airbnb in 2015 where it replaced Tableau, joined the Apache Incubator program in 2017), has a [repository](https://github.com/apache/superset) with 48,000 stars, and is powered by [Apache ECharts](https://echarts.apache.org/en/index.html), one of the most mature and popular charting libraries. 

![Cube and Superset](https://ucarecdn.com/5f3f2313-c055-4b9e-b345-56568c53aca2/)

You’re definitely free to take and use Cube and Superset at your own discretion, including self-hosting them in your private or public cloud. You’re also free to explore and get started with [Cube Cloud](https://cube.dev/cloud) and [Preset Cloud](https://preset.io/product/), fully managed platforms with freemium pricing models and plenty of quality of life features on top of open source products. Compare that to Looker’s “[request a demo](https://cloud.google.com/resources/looker-demo/?_ga=2.157282873.347220352.1663670208-344442256.1663670208)” paywall that would cost your team a few weeks of qualification calls on Google Meet before you start building.

## Simple as in “separation of concerns”

Cube is intentionally [headless](https://cube.dev/blog/headless-bi) and agnostic to the data presentation layer. However, it does a number of things and does them well:

- connects to a plethora of [data sources](https://cube.dev/docs/config/databases), from cloud data warehouses like BigQuery, Snowflake, and Redshift to streaming platforms like Kafka and Redpanda;
- provides a [data modeling](https://cube.dev/docs/schema/getting-started) layer as the centralized source of truth for metrics definitions;
- secures data with its [access control](https://cube.dev/docs/security) layer;
- makes sure any query runs well under a second with its [caching](https://cube.dev/docs/caching) layer;
- delivers data to any downstream tool, including BI tools like Superset, via its API layer that provides [SQL API, REST API, and GraphQL API](https://cube.dev/docs/config/downstream#ap-is-references).

Superset works by connecting to a data source and querying the data where it lives. The ideal way to use Superset is with a [dataset-centric approach](https://preset.io/blog/dataset-centric-visualization/) to modeling and visualization. 

A dataset-centric approach implies that a well-normalized dataset is used to “derive” richer datasets with extra semantics, making them valuable for analysis and visualization. By creating derived datasets that incorporate aggregate metric calculations earlier in the ETL/ELT process *before* surfacing for visualization, many of the issues with change management, maintenance, logic reuse within dashboards, etc. are significantly reduced for a better experience. 

Cube complements Superset in this way, and this allows for Superset-generated SQL queries to work with no issues when connected to a Cube environment (which is treated as a PostgreSQL database connection). 

![Looker vs Cube + Superset](https://ucarecdn.com/59031c52-890f-402f-9400-0d0d7034c887/)

You’re free to configure and fine-tune each tool and each layer within every tool when building with Cube and Superset. Want to build and schedule a report? Check Superset dashboards. Need to expose new metrics in that report? Cube’s data modeling layer is at your service.

## Compatible as in “no vendor lock-in”

Speaking of data modeling, Cube provides a [data modeling language](https://cube.dev/blog/open-source-data-modeling) that is not tied to any specific interface, visualization, or user experience. Unlike proprietary LookML definitions that require Looker IDE for syntax highlighting and linting, Cube’s data model is written in JavaScript (and [soon-to-be-written](https://forms.gle/4CZRRKK8nD4PvtxZA) in YAML) and feels native in your editor of choice. On top of that, you are able to generate a data model dynamically or even fetch it from a remote endpoint.

Consider an excerpt of Looker view from the recent [deep dive](https://pedram.substack.com/p/what-is-the-metrics-layer) by Pedram:

```yaml
view: workspace_activation {
  sql_table_name: "METRICS"."WORKSPACE_ACTIVATION" ;;

  dimension_group: date {
    type: time
    timeframes: [
      raw,
      time,
      date,
      week,
      month,
      quarter,
      year
    ]
    sql: ${TABLE}."REPORTING_DATE" ;;
  }

  dimension: is_active_workspace {
    type: yesno
    sql: ${TABLE}."IS_ACTIVE_WORKSPACE" ;;
  }

  dimension: workspace_id {
    type: string
    primary_key: yes
    sql: ${TABLE}."WORKSPACE_ID" ;;
  }

  measure: count_workspaces {
    type: count_distinct
    description: "# of Workspaces"
    sql: ${workspace_id} ;;
  }

  measure: count_active_workspaces {
    type: count_distinct
    description: "# of Unique Workspaces Active within 1 Day"
    sql: ${workspace_id} ;;
    filters: [is_active_workspace: "yes"]
  }

  measure: activation_rate {
    type: number
    sql: ${count_active_workspaces} / ${count_workspaces} ;;
    value_format_name: percent_1
  }
}
```

Compare with the equivalent Cube data model (way less of `;;`, to say the least):

```javascript
cube(`WorkspaceActivation`, {
  sql: `SELECT * FROM public.active_workspace_details`,

  measures: {
    count: {
      type: `count`
    },

    n_workspaces: {
      sql: `workspace_id`,
      type: `countDistinct`,
    },

    n_active_workspaces: {
      sql: `workspace_id`,
      type: `countDistinct`,
      filters: [ {
        sql: `${is_active_workspace} IS TRUE`
      } ],
    },

    activation_rate: {
      sql: `ROUND(100 * (1.0 * ${n_active_workspaces} / ${n_workspaces}), 2)`,
      type: `number`
    }
  },

  dimensions: {
    workspace_id: {
      sql: `workspace_id`,
      type: `number`,
      primaryKey: true
    },

    reporting_day: {
      sql: `reporting_day`,
      type: `time`
    },

    is_active_workspace: {
      sql: `is_active`,
      type: `boolean`
    }
  },

  preAggregations: {
    main: {
      measures: [ activation_rate ],
      timeDimension: reporting_day,
      granularity: `week`,
      refreshKey: {
        every: `1 day`
      }
    }
  }
});
```

In a similar fashion, Cube’s API layer is compatible with practically any data consumer. When the time comes, and you realize that, alongside business users using dashboards and reports in Superset, you’d like to have data analysts crunching numbers and seeking insights in data notebooks, Cube will be there to help liberate your data.

![Data consumers](https://ucarecdn.com/b35f44ef-12a5-42d7-a618-74637fc3e0cd/)

You can expect any SQL query generated by your data exploration tool like Superset to work with Cube’s SQL API. Needless to say, equivalent queries to Cube’s REST API and GraphQL API will yield the same data:

```sql
SELECT
  DATE_TRUNC('WEEK', reporting_day) AS reporting_week,
  activation_rate
FROM WorkspaceActivation
ORDER BY 1 DESC;
```

Let’s have a bird’s-eye view here.

The generation of BI tools like Looker encouraged end-users to invest heavily in building out tons of LookML models to populate the semantic layer. While this established Looker as a source of truth in an organization for business metrics, it also created an immense amount of lock-in in your BI tool. You couldn’t take your LookML models with you and use them with another tool if you decided to switch BI tools. Google’s acquisition of Looker accelerated organizations’ anxieties around this lock-in, to the point that Google was actually [forced to integrate LookML with Tableau](https://roundup.getdbt.com/p/look-ml-tableau-metrics-layer). This way, organizations could use LookML for transformation (Looker’s strength) and Tableau for visualization (Tableau’s strength).

The future however is looking to be very different from the past:

- Open-source headless BI platforms like Cube provide unified semantic layers to sit between the database and BI later, eliminating the need, or providing alternate implementations, for expensive solutions like Looker that lock you in.
- Open-source BI tools like Superset with an intentionally thin semantic layer complement Cube and enable *last-mile data transformation* for the explicit purpose of data visualization in your BI tool.

When the need for a complex yet flexible metrics/semantic layer arises, Cube and Superset is a winning open source-based combo!

## Ready-to-go as in “today”

Now, you’re very much welcome to try Cube and Superset today and see them in action. Please join the Slack communities of [Cube](https://slack.cube.dev) and [Superset](https://join.slack.com/t/apache-superset/shared_invite/zt-16jvzmoi8-sI7jKWp~xc2zYRe~NqiY9Q) to share your feedback.

Please tune in to the [webinar on October 4](https://cube.dev/events/replacing-looker-with-cube-and-superset) with [Shreesham Mukherjee](https://www.linkedin.com/in/shreesham/), Developer Relations Engineer at Preset, and [Igor Lukanin](https://www.linkedin.com/in/igorlukanin/), Head of Developer Relations at Cube. We’ll discuss Cube and Superset as an open source alternative to Looker in popular use cases, demo both tools, and take your questions.

See you!

Shreesham Mukherjee

Looker

Respectfully unbundling the elephant in the BI room.

Open source Looker alternative with Cube and Apache Superset

Demand to make data operational and real-time is ever-growing. Both consumers and enterprises expect applications to react to changes in real time, and be proactive with intelligent notifications and alerts.

## How can Cube help?
At Cube, our primary [sources of data][1] have been cloud data warehouses with batched data—but since our first days, we have wanted to work with streaming data too. Our ambition has been to create a seamless experience for data engineers to build applications on top of a single, headless data access layer, regardless of whether the data is a stream, a batch, or a union of both. 

Today, we’re happy to announce a major step in building streaming support into Cube, and to share our plans for what’s next.

> We are hosting an online event to reveal these features. [Register now][2] to join us on October 13.

## The challenge

Historically, it has been hard to work with truly real-time data. Specifically, it’s been challenging to combine historical and real-time data. For example, in use cases such as notifications or recommendations, we can’t rely *only* on last-minute data; we need to look back to run analysis, and then act in real-time, based on *both* the most recent data and historical trends.

There’s also the concern of needing to master unique languages for specific data sources; in the past, it often was necessary to build one-off integrations for each streaming application to accommodate real-time streams.

## The opportunity

Streaming data enables a variety of use cases, including real-time, in-product analytics; automation; personalization; and alerting. Fortunately, recent innovations have made it easier for us to accommodate real-time streams.

In the last few years, streaming SQL technologies such as [ksqlDB][3], [Materialize][4], and [Apache Flink][5] have significantly progressed. These technologies enable us to process streaming data and run analysis with SQL—without needing to learn a new language or build specific language-unique integrations.

## ksqlDB support

We’re excited to announce that **it’s now possible to use Cube to build data modeling, caching, and access control layers on top of streaming SQL**, just as you can with data from cloud data warehouses.

![Cube schema with ksqlDB](https://ucarecdn.com/e96f17b3-82be-431b-ad37-bed225c49322/)

Cube now can connect to streaming SQL engines and expose streaming data via our REST, GraphQL, and SQL APIs to your downstream applications. Our REST API already supports websockets and provides a seamless experience for developers and data engineers to build real-time data apps. Our data modeling, caching, access control, and querying APIs are the same whether you work with streaming or batch data.

Want to see it in action? [Check out our live ksqlDB demo application][9] featuring a real-time dashboard built with Kafka, ksqlDB, and Cube.

That alone opens many opportunities for building on top of streaming data, but we’re also committed to addressing the complex problem of merging batch and streaming data, so that we can provide a single interface for unioned data.

## Lambda architecture support

We’re also introducing [lambda pre-aggregations](https://cube.dev/docs/caching/pre-aggregations/lambda-pre-aggregations). These follow the [lambda architecture][6] design to create a union of streaming and batch data.
Cube pre-aggregates batch data from the cloud data warehouse into one pre-aggregation, data from streaming SQL engines into another, then unites these pre-aggregations during query processing—so we can return merged data.

![Cube schema with lambda architecture](https://ucarecdn.com/466fe9e6-201e-483f-a142-5eefd1987576/)

For developers or data consumers, the interface remains the same—REST, GraphQL, or SQL queries. Our Lambda architecture makes it possible to consume data as a single dataset regardless of whether it is streaming, batch, or combined data.

Cube hides the complexity of this architecture within our data modeling layer, and therefore significantly simplifies the data consumption process. In addition, our access control and security layers are applied using the same rules as you’ve configured them for batched data.

## Coming next

Naturally, our work isn’t done yet. The above features are generally available soon—and available today to Cube users who request access. Soon, you’ll also have access to

- [Materialize](https://cube.dev/docs/config/databases/materialize) streaming pre-aggregations
- Flink SQL support
- Spark Streaming support

If you’re interested in early access to these features, to help influence their development and craft the future of streaming headless BI, simply [get in touch][7].

## Everything you need to build powerful real-time applications

We’re excited to launch our Kafka and ksqlDB integration and lambda pre-aggregations today because these bring the power and simplicity of our headless BI architecture to new types of data applications.

These innovations are available to everyone, for free: [request ksqlDB preview access][8], connect a real-time data source, then (please!) give us your feedback.

We look forward to powering the next generation of data applications—we can’t wait to see what you build.

[1]: https://cube.dev/docs/config/databases
[2]: https://cube.dev/events/building-real-time-streaming-data-applications-with-cube
[3]: https://ksqldb.io
[4]: https://materialize.com
[5]: https://flink.apache.org
[6]: https://en.wikipedia.org/wiki/Lambda_architecture
[7]: https://cube.dev/contact
[8]: https://cube.dev/contact
[9]: https://ksql-demo.netlify.app/

All Things Cube

Headless BI with streaming data

Data pipelines, data modeling, and BI tools to collect, store, and analyze data at scale.

Data Engineering

Front-end applications and low-code tools to bring data insights to end users.

Application Development

Company news, product updates, and engineering deep dives.

Latest news and insights from Cube