We [launched Cube Cloud](https://cube.dev/blog/announcing-cube-cloud) back in 2021 and have been busy ever since launching new features, growing Cube Cloud usage, and listening to your feedback. Today, we are happy to make a major update to Cube Cloud pricing.

This update—measuring usage with Cube Compute Units—makes pricing more predictable and reflects the diversity of enterprise customers using Cube Cloud with custom-tailored pricing plans. Let’s walk through these changes.

## Transparent and predictable pricing

We have always wanted Cube Cloud pricing to be transparent and predictable for Cube Cloud users, so that it reflects the actual usage of the service and can be forecast in advance. That is why we think that *consumption-based pricing* is the best option for Cube Cloud.

Initially, we chose *data pass through*—defined as the amount of data transferred between Cube Cloud and API clients or database—as the measure of resource consumption. It definitely helped users quantitatively understand how much data flows from a data warehouse to Cube and from Cube to data consumers.

However, this model turned out to not be transparent and predictable enough. Often, the most pragmatic way to understand the expected data pass through of a deployment was to actually make it live and then measure. Also, changes to data model, caching configuration, queries, etc. would sometimes result in step changes and make data pass through hard to predict. For these reasons, we’ve decided to introduce a better alternative.

### Cube Compute Units (CCUs)

Now, our users’ resource consumption is measured in *Cube Compute Units*, which precisely and transparently reflect the compute required to run certain Cube components, i.e., API instances and development instances, refresh workers, Cube Cloud routers and workers. (Note that Redis instances are not present in this list since we have [migrated from Redis](https://cube.dev/events/migrating-from-redis) to Cube Store.)

![Cube Compute Units](https://ucarecdn.com/be1a7fa4-0f3e-446e-8de0-aa0c9705ae3d/)

Let’s explain. Assume that you have a minimal yet very potent Production Deployment that includes two Cube API Instances (enough for serving up to 20 requests per second), a Cube Refresh Worker, and two Cube Store Routers. You also choose to include two Cube Store Workers because you want to use pre-aggregations and provide your users with blazing-fast experience:

![Architecture](https://ucarecdn.com/d7dc276b-54b8-422e-a549-6710a1ba5eed/)

We can go through a few real-world scenarios for that deployment:

1. Running this deployment would consume 4 to 6 CCUs per hour or approximately 2900 to 4400 CCUs per month (keep in mind that when pre-aggregations are not being refreshed or used to serve queries, Cube Store Workers are scaled down).
2. Additionally, if every workday at peak hour your API autoscales from 2 to 10 instances, that would add roughly 200 CCUs per month.
3. Also, if two engineers on your team hack on the data model for a couple of hours every workday while running their Development Instances, that would add about 100 CCUs per month.

For simplicity, we used whole hours in the calculation. However, __Cube Cloud will actually scale instances up and then back down within minutes, so you’re never overcharged for the unused resources.__

### Updated Billing Experience

We believe that this flavor of consumption-based billing, popularized by many industry leaders such as Snowflake (credits) and Databricks (DBUs), is already well-known to many Cube Cloud users and the overall data community.

To further help Cube Cloud users understand and forecast their usage and related costs, we also updated the Billing page in Cube Cloud. Now, you’re able to break down usage by deployments and services, review it on a daily and monthly basis, and monitor spend in CCUs and USD:

![Billing interface](https://ucarecdn.com/eb6bc535-82e0-4545-b3cb-11c8cad872f7/)

Needless to say, this page is powered by Cube and provides near real-time experience.

## New plans for production deployments

Over the last couple of years, we’ve accumulated vast knowledge of Cube Cloud users. Our familiarity with our user base provided us with the insights to launch many new features and update pricing plans so they’re tailored to their everyday needs.

You can see the entirety of new plans on the updated [pricing page](https://cube.dev/pricing).

### Try a Development Instance of Cube Cloud, free of charge

One hill we’ll die on? Using Cube Cloud is the *best* way to get started with Cube.

Fully managed, secure, and packed with features that provide a smooth developer experience, Cube Cloud should be available to every single engineer getting started with Cube. So, in that vein,, we continue to provide an option to [try Cube Cloud for free](https://cubecloud.dev/auth/signup) for your non-production or proof-of-concept deployment.

This option empowers you to evaluate Cube indefinitely and upgrade to any of the following plans when you’re ready to go into production.

### Choose a plan for production deployments

We’re introducing four new plans: [Starter, Premium, Enterprise, and Enterprise Premier](https://cube.dev/pricing).

Every plan has a different price per CCU—and for a good reason. We’ve built these plans so they reflect the needs of different Cube Cloud users: from seed-stage startups to Fortune 500 companies with mission-critical data applications.

A few highlights regarding these new plans:

* __Starter__ plan will let you run deployments on AWS and GCP, collaborate as a team using [development mode](https://cube.dev/docs/cloud/workspace/development-api) and [data model IDE](https://cube.dev/docs/cloud/workspace/cube-ide), and have full observability with real-time application performance monitoring (APM) and [alerts](https://cube.dev/docs/cloud/workspace/alerts).
* __Premium__ plan guarantees 99.950% uptime, lets you run deployments in a [dedicated virtual private cloud (VPC)](https://cube.dev/docs/cloud/configuration/connecting-with-a-vpc), supports Azure in addition to AWS and GCP, and provides your team with single sign-on (SSO) via Okta.
* __Enterprise__ plan guarantees 99.990% uptime, provides VPC peering, [logs and metrics export](https://cube.dev/docs/cloud/workspace/logs), SSO via SAML/LDAP as well as [role-based access control (RBAC)](https://cube.dev/docs/cloud/access-control/). You will also get a dedicated customer support manager (CSM) who would be in direct contact with you at every step of your Cube Cloud experience.
* __Enterprise Premier__ plan guarantees 99.995% uptime, provides unlimited scalability with multi-cluster deployments as well as connectivity to Elastic and [Kafka (via ksqlDB)](https://cube.dev/docs/config/databases/ksqldb) for [real-time streaming](https://cube.dev/events/building-real-time-streaming-data-applications-with-cube). You will also get the utmost level of support for your mission-critical data applications.

Indeed, Cube Cloud is packed with features for organizations of all sizes, so one could get overwhelmed with the available options—but don’t fret.  We’re always happy to help out, so just [get in touch](https://cube.dev/contact) with us.

## What’s next

Our updated pricing with new plans, Cube Compute Units, and great billing experience are already available to all new Cube Cloud users. Check our updated [pricing page](https://cube.dev/pricing) and [start using Cube Cloud](https://cubecloud.dev/auth/signup) today. Also, don't hesitate to [talk to us](https://cube.dev/contact) so we can help you get started.

Note that nothing is changing for existing Cube Cloud users on Standard and Enterprise plans as of right now. If you’re in this amazing category, thank you for your support. If you see something in the new plans that gets you excited, [please reach out](https://cube.dev/contact) and we will get you set up. Or—just carry on using Cube Cloud on your current plan and we’ll get in touch sometime soon to assist you with upgrading to new plans.

If you have any questions or feedback regarding our updated pricing, [please reach out](https://cube.dev/contact); we’d be happy to walk you through how it works. Or share your thoughts in the `#cube-cloud` channel in our [Slack community](https://slack.cube.dev)—because even with almost 8,000 data practitioners, it’s still never too late to join 🙂

Igor Lukanin

News & Product Updates

Pay for what you use: Cube Cloud pricing gets back to its roots

__Today, we are immensely proud to announce that [Jen Grant](https://www.linkedin.com/in/jencgrant/) is joining us as COO.__ 

How to sum up Jen Grant in three words? Seasoned-leader. Skiing-aficionado. Uniquely-good-at-taking-companies-from-early-stages-to-billion-dollar-outcomes. 

We’ll be transparent—it’s in our best interest to brag here about our new COO’s achievements; fortunately for our marketer, her resume reads like the player card of a career NBA MVP (but, you know, tech). So, here are some of Jen’s stats: 

A graduate of Princeton and Wharton, she began her marketing career at [Google](https://about.google/); she closed her time there as a leader of several product marketing teams, having launched products such as Gmail, Calendar, Talk, and Blogger. 

After Google, Jen joined [Box](https://www.box.com/) as its first executive marketing hire—she took the company from obscurity to serving 95% of F500 companies and all the way to a [$1.7B IPO](https://techcrunch.com/2015/01/23/box-skyrockets-50-to-more-than-21-per-share-in-first-minutes-as-a-public-company/). 

Jen then served as CMO of [Elastic](https://www.elastic.co/), where she led its rebranding and established its marketing team. 

And subsequently, Jen continued on to become [Looker](https://www.looker.com/)’s CMO, where she grew the marketing team from 60 to 150+ and led the company to its [$2.6B acquisition by Google](https://techcrunch.com/2020/02/13/google-closes-2-6b-looker-acquisition/).

Following *that* absurdly successful tenure, she co-founded and served as [Appify](https://appify.com/)’s CEO, where she is now an advisor to the board. She also sits on the boards of Kyndi, Dialpad, and Pecan AI and advises Stytch, Waitlist.me, and Postal.io.

As Cube continues to expand and hit major milestones, our product-market fit is evident—and so is our need to continue to grow our team of (humble) rockstars with (humble) rockstar leadership. From her [widely recognized](https://csuitespotlight.com/2021/06/06/the-top-100-princeton-alumni-in-technology-of-2021/) experience in data, it's clear that Jen has a love for building organizations and a knack for spotting the critical problems to solve and the right solutions to solve them—making our pairing an ideal match.  

With all of this, it's easy to understand why we are so excited to have Jen join us as our COO. 

Welcome, Jen!

## A note from Jen:

> “There is so much potential here that I just couldn’t sit on the sidelines any longer.” 

Cube came across my desk about half a year ago—when I had resolved to take the winter off from work. The thing is, my intrigue in the massive problem Cube is solving and how it's solving it eclipsed even my love of skiing. 

As the CMO of Looker, I was acutely aware of how costly inconsistent data was to organizations. Businesses had many data sources, even more data apps, and consequently, way, way too many data silos which stood in the way of producing meaningful insights. 

What I also understood was that this was happening in almost all companies, [86% of which use two BI tools simultaneously, and 61% of which use four or more](https://www.forrester.com/blogs/the-bi-fabric-baby-is-slowly-but-surely-growing-up/): businesses *wanted* to do analysis over all their data sources but knew that the tech to make that possible didn’t exist (even Looker was only a step towards solving this challenge)

Too late for me to use it at Looker, but at the perfect time for me to jump at the incredible opportunity to help build it, I learned about Cube. 

As a [universal semantic layer](https://cube.dev/use-cases/semantic-layer), Cube solves the problem we shared at Looker with so many other organizations. Cube allows companies to easily connect to multiple cloud data warehouses, define metrics consistently through a single upstream data model, and feed that consistent data to any data app. It addresses security issues with its centralized data access controls, data performance issues with its two-level caching layer, and stack incompatibility issues with its instant APIs. 

So, yes, I was *personally* familiar with the problem Cube is fixing. 

But also, the proof of the entire market’s need for Cube is in the product-led growth pudding: initially just an open-source product, the company surpassed $1MM ARR in only a year—and by a healthy amount. We’re continually growing, [we’re selling](https://cube.dev/contact), we’re raising, and [we’re hiring](https://cube.dev/careers)—and knowing all of this, the decision to put away my skis and join Cube as COO was easy to make. 

There is so much potential here that I just couldn’t sit on the sidelines any longer.

Onwards and upwards,

Jen


Jen Grant

“There is so much potential here that I just couldn’t sit on the sidelines any longer.”

Announcing Jen Grant as Cube’s COO

At Cube, we're on a mission to provide developers with an analytical data access layer to help them build modern applications. We call this data access layer a [’headless BI’ platform](https://cube.dev/blog/headless-bi).

The goal of headless BI is to facilitate consistent, performant, and secure access to all of your data. Of course, this mission only works if we—that is, [Cube, the headless BI platform](https://cube.dev/)—can simultaneously support all of your data sources and make those data sources compatible with the broadest array of downstream tools possible. 

So today, we’re proud to launch our new [integrations page](https://cube.dev/integrations)—so that you can find and read all about how your favorite tools integrate with Cube (and in that, the rest of your stack.)

## The inspiration: complexity

Cube was borne out of the sheer complexity of the modern data stack—one that exists not only because of the fractal-like explosion of data sources (cloud data warehouses, transactional databases, streams, document databases, query engines, etc.) but also in the types of tools with which users consume data.

Yes, for better or worse, gone are the days of an organization simply choosing a single business intelligence tool and connecting it to a transactional database. 

Now, users expect to gain insights from their data with both visualization and business intelligence tools, and with notebooks, and by building their own applications with low-code tools, powerful front-end frameworks, and charting libraries. And naturally, this expectation and the remarkable number of options available result in incredible building ability—but potentially mind-boggling connectivity complexity.

![](https://ucarecdn.com/60b3a40f-79aa-40d6-803a-928de8d7bfcc/)

## The goal: universal compatibility 

Universal compatibility—meaning the ability of every data source to connect to every downstream use case—is a tall order. The good news? We have an exceptionally well-known, durable, and scalable language on our side: SQL. [Cube’s Postgres-compliant SQL API](https://cube.dev/docs/backend/sql) is the magic translation layer by which we’re able to make universal compatibility happen. 

So, perhaps your business intelligence tool doesn’t have a native driver for your fancy new analytical query engine. Well, now there’s a solution for that: if you connect your query engine to Cube, Cube’s SQL API can expose it to whichever downstream, SQL-speaking tools you choose with a simple Postgres connection. 

Or, let’s take another scenario: what if you were building a [customer-facing analytics application](https://cube.dev/use-cases/embedded-analytics)? In that case, maybe you’d prefer to expose connections to your data sources as a [REST](https://cube.dev/docs/rest-api) or a [GraphQL API](https://cube.dev/docs/backend/graphql)? Cube could do that too. A rigid out-of-the-box business intelligence tool just can’t offer the same flexibility as Cube, with which you could then use any number of front-end frameworks or low-code tool builders.

![](https://ucarecdn.com/86bdb379-b989-4e68-bb71-df784c819bd9/)

## The future: new data sources, new tools, and much (much) more. 

We believe that you should be able to curate your stack with any data source and downstream tools you want. 

And we believe in open standards and in the power of a broad ecosystem.

These tightly held values mean that if you’ve come across a fantastic tool for which we haven’t yet listed an integration, [we’d love to hear from you](https://cube.dev/contact). Because really—we built this tool with a dedication to the data community and are determined to hone it further for the community (you can also find us on [Slack](https://slack.cube.dev/) and [Github](https://github.com/cube-js/cube.js).)

Even with all of the integrations we’ve built out already, this is just the beginning of our plans for the Cube ecosystem. So looking ahead, we’ll continue adding more data sources, more downstream tools, and more categories within the modern data stack. 

We’re on an exciting journey, here—and we hope you’ll join us for the ride :) 


Brian Bickell

So you can easily find your favorite data source or tool in Cube's integrations. 

Introducing Our Integrations Gallery

Today, we're happily announcing that Cube now works with [Trino](https://trino.io), the fast distributed SQL query engine for big data analytics, formerly known as PrestoSQL.

> [Meet the Cube team at Trino Summit](https://cube.dev/events/cube-at-trino-summit) on November 10, 2022. We'll be happy to chat and explore how Cube can augment your Trino experience.

## What is Trino?

Trino is an [open-source](https://github.com/trinodb/trino) query engine designed to work with big data.

It's fast, scalable, SQL-compliant, and has almost universal connectivity to all kinds of data sources and business intelligence tools. Let's explore these in more detail:

* **Scale.** Trino is capable of querying exabyte-scale data lakes and massive data warehouses, as confirmed by [Trino installations](https://trino.io/users.html) at companies such as LinkedIn, Netflix, and Shopify.
* **Performance.** Trino is a highly parallel and distributed query engine with two types of [cluster nodes](https://trino.io/docs/current/overview/concepts.html#server-types): coordinator nodes, responsible for query planning and execution, and worker nodes, responsible for fetching and processing data. You can fine-tune your Trino cluster to allow for efficient, low latency analytics.
* **Connectivity.** Trino can natively query data from a plethora of [data sources](https://trino.io/docs/current/connector.html) without the need to extract and load the data into a data warehouse, even if you'd like to do a cross-database join. Also, Trino is an ANSI SQL-compliant query engine that works with BI tools such as Tableau, Power BI, and Superset as well as headless BI tools such as Cube.

Most common use cases for Trino include ad-hoc analytics at interactive speeds, massive multi-hour batch queries, and high volume apps that perform sub-second queries.

Trino was originally [designed and developed at Facebook](https://trino.io/blog/2020/12/27/announcing-trino.html) in 2013 and beared the name *Presto* at that time. In 2019, Presto development forked and two query engines emerged: PrestoDB maintained by Facebook and PrestoSQL maintained by the Presto Software Foundation and the original creators. Later in 2020, PrestoSQL was renamed to Trino.

If you're not a Trino user yet, consider [self-hosting](https://trino.io/docs/current/installation.html) or using a managed platform provided by [Starburst](https://www.starburst.io) or AWS (as [Amazon Athena](https://aws.amazon.com/athena/), serverless interactive query service). 

## What is Cube?

Cube is the [headless BI](https://cube.dev/blog/headless-bi) platform for accessing data from modern data stores (including Trino), organizing it into consistent metrics definitions, and delivering them to downstream applications.

![Cube schema](https://ucarecdn.com/5f6287b5-fd33-47c8-95be-2dffd84ccfe2/)

Cube is designed to take the central part in the data pipeline, delivering consistent data to all downstream teams and data consumers. It serves as a source of truth for the metrics definitions, access control rules, and caching settings. Regardless of how many data consumers you have (e.g., front-end applications with embedded analytics or BI tools), Cube will deliver consistent data to all of them with its [REST API](https://cube.dev/docs/rest-api), [GraphQL API](https://cube.dev/docs/backend/graphql), or [SQL API](https://cube.dev/docs/backend/sql).

## How Cube works with Trino

In a typical data pipeline, Trino is placed upstream of Cube, providing unrestricted access to all data sources as well as data federation capabilities. You can use Cube to create an additional [semantic layer](https://cube.dev/blog/open-source-data-modeling) or a last-mile [caching layer](https://cube.dev/docs/caching) on top of Trino.

More importantly, you can use the set of APIs that Cube provides, including REST API and GraphQL API, to deliver the data directly to custom-built front-end applications, retaining low latency and high concurrency.

![Cube and Trino architecture](https://ucarecdn.com/cc996b6f-5f7d-4efb-83e6-960fc63546cd/)

Also, you can leverage Cube's [multitenancy](https://cube.dev/docs/multitenancy-setup) and [access control](https://cube.dev/docs/security) features to fully control and customize how every user, role, or data consumer accesses the data in Trino and underlying data sources.

## Try Cube with Trino

Cube is [open source](https://github.com/cube-js/cube.js), so you can feel free to [self-host and deploy](https://cube.dev/docs/deployment/overview) it. You can also run fully managed Cube in [Cube Cloud](https://cube.dev/cloud) which is the fastest and most convenient way to use Cube.

Start by signing up for a free [Cube Cloud account](https://cubecloud.dev/auth/signup). You'll be prompted to select the cloud provider (AWS, GCP, or Azure) and a region for your deployment:

![Cube Cloud: 1st step](https://ucarecdn.com/7ae7b4ae-57a1-4fb9-9e3d-bfa3fc011a67/)

Then, pick Trino from the data source options:

![Cube Cloud: 2nd step](https://ucarecdn.com/9dcf732a-eb10-4be9-bb8b-685e9060c5f0/)

Lastly, provide the credentials for the Trino connection:

![Cube Cloud: 3rd step](https://ucarecdn.com/aa796b76-ee85-4aae-ad66-c2a7e8ced05a/)

In a few seconds, you'll get your Cube Cloud deployment up and running, ready to query your Trino installation and deliver data to downstream applications.

## What's next?

Would you like to learn more about Cube and what it brings to the table? [Check the docs](https://cube.dev/docs/) and create a free [Cube Cloud account](https://cubecloud.dev/auth/signup) today.

Also, please feel free to [join our community](https://slack.cube.dev/) of almost 7000 data practitioners on Slack, give Cube a [star on GitHub](https://github.com/cube-js/cube.js), or [schedule a time](https://cube.dev/contact) to chat 1:1.

Trino

Perform interactive big data analytics with a fast and distributed query engine.

Announcing the Cube integration with Trino, the SQL query engine for big data 

Since we open-sourced Cube in 2019, we’ve added quite a few new features, such as our [new SQL API](https://cube.dev/blog/expanded-bi-support), and integrations, such as our recently-announced [support for streaming databases](https://cube.dev/blog/headless-bi-with-streaming-data). However, we haven’t changed our core [data modeling layer](https://cube.dev/blog/open-source-data-modeling)—until now.

> Today, we’re announcing a new feature in Cube’s data model: views.

Designed to sit on top of the data graph of cubes, views create a façade of your whole data model with which data consumers can interact. They’re the place to define metrics, manage governance and data access, and control ambiguous join paths.

> You can watch the recording of our [webinar](https://cube.dev/events/using-cube-views-for-metrics-management) with an in-depth conversation about views.

## Why views? Why now?

As the [headless BI](https://cube.dev/blog/headless-bi) platform, Cube helps define and maintain metrics definitions. The fundamental constraint is that metrics should make sense for every downstream application (solved by Cube's SQL API) and every business user at the same time. They should represent substantial entities in a user's business domain and not be obscured by any tool in the data stack. However, we have to admit there's an [open debate](https://benn.substack.com/p/ghosts-in-the-data-stack) whether metrics should only describe tangible, real-world entities like *users* or *orders* or they're also allowed to reflect more ephemeral concepts like *sales funnel*, *recurring revenue*, or *storage quota usage*.

We're introducing views so Cube's users are well-equipped to build fully semantic layers, i.e., expose metrics that make sense in the business domain at all times and don't include anything obscuring or unnecessary. Using an analogy from software engineering, views allow to define the "public API" on the surface of your semantic layer. 

![Views Schema](https://ucarecdn.com/06ef6be9-14a3-40ec-a05c-88151781ee83/)

Over the past years, we’ve accumulated a lot of feedback across many use cases that has boiled down to these three questions:
1. How should single-measure metrics be defined?
2. How should you manage the visibility of cubes to, e.g., restrict access for specific data consumers?
3. How can you control the join path within the data graph?

We’ve designed views to tackle these and many other use cases. Now, let’s take an in-depth look at them. (It's worth noting that, if you're familiar with Looker and LookML, you can find some similarities between views and Looker's [explores](https://cloud.google.com/looker/docs/reference/param-explore-explore).)

## 1. Defining metrics with views

Views with a single measure can be used to define metrics.

Here’s a simple example that demonstrates the relationship between a view and its underlying cubes. To define a metric with a view, pick a measure from your data model together with the dimensions:

![Views explained schema](https://ucarecdn.com/90c5e207-9bdc-48a6-b8d7-f58a760ed2c4/)

You can use the [includes](https://cube.dev/docs/schema/reference/view#parameters-includes) property to bulk add measures or dimensions to a view or define specific member as a reference in case you need more control, e.g., to rename the dimension.

<CodeTabs>
```JavaScript
view(`ActiveUsers`, {
  description: `14 days rolling count of active users`,

  includes: [
    // Measure
    Users.rollingCount,
    // Dimensions
    Users.isPaying,
    Users.signupDate
  ],

  dimensions: {
    companyName: {
      sql: `${Company.name}`,
      type: `string`
    }
  }
});
```
```YAML
views:
  - name: active_users
    description: "14 days rolling count of active users"
    includes:
      # Measure
      - users.rolling_count
      # Dimensions
      - users.is_paying
      - users.signup_date
    dimensions:
      - name: company_name
        sql: "{company.name}"
        type: string
```
</CodeTabs>

Within a view, you can use expressions to define a new metric that’s derived from calculations of one or more measures:

<CodeTabs>
```JavaScript
view(`OrdersMargin`, {
  description: `Total margin of orders`,

  measures: {
    margin: {
      sql: `${Orders.revenue} - ${Orders.costOfGoodsSold}`,
      type: `number`
    }
  },

  includes: [
    Orders.date,
    ProductCategory.categoryName
  ]
});
```
```YAML
views:
  - name: orders_margin
    description: "Total margin of orders"
    measures:
      - name: margin
        sql: "{orders.revenue} - {orders.cost_of_goods_sold}"
        type: number
    includes:
      - orders.date
      - product_category.category_name
```
</CodeTabs>

You can query these metrics views via REST or SQL API the same way you query cubes. In Cube’s SQL API, views are exposed as tables in which measures and dimensions are columns.

```sql
SELECT
  margin,
  categoryName
FROM OrdersMargin
GROUP BY 2
```

> Curious about YAML? We have seen a lot of requests to support YAML for data modeling, and we’re planning a release later this year. Please stay tuned for updates and [reach out to us](https://forms.gle/4CZRRKK8nD4PvtxZA) if you’d like to sign up for early access to YAML support.

## 2. Managing visibility and governance

Views can help you manage visibility of your data model through the [shown](https://cube.dev/docs/schema/reference/view#parameters-shown) property. Some data may be available only to specific roles within an organization or to specific customers.

In the example below, we’re making the `ARR` view only visible to users with finance permissions:

<CodeTabs>
```JavaScript
view(`ARR`, {
  description: `Annual Recurring Revenue`,
  shown: COMPILE_CONTEXT.permissions.finance,
  includes: [
    Revenue.arr,
    Revenue.date,
    Customers.plan
  ]
});
```
```YAML
views:
  - name: ARR
    description: "Annual Recurring Revenue"
    shown: COMPILE_CONTEXT.permissions.finance
    includes:
      - Revenue.arr
      - Revenue.date
      - Customers.plan
```
</CodeTabs>

You can also use the `shown` property to manage visibility of the specific measures and dimensions based on the users’ permissions. 

## 3. Controlling join paths 

Sometimes your data model may feature more than one way to join two cubes. The schema below illustrates a simplified example of that case, where the `Countries` cube can be accessed from `Orders` either directly or via the `Users` cube.

The join path is a part of your semantic model since it affects the meaning of calculations. In the example below, we can look either at an order’s warehouse country or the country of the user who placed the order. 

![Join in Cube data model](https://ucarecdn.com/c9f27f2c-2185-4016-b06a-bdedd3546bcc/)

Use views to control the join path. In the example below, we are creating two dimensions with the same `Countries` cube and name dimension, but with different semantic meaning because of the join path.

<CodeTabs>
```JavaScript
view(`CompletedOrders`, {
  description: 'Count of orders',

  includes: [
    Orders.count
  ],

  dimensions: {
    orderWarehouseCountry: {
      sql: `${Orders.Countries.name}`,
      type: `string`
    },
    userCountry: {
      sql: `${Orders.Users.Countries.name}`,
      type: `string`
    }
  }
});
```
```YAML
views:
  - name: completed_orders
    description: "Count of orders"
    includes:
      - orders.count
    dimensions:
      - name: order_warehouse_country
        sql: "{orders.countries.name}"
        type: string
      - name: user_country
        sql: "{orders.users.countries.name}"
        type: string
```
</CodeTabs>

## Available now

Views are available in both [Cube Core](https://github.com/cube-js/cube.js) and [Cube Cloud](https://cube.dev/cloud) starting from [version 0.31](https://github.com/cube-js/cube.js/releases/tag/v0.31.0) ([see docs for details](https://cube.dev/docs/schema/reference/view)). However, they’re optional: if you don’t need them, don’t use them.

Views can be very helpful to develop and maintain large data models or provide governance for many data consumers, but they may be overkill for smaller projects.

In addition to the three cases detailed here, we believe views will enable many additional opportunities, so we’re eager to see how you use them. Please [get in touch](https://slack.cube.dev) with your feedback, questions, and examples.

Artyom Keydunov

Our biggest update to Cube’s data modeling layer.

Introducing Views for metrics management

Demand to make data operational and real-time is ever-growing. Both consumers and enterprises expect applications to react to changes in real time, and be proactive with intelligent notifications and alerts.

## How can Cube help?
At Cube, our primary [sources of data][1] have been cloud data warehouses with batched data—but since our first days, we have wanted to work with streaming data too. Our ambition has been to create a seamless experience for data engineers to build applications on top of a single, headless data access layer, regardless of whether the data is a stream, a batch, or a union of both. 

Today, we’re happy to announce a major step in building streaming support into Cube, and to share our plans for what’s next.

> We are hosting an online event to reveal these features. [Register now][2] to join us on October 13.

## The challenge

Historically, it has been hard to work with truly real-time data. Specifically, it’s been challenging to combine historical and real-time data. For example, in use cases such as notifications or recommendations, we can’t rely *only* on last-minute data; we need to look back to run analysis, and then act in real-time, based on *both* the most recent data and historical trends.

There’s also the concern of needing to master unique languages for specific data sources; in the past, it often was necessary to build one-off integrations for each streaming application to accommodate real-time streams.

## The opportunity

Streaming data enables a variety of use cases, including real-time, in-product analytics; automation; personalization; and alerting. Fortunately, recent innovations have made it easier for us to accommodate real-time streams.

In the last few years, streaming SQL technologies such as [ksqlDB][3], [Materialize][4], and [Apache Flink][5] have significantly progressed. These technologies enable us to process streaming data and run analysis with SQL—without needing to learn a new language or build specific language-unique integrations.

## ksqlDB support

We’re excited to announce that **it’s now possible to use Cube to build data modeling, caching, and access control layers on top of streaming SQL**, just as you can with data from cloud data warehouses.

![Cube schema with ksqlDB](https://ucarecdn.com/e96f17b3-82be-431b-ad37-bed225c49322/)

Cube now can connect to streaming SQL engines and expose streaming data via our REST, GraphQL, and SQL APIs to your downstream applications. Our REST API already supports websockets and provides a seamless experience for developers and data engineers to build real-time data apps. Our data modeling, caching, access control, and querying APIs are the same whether you work with streaming or batch data.

Want to see it in action? [Check out our live ksqlDB demo application][9] featuring a real-time dashboard built with Kafka, ksqlDB, and Cube.

That alone opens many opportunities for building on top of streaming data, but we’re also committed to addressing the complex problem of merging batch and streaming data, so that we can provide a single interface for unioned data.

## Lambda architecture support

We’re also introducing [lambda pre-aggregations](https://cube.dev/docs/caching/pre-aggregations/lambda-pre-aggregations). These follow the [lambda architecture][6] design to create a union of streaming and batch data.
Cube pre-aggregates batch data from the cloud data warehouse into one pre-aggregation, data from streaming SQL engines into another, then unites these pre-aggregations during query processing—so we can return merged data.

![Cube schema with lambda architecture](https://ucarecdn.com/466fe9e6-201e-483f-a142-5eefd1987576/)

For developers or data consumers, the interface remains the same—REST, GraphQL, or SQL queries. Our Lambda architecture makes it possible to consume data as a single dataset regardless of whether it is streaming, batch, or combined data.

Cube hides the complexity of this architecture within our data modeling layer, and therefore significantly simplifies the data consumption process. In addition, our access control and security layers are applied using the same rules as you’ve configured them for batched data.

## Coming next

Naturally, our work isn’t done yet. The above features are generally available soon—and available today to Cube users who request access. Soon, you’ll also have access to

- [Materialize](https://cube.dev/docs/config/databases/materialize) streaming pre-aggregations
- Flink SQL support
- Spark Streaming support

If you’re interested in early access to these features, to help influence their development and craft the future of streaming headless BI, simply [get in touch][7].

## Everything you need to build powerful real-time applications

We’re excited to launch our Kafka and ksqlDB integration and lambda pre-aggregations today because these bring the power and simplicity of our headless BI architecture to new types of data applications.

These innovations are available to everyone, for free: [request ksqlDB preview access][8], connect a real-time data source, then (please!) give us your feedback.

We look forward to powering the next generation of data applications—we can’t wait to see what you build.

[1]: https://cube.dev/docs/config/databases
[2]: https://cube.dev/events/building-real-time-streaming-data-applications-with-cube
[3]: https://ksqldb.io
[4]: https://materialize.com
[5]: https://flink.apache.org
[6]: https://en.wikipedia.org/wiki/Lambda_architecture
[7]: https://cube.dev/contact
[8]: https://cube.dev/contact
[9]: https://ksql-demo.netlify.app/

All Things Cube

Headless BI with streaming data

We started Cube as an [open-source analytics framework](https://web.archive.org/web/20190501161224/https://cube.dev/) in 2019. It was written in JavaScript, installed as an npm package, and had its data modeling language based off JavaScript—hence the “.js”.

Since then, we have written more lines of code in Rust than in JavaScript and moved from npm to [Docker](https://cube.dev/blog/cubejs-loves-docker) for distribution. As we're writing this, GitHub says that Cube’s codebase is 55% Rust and we’re confident that its share will only keep growing. We used Rust to build [Cube Store](https://cube.dev/blog/introducing-cubestore) and quickly realized that this is [the language in which we want to code](https://www.reddit.com/r/rust/comments/vi7pre/rust_tops_stackoverflow_survey_2022_as_the_most/). [Cube’s SQL API](https://cube.dev/blog/expanded-bi-support), which we recently released, was implemented entirely in Rust and we’re going to migrate the rest of Cube’s codebase to it over time.

We have also seen a lot of requests to support YAML and Python for data modeling, and we’re planning to expand to those languages later this year. (Please stay tuned for updates and [reach out to us](https://forms.gle/4CZRRKK8nD4PvtxZA) if you’d like to sign up for early access to YAML and Python support.)

Therefore, for all of those reasons and then some, we realized that having “.js” in our name is outdated and misleading. We also have found it can confuse those who think “.js” implies that Cube may be a JavaScript visualization library similar to [Chart.js](https://www.chartjs.org) or [D3.js](https://d3js.org); while Cube does work very well with many data applications and visualization tools—including said charting libraries—they are definitely not the same thing. 

## And so, Cube.js is now Cube. 

Our customers, community, and team have been referring to our product simply as “Cube” for quite a while already. Now, it’s time to make it official! 

Over time, we’ll update our website and docs to remove the “.js” from “Cube”. As a note, whether you are using self-hosted Cube or [Cube Cloud](https://cube.dev/cloud), no action on your part is required. 

While this change is purely cosmetic, it reflects Cube’s evolution from a open-source JavaScript-based analytics framework to the [headless BI](https://cube.dev/blog/headless-bi) platform it is today. Regardless of whether you’ve been a long-time community member from our open source analytics days, a new user of Cube Cloud, or just checking us out—thank you for being here. We are very proud of the path we have traveled to get here and are even more excited about what lies ahead. 

---

## Learn More

Want to learn more about building data apps with Cube?

Get in touch with us and the rest of the Cube community on [Slack](https://slack.cube.dev/) or [Github](https://github.com/cube-js/cube.js), or sign up for [a free Cube Cloud account](https://cube.dev/cloud).

Want to hear more about this change and what's next in store for Cube? Schedule a time to [chat with us 1:1](https://cube.dev/contact).


Officially dropping the ".js."

Cube.js is now Cube

TLDR; We decided to move on from Redis for queue and in-memory cache management and migrate it to the Cube Store. That means we’ll deprecate Redis for cache and queue management later this year and fully remove its support in new versions of Cube in 2023.

Cube’s mission is to use [headless BI](https://cube.dev/blog/headless-bi) to power your data applications with consistent, secure, and performant data. One of the ways we’ve been guaranteeing data performance has been with Cube’s queue and in-memory caching layers. Until now, these components were based on [Redis](https://redis.io/), an open source, in-memory data store.

However, as Cube has been scaling and seeing significant expansion in its use cases, we’ve realized using Redis doesn’t make sense anymore. Instead, we’ve come to need an internal, more specialized solution to avoid unnecessary dependencies and execute operations smoothly. And so, we’re beginning to sunset its support soon—and are excited to migrate queue management and in-memory cache to [Cube Store](https://cube.dev/docs/deployment/overview#cube-store). 

## How Cube’s in-memory cache and queue management work

Let’s start with some context. Cube has a two-level caching system: pre-aggregations and in-memory cache. There are also Cube’s queues, which sit in front of databases to provide a concurrency burst safety net in case there are too many simultaneous requests. Cube’s in-memory cache and queues allow you to scale stateless API instances horizontally and make APIs idempotent; they are currently powered by Redis. 

Cube’s queues act as a buffer for your database when there's a burst of requests hitting the same data from multiple concurrent users. Its job is to ensure that the data sources are not overloaded and that multi-stage queries are executed and refreshed in the correct order. Cube’s in-memory cache solves a pretty similar problem, offloading query workloads from databases that shouldn’t be refreshed yet by `refreshKey` rules. The in-memory cache also offloads cluster coordination for query execution costs as its overhead is noticeable in deployments with high concurrency.

Cube currently maintains in-memory cache, query execution queues for pre-aggregations, and data queries in Redis. The architecture built with Cube and Redis enables the queues to be idempotent—meaning that if multiple identical queries come in, only one will run against the data source. 

![Idempontent Multi-stage Query Execution - Copy of Data Flow Diagram (Logical) Example](https://images.ctfassets.net/6v81z8q9p5kl/5euHKYWUn73K7XZchIhhhg/ad1e6436aa851a2f750a16f5e3249394/Idempontent_Multi-stage_Query_Execution_-_Copy_of_Data_Flow_Diagram__Logical__Example.png)

## Why is Redis not a fit anymore?

Redis is a unique piece of software that scales very well for a wide variety of use cases. However, Cube continues to grow in terms of use case sophistication, performance requirements, and consistency guarantees.

Redis requires too many complex instructions for simple operations like atomically pushing to a queue, executing mutually exclusive queries, and idempotent result waiting on client nodes. It also yields an excess of optimistic concurrency clashes. As a result, Redis cluster coordination consumes a lot of resouces under the high load, which creates a bottleneck to scaling the API concurrency. 

Finally, even with a lot of community effort, we couldn’t provide stable support for clustered Redis. All of these issues made it clear to our users and us that the current queue management and in-memory caching systems need simplification.

## How will Cube Store replace Redis?

The Cube Store implementation will closely mimick Redis’ functionality. However, we’re going to replace lengthy command batches with single atomic domain-specific instructions. This change will drastically reduce clashes. Additionally, it’ll significantly simplify cache and queue access flow, which should drive a significant throughput boost. 

Cube Store will also employ similar distributed LRU caching semantics as Redis. Unlike Redis, however, it’ll store data in binary format utilizing columnar compression strategies in cases where it makes sense to do so. 

Most importantly to you—in-memory cache and queue management in Cube Store will use the same SQL protocol as pre-aggregation management, which is already occurring in Cube Store. So, you’ll be able to use Cube Store for in-memory cache and queue management without any reconfiguration or additional setup. 

## Timeline

We’re planning to release initial support for queue management in Cube Store and issue a deprecation notice for using Redis with Cube in November 2022. We’ll fully remove Redis support in Cube around Q2 2023.

## Goodbye, Redis—and hello, Cube Store

And so, we’re gradually saying goodbye to Redis support. 

It’s been a great run, but bigger and better performance is coming for Cube users: in-memory cache and queue management with Cube Store is on the horizon.

- Have questions about this change? Ping [me](https://www.linkedin.com/in/paveltiunov/) in [our Slack](https://slack.cube.dev/).

- Need help migration to Cube Store-based queue management? Contact our team via [our contact page](https://cube.dev/contact).

And, if we’ve piqued your curiosity but you don’t have a [free Cube Cloud account](https://cube.dev/pricing) yet, [sign up here](https://cubecloud.dev/auth/signup?plan=free). 


Pavel Tiunov

Goodbye, Redis—and hello, Cube Store.

Replacing Redis with Cube Store

Today, we’re announcing our official integration with [Databricks][1]. Here’s how it works—and what it means.

## What is Databricks?

Databricks is a unified, open, web-based platform for working with Spark. Its multicloud infrastructure provides automated cluster management and IPython-style notebooks to unify data warehousing, AI, and analytics in a single platform. 

The Databricks [Data Lakehouse][2] brings the best of both worlds: the reliability, governance, and performance of data warehouses with the openness, flexibility, and ML capabilities of data lakes. The Data Lakehouse efficiently handles all data types while applying a consistent security and governance approach. 

Unifying data in a single platform fulfills the needs of all your data teams—engineering, analytics, BI, data science, and ML—and beautifully simplifies your modern data stack. 

## What is Cube?

[Cube][3]’s headless business intelligence platform makes lakehouse data consistent, performant, and accessible to every downstream application. We do this with [data modeling that defines your metrics up-stack][4]. 

Cube’s advanced [pre-aggregation][5] capabilities powerfully condense frequently-used queries to optimize response times. Cube then provides this data to every application with GraphQL, REST, and SQL APIs so that every tool—and every data consumer—can work with the same data.

![cube-schema][30]

## The Cube and Databricks Integration

By combining Databricks’ unified analytics and data store with Cube’s data modeling, access control, caching, and APIs, it’s now possible to build a new dashboard or embed analytics features into customer-facing apps in minutes—not weeks. 

Select Cube and Databricks users already have been enjoying this integration in production for use cases like creating dashboards for internal analytics. Here’s one example: by consuming operational data with Kafka, transforming it into insights with Databricks, then using Cube as an API and caching layer to power visualization layers, users have been able to find actionable insights and identify opportunities to optimize their operations.

It’s easy to connect a workspace to Cube using Databricks’ SQL endpoint. Here’s how.

For starters, we recommend using Cube with the [Databricks SQL Endpoint][6]. Follow [Databricks’ docs to create a SQL Endpoint][7]. 

### Creating a new SQL endpoint

 - Log in to your Databricks Workspace account.
 - Navigate to the **SQL Console** on the left hand bar and select **SQL** from the drop-down menu.

![select-sql-tab-in-databricks][17]

 - Click __API Endpoints__, and then on the top right, select __+New SQL Endpoint.__

![click-API-endpoint][18]
![create-new-SQL-endpoint][19]

 - Next, name your endpoint and choose a __Cluster Size__. Make sure to set the __Auto Stop__ to at least 60-minutes of no activity, then click __Create__.

![name-new-endpoint][20]

After you’ve configured data access on the SQL Endpoint, it’s time to get your credentials for the SQL Endpoint. 

 - In the __Databricks SQL Console__, go to __Endpoints__ and click on the endpoint you created.

![go-to-endpoints-in-console][21]
![click-on-endpoint-created][22]

 - Navigate to __Connection Details__ and copy the HTTP Path, Port, and Server Hostname:

![connection-details][23]

Once you have your credentials, [generate a Personal Token][8]. 

 - Navigate to the __Admin Console__, click on __Settings__, and select __User Settings__.

![user-settings][24]

 - Toggle over __Personal Access Tokens__ and click on __+Generate New Token__.

![personal-access-token][25]

 - Name your token and set the expiration to __“Lifetime”__, then click __Generate__.

![name-your-token][26]

 - Copy the generated JDBC URL.
 - Attach your newly-generated personal token to the JDBC URL you copied in step 6, e.g. `jdbc:spark://dbc-xxxxxxxx-3957.cloud.databricks.com:443:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/endpoints/0b27a1dfc18f82ae;UID=token;PWD=PERSON-TOKEN`
 - Finally, select the database you’d like to query with Cube. In our example, we’ll use “default”.

![select-database][27]

### Connecting Databricks to Cube

 - Sign into [Cube Cloud][9] and use the wizard to create a new deployment. Select Databricks from the database connection list.

![sign-into-Cube-Cloud][28]

 - Enter your Databricks JDBC URL and personal token as well as your database name to connect.

![databricks-jdbc-url][29]

 - If needed, copy Cube Cloud’s IP addresses and add them to your Databricks allow-list. ([Databricks’ documentation for IP access lists is here][10].)

That’s it! Now you are ready to use Cube on top of your Databricks to model your data, manage security rules, apply caching, and expose your data definitions to any downstream application via Cube’s REST, GraphQL, or SQL APIs. 

You can check out [more examples of integrations with visualization tools on our website][11].

## Try the Cube and Databricks Integration for yourself

To follow the tutorial above and experience everything this integration offers, check out our [docs][12]—and, if you don’t already have one, sign up for [a free Cube Cloud account][13].

## Tell us what you think

We’d love to hear from you, learn what you’re building, and strategize how to help. Find us via Cube’s [Slack][14] and [GitHub][15] communities—or [schedule a time][16] to chat 1:1.

[1]: https://databricks.com/
[2]: https://databricks.com/product/data-lakehouse
[3]: https://cube.dev/
[4]: https://cube.dev/blog/headless-bi
[5]: https://cube.dev/docs/schema/reference/pre-aggregations
[6]: https://docs.databricks.com/sql/admin/sql-endpoints.html
[7]: https://docs.databricks.com/sql/admin/sql-endpoints.html#create-a-sql-endpoint
[8]: https://docs.databricks.com/administration-guide/access-control/tokens.html#enable-or-disable-token-based-authentication-for-the-workspace
[9]: https://cube.dev/cloud
[10]: https://docs.databricks.com/security/network/ip-access-list.html#add-an-ip-access-list
[11]: https://cube.dev/examples
[12]: https://cube.dev/docs/config/databases/databricks/jdbc#databricks-jdbc
[13]: https://cube.dev/cloud/
[14]: https://slack.cube.dev/
[15]: https://github.com/cube-js/cube.js
[16]: https://cube.dev/contact
[17]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/39cfa0b3-c6b9-44dd-80a2-41bae6dea1a3.png
[18]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/38cfe617-ae91-4a85-a610-12eaba6c9dfe.png
[19]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/02a29df7-26dd-41ba-a442-7fe46cf58246.png
[20]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f64ca517-e89e-437b-92e2-1c48752b8ca4.png
[21]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/5b5644d7-d5e1-4bff-87cd-abf40bbe459d.png
[22]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f8bd03ad-4edf-4452-a866-9d3c3140dc42.png
[23]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/6acf6cd1-0423-45f9-8bb2-884e3115e689.png
[24]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/8fd9c163-ebae-4b1e-b846-6178df628781.png
[25]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/7877ce4f-0e2d-49c1-97f9-56769fef46c7.png
[26]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/9dd552af-3dd4-40b4-9b9a-b3fd6c2bc8d2.png
[27]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/580253c6-1077-4291-a4a3-13303d383cda.png
[28]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/cd5ae87c-f9d0-432b-9a98-b378580d8d1d.png
[29]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/7f3f30a5-33a4-4073-8669-08bc7082c651.png
[30]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/e758755f-d633-45da-8ae3-92bc510cc9a1.png
[31]:https://databricks.com/dataaisummit/?itm_data=mainnav-promo-dais2022
[32]:https://cube.dev/events/data-ai-summit/

Tamar Barlev

Announcing the Cube and Databricks integration

Recently, we’ve been iterating on our blog to improve the UX for Cube’s diverse audience and account for the growing volume of the content.

Now, we’re happy that data engineers and application developers have a better, cleaner way to access the guides and tutorials on using Cube with other tools in the data space.

As a [headless BI](https://cube.dev/blog/headless-bi) platform, Cube works with numerous [data sources](https://cube.dev/docs/config/databases) and [data consumers](https://cube.dev/docs/config/downstream/). We've announced six integrations over the last few months ([Tableau](https://cube.dev/blog/tableau-tutorial), [Ahana](https://cube.dev/blog/ahana-integration-querying-multiple-data-sources-with-managed-presto-and-cube), [Supabase](https://cube.dev/blog/supabase-tutorial-fast-data-visualization-apps), [Materialize](https://cube.dev/blog/materialize-integration), [QuestDB](https://cube.dev/blog/time-series-data-analytics-with-questdb), and [dbt](https://cube.dev/blog/dbt-metrics-meet-cube)) that were all accompanied by hands-on technical guides.

![Cube is headless BI](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/855489ee-4cef-4146-876b-e44d0ebcc711.jpeg)

## Navigation Rehaul

Historically, application developers have a very strong presence in Cube’s audience. However, since the announcement of the [SQL API preview](https://cube.dev/blog/introducing-cube-sql) in late 2021 (matched with the recent [expanded BI support](https://cube.dev/blog/expanded-bi-support) announcement), Cube has received a lot of interest in the data engineering community.

So, now we match that with top-level categories in the blog:

![categories](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/969737a6-dcdf-4ae5-8720-674565609762.png)

There you can can find collections of posts covering categories of tools (like [Cloud Data Warehouses](https://cube.dev/blog/category/warehouses)) or individual tools in the data space (like [Amazon Athena](https://cube.dev/blog/category/athena)).

![categories](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/378a311a-fae8-4829-b274-436b9d7e33af.png)

You can find themed collections in other blogs as well (e.g., [Hasura](https://hasura.io/blog/) or [GitHub](https://github.blog/changelog/label/security/)), and we hope that this pattern will get even wider adoption in technical publications:

![themed collections](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/844b0b23-20fd-4077-aa25-378024683008.jpg)

## Focus on Content and Authors

Catering to Cube’s deeply technical audience, we routinely publish guides and tutorials up to 5-6 thousand words in size that take 15-30 rewarding minutes to read.

Now every blog post is accompanied by a table of contents that provides for the convenient navigation through the content and helps build expectations:

![table of contents](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/67b0f8d3-91ab-43e2-b090-1b495f04db4e.jpg)

We also recognize that the blog is collectively produced by the Cube team and community, and sometimes posts are even co-authored. So, now posts are accompanied by authors’ profiles with social links—feel free to reach out and follow!

![authors](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/8dab8d58-4536-4884-9ad3-9488da367211.jpg)

It’s always good to put the authors in the spotlight and we appreciate how [Evil Martians](https://evilmartians.com/chronicles) and [Smashing Magazine](https://www.smashingmagazine.com) do that:

![smashing magazine](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/3f493317-2380-42c9-8e17-ea6ad39e56de.jpg)

## Sneak Peek Under the Hood

We not only enjoy the end result but we also had fun developing this blog update. Here’s what the Growth Engineering team wants to share.

![Anton and Kseniia](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/5d2aed46-7d09-41df-9292-667c77438061.png)

We hope that rehauled navigation and categories make our blog more like a textbook on data engineering and app development, less like an assorted collection of geeky notes.

![Sasha](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/32d4ef9a-caa1-4d06-98d7-34b8d4689249.png)

We tried hard to make sure that users are getting value as soon as possible and there’s no disruption to content updates in the blog.

![Sofi](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/5e000bf2-410c-4c3d-b15e-df6971db9454.png)

With excitement, we await Sofi’s story in [Smashing Magazine](https://www.smashingmagazine.com) where she reveals the nitty-gritty details behind the blog post feed layout.

Hope you like these updates to the blog—please share your feedback on [Slack](https://slack.cube.dev) or [Twitter](https://twitter.com/thecubejs). 

Nastya Kataeva

Content organized by data tools, better navigation in hands-on technical guides, and authors in the spotlight.

Refreshed look and feel of the Cube blog

Since we [launched our open source offering][1] in 2019, Cube has helped thousands of companies consume data from any data source, organize it into consistent definitions, and use it with data applications. Developers and data engineers have used Cube to build a thriving ecosystem of embedded analytics features and custom data applications.

However, it’s always been our ambition to make Cube’s power more accessible to a greater diversity of data consumers. 

To get there, data needs to be consistent and performant not just for developers—but also for analysts and general business users. This requires bringing Cube into the tools where these users already work: namely, business intelligence tools such as Tableau, Superset, and Power BI.

Today, we’re proud to announce a major milestone in our efforts. Cube’s updated SQL API makes our headless analytic platform available to users of every BI application.


## Consistent data for every tool

We first launched a [preview of Cube’s SQL API][2] late last year. Since then, we’ve been hard at work responding to your thoughtful feedback and adding support for ways specific BI applications generate SQL queries. Now, it’s possible to connect any data visualization and BI application to Cube via the Postgres SQL interface.

Now, an organization can configure Cube—once—to securely access data sources, define data models, and cache queries. After that, every data consumer across an organization can securely access Cube’s definitions and materializations using vastly simplified queries.

No matter which tool a team uses, everyone in the organization will see the same data definitions. ‘Total Orders’ will mean the same thing in executives’ dashboards, marketers’ campaign reports, and the finance team’s revenue models.

By abstracting away complex queries, Cube reduces the difficulty of exploring data with collaborative data tools and notebooks.

By centrally managing access to up-stack data stores, Cube consistently defines, enforces, and audits who can access what; this eliminates the duplicated effort and inherent risks of defining access control within each downstream tool.

And by managing query-level caching and aggregate tables, Cube reduces the bandwidth costs and degraded performance of directly, repeatedly querying raw data stores.


## Here comes Hydra

We’ve already talked about [using Cube as a “headless” BI layer][3]. With our SQL API, now it’s easier than ever to connect all of your “heads.”

Everything you need to connect Cube to BI tools is available from Cube’s “Overview” window. Just click “How to connect your BI tool” to reveal connection details, then connect to Cube as a PostgreSQL data source from within your BI tool.

![fetching connection details from Cube][10]
![adding Cube to Tableau as a SQL data source][11]

Your [cubes][4] will be exposed as tables, where both your [measures][5] and [dimensions][6] are columns. 

![Cube schema and Tableau columns][12]

This makes it possible to query Cube as a data source from within your BI tool, ensuring consistent data for every data consumer.

![Cube data in Tableau][13]


![Cube data in Superset][14]

## Learn more

The new SQL API is available in both the open source Cube tool and Cube Cloud, the managed Cube service hosted within select regions of the Google Cloud Platform and Amazon Web Services. You may sign up for a free Cube Cloud account at [cube.dev/cloud][7].

Interested in getting started? [See our guide to working with BI tools][15].

Need help? Have feedback? We love hearing from you. [Join the Cube Dev Slack community][8] or [drop us a line][9].


[1]: https://github.com/cube-js/cube.js/releases/tag/v0.3.5-alpha.0
[2]: https://cube.dev/blog/introducing-cube-sql
[3]: https://cube.dev/blog/headless-bi
[4]: https://cube.dev/docs/schema/fundamentals/concepts#cubes
[5]: https://cube.dev/docs/schema/fundamentals/concepts#measures
[6]: https://cube.dev/docs/schema/fundamentals/concepts#dimensions
[7]: https://cube.dev/cloud
[8]: https://slack.cube.dev
[9]: https://cube.dev/contact
[10]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f2119e1a-f6d9-461b-88ca-723cc13df543.gif
[11]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/e806c18e-4448-4c8b-a79b-f66eccc6eaad.gif
[12]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/291c4d76-f267-465d-b4f7-990b05e5c4b7.gif
[13]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/4794eb01-db59-4fd4-a720-c2e218b00ce4.gif "Querying Cube in Tableau"
[14]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f955f2e7-692b-4afe-94c8-2d047f42131f.gif "Querying Cube in Superset"
[15]: https://cube.dev/docs/config/downstream

Connect your favorite visualization tools to Cube.

Announcing expanded BI support with our SQL API

[Materialize](https://materialize.com) is a different kind of database: on the surface it presents as Postgres, but internally it maintains the results of a query as a materialized view that stays up-to-date as the data changes. While a traditional database does the evaluation when a SELECT statement is issued, Materialize asks for queries upfront and computes the results on the fly as new data arrives.

As we’ll discuss more below, this makes Materialize great for scenarios where you need the complex join and transformation capabilities of a warehouse, but you want to serve the results to applications.

## What is Cube?

Cube is a headless business intelligence platform that makes data accessible and consistent across every application. Cube provides data modeling, access control, caching, and a variety of APIs (SQL, GraphQL, and REST) on top of any data warehouse, database, or query engine, including Materialize.

If you’re familiar with the concepts of “headless CMS” or “headless ecommerce”, you already have the right mental model for Cube. It’s essentially a “headless BI”—say, open-source Looker with less UI and more API—that delivers consistent data to your BI tools, data apps, and notebooks.

## The Integration

Integrating Cube and Materialize was a perfect example of the benefits of building in public and using well-known protocols.

Cube is open source and has an existing [PostgreSQL driver](https://cube.dev/docs/config/databases/postgres). Materialize [contributed](https://github.com/cube-js/cube.js/pull/4320) a lightly modified version of the PostgreSQL driver to help Cube read from Materialize.

![Materialize integration](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/d8ed003d-aa86-497b-9e51-219539351db4.png)

## Why are Materialize and Cube useful together?

Data warehouses were built with the end goal of showing people data. They work best for data exploration, analysis and reporting. But just looking at the data is no longer sufficient: now we’re connecting *systems* (APIs, applications, tools and automation) to the data in the warehouse.

When people point systems at data warehouses, they inevitably face performance problems:
* Not built for lookup queries
* Not built for low latency and high concurrency
* Reactivity is slow and time-to-insight is high

Also, there’s always a gap between how the data is stored and how the data is analyzed. Systems that consume the data (e.g., data apps or BI tools) and people working with them are most comfortable dealing with business-level metrics definitions (e.g., “revenue growth MoM by countries”) rather than calculations over the raw data in the warehouse.

With Materialize, you get warehouse-style views of your business data at application-style availability (always up-to-date, low latency queries). Combine that with Cube’s semantic layer and a variety of APIs to get a “live read API for your entire business”.

## Show me an example!

Here’s how you can make Materialize work with Cube for fun and profit. Let’s build an application that consumes a live stream of Hacker News stories but only shows the ones that ranked in top 3 so you can still have plenty of free time.

**Running Materialize and Cube.** Both tools have managed cloud-based offerings ([Materialize Cloud](https://materialize.com/materialize-cloud-access/) and [Cube Cloud](https://cube.dev/cloud/)) but, for the sake of simplicity, we can run them in Docker. The following `docker-compose.yml` file will run Materialize, seed it with data, and then run Cube:

```yaml
version: '2.2'
 
services:
 materialize:
   image: materialize/materialized:v0.26.1
   ports:
     - 6875:6875
 
 seed:
   image: jbergknoff/postgresql-client
   volumes:
     - .:/seed
   entrypoint: ["sh", "seed/seed.sh"]
   depends_on:
     - materialize
 
 cube:
   image: cubejs/cube:latest
   ports:
     - 4000:4000
   environment:
     - CUBEJS_DEV_MODE=true
     - CUBEJS_DB_TYPE=materialize
     - CUBEJS_DB_HOST=materialize
     - CUBEJS_DB_PORT=6875
     - CUBEJS_DB_NAME=materialize
     - CUBEJS_DB_USER=materialize
     - CUBEJS_API_SECRET=SECRET
   volumes:
     - .:/cube/conf
   depends_on:
     - seed
```

In the `seed.sh` file, we’ll use a vanilla PostgreSQL client to connect to the Materialize instance and run three SQL queries: the first one will connect a live [PubNub stream](https://www.pubnub.com/developers/realtime-data-streams/hacker-news-articles/) with Hacker News stories data, the other one will transform the stream into a normalized table view with columns, and the last one will create a materialized view of the stories:

```bash
#!/bin/bash
 
cat > seed.sql << EOL
 
CREATE SOURCE hn_raw
FROM PUBNUB
SUBSCRIBE KEY 'sub-c-c00db4fc-a1e7-11e6-8bfd-0619f8945a4f'
CHANNEL 'hacker-news';
 
CREATE VIEW hn AS
SELECT
  (item::jsonb)->>'link' AS link,
  (item::jsonb)->>'comments' AS comments,
  (item::jsonb)->>'title' AS title,
  ((item::jsonb)->>'rank')::int AS rank
FROM (
  SELECT jsonb_array_elements(text::jsonb) AS item
  FROM hn_raw
);
 
CREATE MATERIALIZED VIEW hn_top AS
SELECT link, comments, title, MIN(rank) AS rank
FROM hn
GROUP BY 1, 2, 3;
 
EOL
 
psql -U materialize -h materialize -p 6875 materialize -f ./seed.sql
```

The only missing piece is Cube’s data model file, placed under `schema/HN.js`. Cube will read from the live-updated materialized view and expose metrics via its APIs. 

```js
cube(`HN`, {
  sql: `SELECT * FROM public.hn_top`,
 
  refreshKey: {
    every: '1 second'
  },

  measures: {
    count: {
      type: `count`
    },
 
    countTop3: {
      type: `count`,
      filters: [ {
        sql: `${rank} <= 3`
      } ]
    },
 
    bestRank: {
      sql: `rank`,
      type: `min`
    }
  },
  
  dimensions: {
    link: {
      sql: `link`,
      type: `string`
    },
 
    comments: {
      sql: `comments`,
      type: `string`
    },
 
    title: {
      sql: `title`,
      type: `string`
    },
 
    rank: {
      sql: `rank`,
      type: `number`
    }
  },
 
  segments: {
    show: {
      sql: `${title} LIKE 'Show HN:%'`
    }
  }
});
```

You copy and paste these files or [pick them from GitHub](https://github.com/cube-js/cube.js/tree/master/examples/materialize). Now, we can run the whole stack with `docker compose up`.

**Exploring the data.** Now, if you navigate to Cube’s Playground at [localhost:4000](http://localhost:4000), you’ll instantly be able to run queries against the Materialize instance with connected stream by picking measures, dimensions, segments, and filters in the UI.

Here’s how you can check how many stories we have had in the data stream (30) and how many of them have even been ranked in top 3 (no surprise, just three):

![First query](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/14277bc2-2177-4350-9d2a-8abda583df6c.png)

However, if you wait a few minutes and refresh the query results, you’ll see that data changes because new stories are posted to HN and existing stories change their ranks.

Then, here’s how you can combine segments and filters to pick only “Show HN” stories ranked in top 3:

![Second query](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/615c7251-480b-4bdb-98b4-ed5e821d965f.png)

**Connecting to APIs.** Cube exposes SQL, GraphQL, and REST APIs, each with its own query syntax. For instance, the query directly above is expressed as the following JSON query accepted by the REST API:

```json
{
  "dimensions": [
    "HN.title",
    "HN.rank"
  ],
  "order": {
    "HN.rank": "asc"
  },
  "filters": [ {
    "member": "HN.rank",
    "operator": "lte",
    "values": [ "10" ]
  } ],
  "segments": [
    "HN.show"
  ]
}
```
Let’s run this query against the REST API and extract the result set with the amazing `jq` utility:

```bash
curl localhost:4000/cubejs-api/v1/load -G -s \
  --data-urlencode "query=$(cat query.json)" | jq '.data'
```
![REST API usage](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/a3bd5786-df83-406b-bded-f47e5a099641.png)

Also, if you are a GraphQL aficionado, you can use GraphQL API and an alternative syntax:

![GraphQL API usage](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/63a0e4fc-1f67-4e02-8ebc-71479dc8d923.png)

And, there’s one more thing—Cube’s [real-time data fetch](https://cube.dev/docs/real-time-data-fetch) based on WebSockets is a great match to the real-time experience of Materialize. You can use that to live-feed your data from the streams through Materialize and Cube all the way down to your end-user front-end application. 

**Future improvements.**  The current implementation of Cube’s Materialize driver makes Cube periodiucally poll Materialize for updates. A more effective and native for Materialize way to get data out of Materialize is subscribing to changes using the [TAIL feature](https://materialize.com/docs/sql/tail/). This would require more in-depth changes which we are exploring.

## Try Materialize and Cube for yourself

Feel free to check out Materialize and Cube by running the example above on your machine. The full source code is [available on GitHub](https://github.com/cube-js/cube.js/tree/master/examples/materialize). Also, please don’t hesitate to join Cube’s [Slack community](https://slack.cube.dev) or Materialize’s [Slack community](https://materialize.com/s/chat) to share your feedback.

To give both tools a try in a professional setting, check out [Materialize Cloud](https://materialize.com/materialize-cloud-access/) and [Cube Cloud](https://cube.dev/cloud/). The future is real-time, let’s build it together with Materialize and Cube.



Andy Hattemer

Materialize

With Materialize streaming database and Cube, building real-time applications has never been easier.

Announcing the Cube integration with Materialize: Building real-time applications over data streams

[Murphy’s Law][1] dictates that, even in the best-designed data stack, something will eventually go wrong. Queries time out, builds fail, and APIs sometimes go down. Historically, the main challenge to these errors has been _knowing when they happen—and why_—so that you can diagnose, take corrective action, and get back to work.

Fortunately, Cube Cloud has you covered.

Among Cube Cloud’s valuable features is that you don’t need to read logs to diagnose errors. Instead, Cube Cloud has always included UI tools for managing pre-aggregations and tracing queries. 

Now, to further enhance life in the data pipeline, we’re adding another way to  make diagnostics easier: configurable email alerts.

With timely notifications when something requires your attention, you’ll be able to triage issues faster than you can say “Kubernetes pod” three times in a row—and before they impact your users.

## Four Configurable Alerts

[Cube Cloud Alerts][2] take the guesswork out of understanding what’s gone wrong with a deployment. Once they’re set up, the next time something errors out, you’ll receive one of these messages:

### “API Failure on Cube Cloud”

API outages happen when a deployment’s API becomes unavailable and an API cannot serve requests. This can happen for a number of reasons, including:

 - A bug in the schema code or incorrect deployment
 - A database authentication failure
 - Internal issues with the Cube Cloud platform

If an API failure occurs, you’ll receive an email alert that includes the HTTP error code of what went wrong: 

![][3]

You’ll then be able to log into Cube Cloud and check for any changes that might have caused the error. And if the cause isn’t super apparent—reach out to us via Intercom for some troubleshooting help.

### “Response Time-Outs on Cube Cloud”

API/Database response timeouts are triggered if your APIs or database take longer than expected to respond. It’s helpful to receive a notification when this occurs to alert you to what would otherwise be a silent failure caused, for example, by:

 - Large requests that are taking too long to retrieve from the database
 - Pre-aggregation build configuration errors

If your APIs or database time out, you’ll receive an alert that specifies the failed SQL query. From there, you’ll be able to log into Cube and trace what went wrong and where. 

### “Pre-aggregation Build Failure on Cube Cloud”

During a pre-aggregation build failure, data doesn’t load into the Cube Store warmup cache. Unfortunately, like API/database timeouts, these errors can also be silent—making their root causes hard to pinpoint.

A pre-aggregation build failure can be caused by:

 - Incorrectly configured pre-aggregations that take too long to build or load 
 - Configurations that limit simultaneous open sessions and requests to a database
 - Overloaded refresh workers or refresh workers that aren’t configured to run in parallel

If a pre-aggregation fails to build, you’ll get an alert with a link to it in Cube. Follow the link to the failed pre-aggregation to see the issue’s root cause and fix it.

![][4]

_ **Pro Tip.** To avoid pre-aggregation build failures, you can configure partitions, export buckets, data source constraints, and refresh workers to keep your cache running smoothly._

### “Error Resolution”

Lastly, we’ve added an alert to notify you of error resolutions for all three types of the errors we’ve described here. You’ll get an email that lets you know the problem is fixed so you can continue on your merry way. 

## Available Now 

Alerts are available to all paid Cube Cloud accounts, and turning them is actually _quite_ simple: just follow the docs [here][5].

_Interested in harnessing the power of Cube headless business intelligence in a fully managed deployment? [Sign up for Cube Cloud today][6]._

[1]: https://en.wikipedia.org/wiki/Murphy%27s_law
[2]: https://cube.dev/docs/cloud/dev-tools/alerts
[3]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f349c14a-9a9a-41db-8070-a16859530276.png
[4]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/6634b895-1373-46c4-9cf7-6c81d0f9166d.png
[5]: https://cube.dev/docs/cloud/dev-tools/alerts
[6]: https://cube.dev/cloud

New in Cube Cloud: Alerts

dbt quickly became the major part of the modern data stack and the favorite tool of data practitioners. As Cube community grows, especially among data engineers, we more often see dbt being used upstack from Cube to define transformations and data model.

After a community-driven [feature proposal](https://github.com/dbt-labs/dbt-core/issues/4071) for dbt Metrics and an insightful discussion that followed, our community [requested an integration](https://github.com/cube-js/cube.js/issues/3611) with dbt that would allow keeping metrics definitions in dbt as the single source of truth while leveraging the rest of Cube.

## dbt Metrics meet Cube, the headless BI platform

At Cube, we are building headless business intelligence, and want to make data accessible and consistent across every application, ranging from embedded analytics to internal dashboarding and reporting tools. It’s an amazing opportunity for us to integrate with dbt Metrics, enhance it with Cube’s caching and access control, and deliver to every application via our SQL, REST, and GraphQL APIs.

![dbt.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/d445d8bb-7879-48ca-9ed8-172347d85fd7.png)

Today, we are happy to announce an integration with dbt Metrics. Cube can now read metrics from dbt, merge them into Cube’s data model, provide caching and access control, and expose metrics via our APIs to downstream applications.

In the tutorial below we’ll use dbt to define metrics, Cube to apply caching (and get a ~8x performance boost), provide access control, and expose data to Apache Superset via Cube’s SQL API and to other tools and applications via our REST API or GraphQL API.

## Building a metrics layer with dbt Metrics

Everything starts with a great dataset, so I’ve taken [GitHub Activity Data](https://console.cloud.google.com/marketplace/product/github/github-repos), a snapshot of more than 2.8 million (~10 %) open-source repositories on GitHub, publicly available on Google Cloud Platform. As of March 2022, it contains information about 257 million unique commits, their authors, messages, files, etc.

Let’s say we’re interested in learning all about productivity of individuals and teams expressed in the number of commits and commit message lengths: 

![Screenshot_2022-03-08_at_20.58.18.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/c1300d9a-09d1-4f2b-9af8-4223481a1b4a.png)

We would start with defining a dbt model like this in `commits.sql`:

```sql
SELECT
    TIMESTAMP_SECONDS(`author`.`time_sec`) AS timestamp,
    `author`.`email` AS author_id,
    `author`.`name` AS author_name,
    REGEXP_EXTRACT(`author`.`email`, r'@(.+\..+)$') AS author_domain,
    LENGTH(subject) AS subject_length,
    LENGTH(message) AS message_length
FROM `bigquery-public-data.github_repos.commits`
```

The next step would be to add the dbt metrics definitions into `schema.yml`:

```yaml
version: 2

metrics:
  - name: commits_count
    label: Commits Count
    model: ref('commits')
    description: "A count metric for commits"

    type: count

    timestamp: timestamp
    time_grains: [day, week, month, year]

    dimensions:
      - author_domain
      - author_name

  - name: users_count
    label: Users Count
    model: ref('commits')
    description: "A count metric for users"

    type: count_distinct
		sql: author_id

    timestamp: timestamp
    time_grains: [day, week, month, year]

    dimensions:
      - author_domain
```

Defined metrics will be available for programmatic introspection via [Metadata API](https://docs.getdbt.com/docs/dbt-cloud/dbt-cloud-api/metadata/metadata-overview) and manual exploration in [GraphiQL](https://metadata.cloud.getdbt.com/graphiql) IDE:

![Screenshot_2022-03-08_at_21.41.15.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/4a98a8a5-e015-4c12-b7f9-cae97d945e92.png)

## Exposing dbt Metrics via an API

As a headless BI platform, Cube consists of four logical layers: metrics, acceleration, access control, and API. Our metrics layer is able to read metrics definitions from dbt via the Metadata API and translate them into equivalent Cube data model.

You can get started with Cube within minutes in [Cube Cloud](https://cubecloud.dev/auth/signup) or set up Cube locally with [Docker](https://cube.dev/docs/getting-started/docker/compose). You’ll need to provide the credentials for your data warehouse; there are plenty of [data sources](https://cube.dev/docs/config/databases) supported by Cube.

Next, connecting Cube to dbt is as trivial as adding this code snippet to your data model:

```jsx
import dbt from '@cubejs-backend/dbt-schema-extension'
import { dbtJobId, dbtApiKey } from '../config'

asyncModule(async () => {
  await dbt.loadMetricCubesFromDbtCloud(dbtJobId, dbtApiKey)
})
```

In an instant, you’ll be able to explore the data in Cube Playground:

![Screenshot_2022-03-09_at_14.51.42.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/beeed292-7886-454f-af75-8dab41b3cfd1.png)

Cube will take care of SQL generation and querying the data in your data warehouse that was previously transformed with dbt:

```sql
SELECT
  `github_commit_stats_commits`.author_domain `author_domain`,
  `github_commit_stats_commits`.author_name `author_name`,
  COUNT(*) `commits_count`
FROM
  `cube-devrel-team`.`dbt_prod`.`commits` AS `github_commit_stats_commits`
WHERE
  `github_commit_stats_commits`.author_domain IN (?, ?)
GROUP BY 1, 2
ORDER BY 3 DESC
LIMIT 10000
```

This gives you the luxury of using a plenty of API flavors provided by Cube:

- [SQL API](https://cube.dev/docs/backend/sql) — to expose the metrics to popular BI tools, e.g., Apache Superset (check [this tutorial](https://cube.dev/blog/building-metrics-dashboard-with-superset/) to see it in action)
- [REST API](https://cube.dev/docs/rest-api) — to expose the metrics to even more BI tools (e.g., [Appsmith](https://cube.dev/blog/building-an-appsmith-dashboard-with-cube/), [Bubble](https://cube.dev/blog/building-a-bubble-dashboard-with-cube/), or [Retool](https://cube.dev/blog/building-an-internal-dashboard-with-retool-and-cube/)) or integrate Cube into your report-generation or machine learning pipeline
- [GraphQL API](https://cube.dev/docs/backend/graphql) and integrations with popular front-end frameworks such as React, Angular, and Vue — for embedded analytics in custom-built applications (check one of [many tutorials](https://cube.dev/blog/category/tutorials/) that we have)

Fetching data via the REST API is as simple as sending a JSON-encoded query and parsing the JSON-encoded result set. Cube provides convenient JavaScript client libraries for that purpose, but even a `curl` or `wget` call will work just great:

![Screenshot_2022-03-11_at_16.57.38.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f336e7f9-c9ac-49e3-8c8d-1bb500fc0727.png)

Bringing the same data into Apache Superset or another data visualization tool via Cube’s SQL API requires just a few clicks in the UI that will yield an equivalent SQL query and result set:

![Screenshot_2022-03-11_at_17.02.12.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/bd3e1389-1793-4ec7-9f74-54cef22c82e1.png)

Take a closer look! Superset generates a fairly straightforward SQL query (shown on the screenshot above) and sends it to Cube. Actually, you can write your custom SQL queries, like the one below, where you can reference measures and dimensions from your data model. Curious what `MEASURE(commitsCount)` is? Well, we see a lot of value in valid SQL, that why you can list your dimensions in the `GROUP BY` statement and enclose your measures using the `MEASURE` aggregate function (if your BI tool of choice would generate something like `COUNT` instead of `MEASURE`, Cube will readily accept that, too):


```sql
SELECT authorDomain, MEASURE(commitsCount)
FROM db.`GithubCommitStatsCommits`
WHERE authorDomain IS NOT NULL
GROUP BY authorDomain
```


Now we’ve learned how dbt Metrics can be exposed to other tools via various APIs provided by Cube. Next, let’s explore how dbt users can leverage Cube’s acceleration and access control layers. 

## Accelerating dbt Metrics with pre-aggregations

Cube’s acceleration layer serves as a smart cache on top of the metrics layer.

Based on the configuration provided, Cube will pre-emptively fetch data from the data warehouse, pre-aggregate it into analytical rollups, and store them in Cube Store, a custom-built distributed data store, in a columnar format. In most cases, Cube Store enables Cube to fulfill API requests within 300 ms, and allows for concurrencies up to 100 QPS.

You can see on the Apache Superset screenshot above that an unaccelerated request took more than 2 seconds. Of course, if we run the same request once again, we’ll get the result mush faster because it will be cached by Superset. Let’s see what can be done so any request would yield a sub-second response at all times.

The Cube way to achieve this is to use pre-aggregations. We’ll extend the data model with a new *cube* (think, “a namespace for metrics”) containing the same metrics and a single pre-aggregation:

```jsx
cube('GithubCommitStatsCommitsCached', {
  extends: GithubCommitStatsCommits,
  
  preAggregations: {
    main: {
      measures: [ commitsCount ],
      dimensions: [ authorDomain, authorName ],
      timeDimension: timestamp,
      granularity: 'day',
      partitionGranularity: 'year'
    }
  },
})
```

This caching configuration instructs Cube to use the `main` pre-aggregation for all queries that *match* it, i.e., contain a subset of listed measures and dimensions. It also says that `day` is the minimum time grain we’re interested in using. On top of that, it advises Cube to split the pre-aggregated data into partitions (one per `year`) and distribute these partitions across all Cube Store nodes so they all can be used in parallel to fulfil queries.

That’s basically it. Cube will seamlessly build the pre-aggregation, store it in Cube Store, and transparently dispatch queries that match it against Cube Store rather that the original data source, e.g., BigQuery. Here’s a side by side comparison of the query hitting BigQuery (1.8 s) ⚠️ and the same query fulfilled by a pre-aggregation in Cube Store (215 ms) ⚡

![Group_40193.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/4f264d1c-6cae-4381-806c-9100a2dc2b1d.png)

Also note that, depending on your usage patterns, not hitting your cloud data warehouse for every client-side query might not only improve the performance of your data application but allow for a substantial total cost reduction.

That’s how Cube’s acceleration layers works. What about access contol? 

## Securing dbt Metrics and implementing multitenancy

Cube’s access control layer lets you manage who is able to access data and which data it is. Subsequently, Cube has built-in mechanisms for row-level security and [multitenancy](https://cube.dev/docs/multitenancy-setup).

You can mandate that every query need to be accompanied by a *security context* (think “securely signed set of credentials and meta data”) that identifies a user or an application running a query. Cube allows you to validate the security context and the query, possibly amending the contents of the query to enforce access control rules.

We’ll use the `queryRewrite` extension point to require certain meta data to be provided with every query and used in a mandatory filter. With this configuration, no query can be run “unfiltered”:

```jsx
module.exports = {
  queryRewrite: (query, { securityContext }) => {
    if (!securityContext.domain) {
      throw new Error("Please specify the domain");
    }

    query.filters.push({
      member: 'GithubCommitStatsCommitsCached.authorDomain',
      operator: 'equals',
      values: [ securityContext.domain ],
    });

    return query;
  }
};
```

If anyone tries, they’ll get an error message. However, with a security context provided (see its JSON representation below), the query will yield the properly filtered results according to access control rules:

![Group_40193 1.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/4b32c605-dbc7-4f39-9073-d68940c676ba.png)

And that’s the shortest demo of how Cube’s access control layers works.

## Next steps for dbt users

Please feel free to check out the [full source code](https://github.com/cube-js/cube.js/tree/master/examples/github-commits-stats) of dbt models and the Cube project shown above on GitHub. Also, don’t hesitate to give Cube a star on [GitHub](https://github.com/cube-js/cube.js) as well as to follow us on [Twitter](https://twitter.com/thecubejs) or [LinkedIn](https://www.linkedin.com/company/cube-dev/mycompany/?viewAsMember=true) 🤘

If you’re a dbt user and you haven’t tried [dbt Metrics](https://docs.getdbt.com/docs/building-a-dbt-project/metrics), please do. You’ll be able to abstract your metrics definitions from the presentation layer and keep them in the single source of truth. Note that, as of March 2022, dbt Metrics are still an experimental feature, so if things doesn’t work as expected, please provide your feedback and find the time to [file issues](https://github.com/dbt-labs/dbt-core/issues) in dbt’s GitHub repository. That is much appreciated.

Also, if you see the value in integrations with BI tools and custom-built applications as well as query acceleration and access control that are built into Cube, please follow the steps outlined in this tutorial and share your success or ask questions in Cube’s [Slack community](https://slack.cube.dev).

Data Engineering

Combining dbt Metrics with API, Caching, and Access Control


Today, we’re happy to announce that Cube Dev is now SOC 2 Type I compliant. To understand how this affects all Cube users and your data, read on.

## What is SOC 2 compliance?
The [American Institute of CPAs (AICPA)][1] manages a Service Organization Control reporting platform that helps businesses document the technical and policy controls they’ve put in place to keep customer data secure.

By complying with SOC 2, a business signifies that it has implemented proper systems to ensure the security, availability, processing integrity, confidentiality, and privacy of the customer data that it processes. This enables customers and users to make informed decisions when entrusting external vendors—like Cube—with building and operating a piece of their data stack.

There are two types of SOC 2 audits, Type I and Type II. Type I describes a business’s systems and processes at a point in time. Subsequently, Type II describes how these systems and processes performed over a historical period, typically at least six months. Today, Cube Dev, Inc. has attained Type I certification, and we look forward to complementing this with Type II certification with the passage of time.

## What does SOC 2 certification entail?

There are five Trust Service Criteria—basically, principles that cover an area of safety and control.

 1. **Security.** A company should be able to show that its systems are protected from unauthorized access.
 2. **Availability.** The company’s systems should have controls in place to ensure that the systems are available when their end users need it.
 3. **Processing integrity.** The company’s systems deliver the right data at the right time—and there are monitoring and quality assurance procedures in place to maintain this integrity.
 4. **Confidentiality.** Customer data is restricted to specific people or organizations, using technical controls including network and application firewalls and role-based access controls.
 5. **Privacy.** When customer data includes personal information, that personal information is collected, used, retained, disclosed, and disposed of in compliance with the business’s privacy policies.

## Why does SOC 2 compliance matter?

Cube’s mission is making the world’s data consistent and accessible, supporting powerful and fast data applications. We achieve this mission by consuming data from modern data stores, organizing it into consistent definitions, and delivering the data to every tool—and, obviously, this requires your trust.

To build a modern data stack, every company has to make informed decisions to trust external vendors. This begins with the hosting providers that store raw data in the cloud data warehouse, and continues across the stack with the tools that transform, process, and display data in all its forms.

Without basic trust in these components, each company would have to build and maintain an entire data stack from the ground up—which is not core to most companies’ interest or core capabilities. It’s an understatement that this would significantly slow the pace of innovation.

SOC 2 is one of the most reputable audit reports available. We believe it helps our customers understand Cube’s commitment to data security and privacy, so that you can build data applications with confidence.


## What’s next?

With the passage of time, we look forward to complementing our SOC 2 Type I certification with an SOC 2 Type II certification. This will pose additional challenges to our team—not only must we continue to follow our documented procedures, but we will need to do so in a way that is auditable—but these are challenges we look forward to meeting.

We’ll also continue to invest in security training and oversight, hardening our systems, and growing a team and a culture that treats our customers’ data safety as our top priority. Put more succinctly, today’s announcement is not the finish line, but a milestone in our journey.

Along the way, if you have any questions about our controls or policies, or need advice about navigating the processes to bring your own business into SOC 2 compliance, please accept our invitation to get in touch.

We know firms in every industry that rely on SOC 2 certification to evaluate the vendors in their data stack. We’re proud to have earned your trust and excited to meet your requirements.





[1]: https://www.aicpa.org/home

Jonathan E Cowperthwait

Cube Dev is SOC 2 compliant—here’s what that means

At Cube, we’ve been excited to watch GraphQL’s growth in popularity and the rise of a rich and mature ecosystem of GraphQL front-end integrations and back-end tools. GraphQL has quickly become the most popular query language for APIs because it provides a complete description of the data in your API—giving clients the power to ask for exactly the data they need and nothing more.

However, I believe that while the GraphQL ecosystem is rich in tools focused on creating transactional (a.k.a. CRUD) GraphQL servers, analytical workloads are still underserved. This is particularly true for workloads designed on top of data warehouses.

For more than a year, a Cube GraphQL API has been the top requested feature from the Cube community. There’s clearly tremendous value for GraphQL users in having a uniform way to access both transactional and analytical APIs, in order to keep the data layer unified and organized. 

**Today, we are excited to announce GraphQL API support in Cube.** With this release, you can use Cube as part of your GraphQL layer to deliver metrics to your application.

We owe tremendous thanks to [Luc Vauvillier](https://twitter.com/lvauvillier), the COO and a co-founder at [Gadsme](https://cube.dev/blog/gadsme-runs-on-cube/), who contributed initial support for this GraphQL API and layed out the principal design:

> “With GraphQL support in Cube we can have a unified typed API for our frontend clients and be able to run code generators”. We built a GraphQL proxy on top of the Cube REST API internally first. Then we started to share our API design with the community, and when we realized that the additional work to make it generic would not be too substantial, we decided to contribute to have it officially supported. At [Gadsme](https://www.gadsme.com/), contributing to open source projects is part of our values and we really enjoyed working on this.”

With this API and [our recently-released SQL API](https://cube.dev/blog/introducing-cube-sql/), Cube can provide analytics data to a wide variety of tools and clients building towards our vision of a universal, headless analytics layer to provide metrics to any application. 

![image2.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/c957b4c8-5ea0-4d61-a610-21749d799953.png)

### Getting Started with the GraphQL API

Cube uses a JSON-based metrics framework to describe data calculations that can be exposed via GraphQL. Once defined, Cube will use these definitions to translate an incoming GraphQL query into a native SQL query to the underlying data storage layer.

Let’s consider the following example of Cube’s metrics framework that can be used to model orders in a hypothetical ecommerce data store:

```js
cube(`Orders`, {
  sql: `SELECT * FROM public.orders`,

  measures: {
    count: {
      type: `count`,
    },
  },

  dimensions: {
    status: {
      sql: `status`,
      type: `string`,
    },

    createdAt: {
      sql: `created_at`,
      type: `time`,
    },
  },
});
```

The main object in Cube’s merics framework is **cube**. Cubes always have the **sql** property. Usually, this points to the SQL table to which this configuration applies, but in more sophisticated use cases, a custom SQL query can be used as a property.

**Measures** are quantitative data—such as the number of units sold, unique visits, profit, and so forth. **Dimensions** are categorical data, such as state, gender, product name, or units of time. (You can learn more about Cube’s data schema [in our docs](https://cube.dev/docs/schema/fundamentals/concepts).)

Once the above definitions are created, we can start querying Cube via the GraphQL API. To get **“Number of orders grouped by status,”** we can use the following query:

```graphql
{
  cube {
    orders {
      count
      status
    }
  }
}
```

Cube will translate the above GraphQL query into a native SQL for your database and execute it. For example, if your database is Postgres, it would be the following SQL query: 


```sql
SELECT
  "orders".status "orders__status",
  count("orders".id) "orders__count"
FROM
  public.orders AS "orders"
GROUP BY
  1
ORDER BY
  2 DESC
LIMIT
  10000
```

Cube users probably are aware that Cube comes with [Developer Playground](https://cube.dev/docs/dev-tools/dev-playground), a useful tool to build and test metrics layer. Now, the Playground includes a built-in GraphiQL to test GraphQL API calls.

![image3.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/ab96e549-46c4-42d5-889b-f4edfc530256.png)

To get started with the GraphQL API from scratch, you can follow this [step-by-step guide](https://cube.dev/blog/graphql-postgres-metrics-dashboard-with-cube/). 

We’re excited about finally bringing GraphQL support to Cube, and we hope you are, too!  We’d love to hear your feature requests and use cases; please provide your feedback in our [Slack](http://slack.cube.dev/).

Want to learn more? [Give Cube a free try](https://cubecloud.dev/auth/signup).

Introducing the Cube GraphQL API: Building GraphQL Endpoints for Metrics

Today, Cube powers analytics features inside thousands of applications where developers have leveraged Cube’s data schema as a **metrics layer**—a consistent, single source of truth. By powering a single repository of metrics, Cube helps developers quickly and reliably ship embedded analytics features and other data-powered applications with the peace of mind that their metrics definitions remain consistent.

Cube solves this problem very well for application developers, but it’s been unsolved for users of dashboards and business intelligence tools, and the data engineering teams that support them.

In most modern organizations, there are multiple tools that consume data from the same single warehouse but perform their own downstream metrics calculations. This leads to inconsistent calculations and disagreements between teams: **How do we calculate revenue? Does it include revenue from X cohort? How do monthly payments affect annual projections?**

The more tools an organization uses, the more inconsistencies and conflicts arise, and the harder it is for a business to make decisions using accurate data. To avoid this, they need a centralized, upstream location in which to build and maintain their metrics, so that every tool works from the same source of metric truth.

Today, I’m excited to share a way for Cube to function as a metrics store for any data consumer: we’re proud to announce the new [Cube SQL API](https://cube.dev/docs/backend/sql/). With the addition of this API, Cube now functions as a headless BI layer to provide consistent metrics to any querying and visualization tool.

![](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/ebff03e0-fbcd-4941-9a3c-cc71b6142561.png)

## How it works

In the modern data stack, Cube acts as a proxy for data warehouses and translates every incoming query, whether it is JSON, GraphQL, or SQL, into native queries to the underlying data storage.

![](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/7bf46e5f-f4b8-4f2a-a236-c067ff894d86.png)

Cube translates queries by using a JSON-based data modeling layer consisting of cubes. Think of these as database views, backed by either a reference to an existing database table or a new table created by SELECT statement. Cubes contain defined measures and dimensions.

**Measures** are quantitative data, such as the number of units sold, unique visits, profit, and so forth. **Dimensions** are categorical data, such as state, gender, product name, or units of time. (You can learn more about Cube’s [data schema](https://cube.dev/docs/schema/getting-started) in the docs.)

Below is an example of the cube we can use to describe metrics about leads for a sales organization.

```javascript
cube(`Leads`, {
  sql: `
    SELECT
      persons.id,
      persons.created_date,
      deals.id as deal_id
    FROM persons
      LEFT JOIN deals ON deals.person_id = persons.id
  `,

  title: `Leads`,
  description: `Leads for Insides Sales`,

  measures: {
    count: {
      type: `count`
    },
    converted_to_deal_count: {
      type: `count`,
      filters: [
        { sql: `(${is_converted_to_deal}) = true` }
      ]
    },
    to_deal_conversion_rate: {
      type: `number`,
      sql: `ROUND(${converted_to_deal_count} / NULLIF(${count}, 0) * 100, 2)`,
      format: `percent`
    }
  },

  dimensions: {
    time: {
      sql: `created`,
      type: `time`
    },
    is_converted_to_deal: {
      type: `boolean`,
      sql: `deal_id IS NOT NULL`
    }
  }
})
```

With the above definitions of our metrics we can ask questions like **"What is our monthly lead-to-deal conversion rate over the last year?"**

When we query Cube’s metrics layer via the SQL API, cubes will be presented as tables and measures and dimensions as columns. To answer the above question with SQL we need to write the following query:

```sql
SELECT DATE_TRUNC('month', time),
       to_deal_conversion_rate
FROM leads
WHERE time >= '2021-01-01'
  AND time < '2022-01-01'
GROUP BY 1
```

Cube will translate this query into the query for the underlying database.

```sql
SELECT
  DATETIME_TRUNC(
    DATETIME(created_date, 'UTC'),
    MONTH
  ) `leads__time_month`,
  ROUND(
    count(
      CASE WHEN ((deal_id IS NOT NULL) = true) THEN 1 END
    ) / NULLIF(count(*), 0) * 100,
    2
  ) `leads__to_deal_conversion_rate`
FROM
  (
    SELECT
      persons.id,
      persons.created_date,
      deals.id as deal_id
     FROM persons
      LEFT JOIN deals ON deals.person_id = persons.id
  ) AS `leads`
WHERE
  (
    created_date >= TIMESTAMP('2021-01-01T00:00:00.000Z')
    AND created_date <= TIMESTAMP('2021-12-31T23:59:99.999Z')
  )
GROUP BY
  1
ORDER BY
  2 DESC
LIMIT
  10000
```

You can query Cube with SQL from your favorite programming language, such as Python.

![](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/2b0d23c8-37fa-4550-8c99-53196c832a26.gif)

Most powerfully, because the Cube SQL API speaks MySQL-compatible SQL, you also can connect your favorite BI tool—like Superset, Metabase, or Tableau—directly to Cube, and let Cube generate SQL to fetch and display data. Here’s an example of using Superset with the Cube SQL API.

![](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/a775a871-8e89-451c-afb1-671b6dd22e3f.png)

You can check our documentation for the full [Superset integration tutorial](https://cube.dev/docs/recipes/using-apache-superset-with-cube-sql).

## Built-in relational cache

Every query can leverage Cube’s pre-aggregations layer—a database-agnostic materialization engine—in order to make dashboards and reports load in milliseconds instead of minutes.

While defining metrics, developers can specify which metrics they want to pre-aggregate. Cube will run complex calculations beforehand in the background and will create a cache table with the results. All queries will be served from this cache table, dramatically increasing dashboards’ performance.

## Where we go next

We’re excited about today’s launch, but it is only the beginning of the journey!  We’re committed to supporting all the major SQL operations, and improving error handling to provide more useful tips to users. Please let us know in our Slack channel if you see something that is not working as you expected.

Everything described here is available in Cube’s OSS offering, licensed under Apache 2.0. In addition, we're working on cataloguing and collaboration tools for the Cube metrics layer in Cube Cloud, our fully-managed Cube service. Watch this space.

If you’re interested in learning more, [please give us a try.](https://cubecloud.dev/auth/signup)

Introducing the Cube SQL API: Building Metrics Stores with Cube

Today, we’re excited to announce general availability of Cube Cloud, a fully managed service for running Cube applications in the cloud of your choice.

**You can try it out for free at [cube.dev/cloud][cubecloud].**

The launch follows more than two years of collecting community feedback about running Cube in production, which started with my cofounder Pavel and me helping community members manage Cube under load. We learned a lot about different load footprints, use cases, and patterns of how to scale Cube to match different use cases, and began work on what later became Cube Cloud.

> **Our mission:** to help developers focus on building data applications with Cube, not on running, monitoring, and scaling Cube infrastructure.

**Our plan:** a platform purpose-built to run Cube applications, smart enough to autoscale to accommodate usage growth, and rich with controls to trace queries, inspect load, and work with Cube pre-aggregations.

I’m excited to share what we’ve built. Let’s take a look at some important new features:

## Auto-scale Cube API instances
Cube API instances are designed to be scaled horizontally, but it’s hard to say at what CPU, memory, or requests threshold you need to scale, since this depends on a variety of factors, including your schema structure and footprint, API workload patterns, and database resources.

In Cube Cloud, you don't need to worry about any of it—how many instances to run, how much memory to allocate, etc. Cube Cloud knows how to autoscale your deployment based on API workloads and a multi-tenancy structure to load-balance schemas per API instance.

Cube Cloud also auto-scales pre-aggregations of the Cube relational caching layer. Auto-scaling ingestion helps to keep cache refresh time constant even when the underlying data grows, and the same is true for the querying side: Cube Cloud takes care of bringing more Cube Store workers to process all the incoming requests.

![autoscale.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/9c62128c-195a-4ddf-b887-f63e61eb2033.png)

## Cache and pre-aggregations warm-up
To ensure a good user experience, we recommend pre-compiling your schema and warming up both in-memory and pre-aggregations caches before serving user requests. It usually happens during the rollout of updates to the data schema. When building multitenant applications and load balancing different schema versions across API instances, it is important to pre-compile versions of the schema per serving API instance, especially for dynamic schemas that make third-party requests.

Cube Cloud takes care of schema pre-compilations and cache warm-ups for you, including balancing multi-tenant schema versions.

![multi-tenant-schema.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/148c34dd-1367-4e35-8d28-d5936f23dcc7.png)

## GitHub integration and collaborative schema editing
Cube Cloud integrates with GitHub to streamline the process of developing and testing schema edits with your team.

First, Cube Cloud can automatically deploy changes from a connected GitHub repository. Simply commit and git push to trigger a deployment.

Additionally, if you want to edit and test your schema directly in Cube Cloud, you can do so via Development Mode, which deploys your own personal API endpoint to test out via Playground as you edit the schema. When you’re happy with your updates, you can commit and push your changes via the git integration, then share these updates with the rest of your team or push them to a production environment in Cloud.

![schema_editing.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/7da053d1-21d4-46e1-bb63-372ef3e08eed.png)

We’re now actively working on some neat features to support staging environments and more collaborative workflows. For example, multiple branches on each deployment will soon allow you to set up multiple environments with the same backend data source and caches, while using different schema versions and resource configurations. We’ll also enable you to open pull requests and merge branches directly from Cube Cloud. Watch this space.

## Monitoring
Cube Cloud provides performance monitoring for API instances and your source database. Use this tool to monitor overall load, and spot slow queries to optimize.

![monitoring.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/6f0b86f3-6952-4460-aa82-168b306c2e97.png)

Additionally, we're working on integrations with third-party alerting and APM tools like Datadog and PagerDuty, so you can export metrics and logs of your Cube Cloud deployment to these tools.

## Query Tracing
Query tracing provides a historical view of all your Cube API and database queries. This can be very helpful both during development and in production to find slow queries, understand the lifecycle of each query, and optimize performance and caching configuration.

![query_tracing.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/2c985fed-e80c-4a70-be8a-9f43bfb42372.png)

## Pre-aggregations management
Cube can accelerate a query by utilizing pre-aggregations—aggregated and optimized representations of source data—instead of processing raw data. In Cube Cloud, it’s easier to create and configure pre-aggregations during development and maintain them in production. We've built a control panel to view all your pre-aggregations in one place.

![pre-aggregations.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/6b07ade3-5246-40cc-a033-47c94389bcde.png)

This tool lets you view build history, partitions, build times, refresh times, and much more. You can also build pre-aggregations manually with the `Build All` button, or select individual partitions to build.

![pre-aggregations-3.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/db60d2cd-d8ff-45a3-8891-49d6729627ad.png)

To track the history of pre-aggregations over time, Cube Cloud provides a list of every build that happened.

![pre-aggregations-2.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/45bac091-e3f4-4dc8-b235-f40f7724e3df.png)

## Security
Cube Cloud is not just the fastest way to run Cube; we aim for it to be the most secure.

For starters, all communications between Cube Cloud and your databases can be encrypted. In addition, it’s possible to run Cube Cloud in an isolated, dedicated VPC that can be peered to your database's VPC.

Encryption at rest for pre-aggregations using Customer Key Management is available for dedicated VPC instances.

To make it easier to use Cube Cloud in large enterprise environments, we’re actively in the process of earning our SOC-2 certification. If you’d like to be updated on our progress with this certification, please let us know.

## Starting price: $0
We’re so confident that Cube Cloud is the superior way to develop and run Cube, and proud of the features that we’ve added, that we encourage every developer to use it.

Cube Cloud is totally free for up to 1GB/month of processed data—with no limitations on access to the Cube Cloud IDE and no cap on the number of data stores you connect or API instances you create. [Give it a try now][signup]. (For additional data as well as auto-scaling and high availability, pricing starts at just $99/month. [Learn more about Cube Cloud pricing here][pricing].)

If your team requires dedicated infrastructure, support and training, SLAs, or other custom enterprise features, [get in touch][contact]: we’re happy to help.

## This is just the beginning
More is definitely in store! We will continue to build more tools in Cube Cloud to ensure smooth development, staging, and production for Cube-powered data applications. We also hope you’ll share your feedback and feature requests by [dropping us an email][email], [finding us on Twitter][twitter], or [participating in our Slack community][slack].


[cubecloud]: https://cube.dev/cloud
[signup]: https://cubecloud.dev/auth/signup
[pricing]: https://cube.dev/pricing
[contact]: https://cube.dev/contact
[email]: mailto:hello@cube.dev
[twitter]: https://twitter.com/thecubejs
[slack]: https://slack.cube.dev/

Announcing Cube Cloud: Managed hosting of Cube applications


Happy summer from Cube! Here’s some of what we’ve been working on to help you build powerful analytics applications with your data…

## Release notes: Cube.js v0.28
Cube.js v0.28 includes tons o’ goodies for developers, improvements to Cube Store, and a raft of reliability improvements to our database drivers.

**[Read the release notes →][7]**


## Awesome dataviz tools for software developers

![awecome_dataviz.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/552be985-edec-4d7b-b886-af291f497705.png)

We’ve assembled an open-source list of awesome data visualization tools to help you select the right tool or library for your project. [awesome.cube.dev][1] is an interactive way to select tooling based on criteria including compatibility with your front-end framework, license terms, specific features, and recent updates. The tool also features links to tutorials and guides.

**[Learn more about awesome.cube.dev →][2]**


## Video workshop: Mastering Cube.js Pre-Aggregations

![mastering_pre-aggregations.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/3d8f5128-a65c-420e-b547-8fc0eaf2bb73.png)

Cube [pre-aggregations][3]—materialized query results persisted as tables—are a key way to speed up Cube.js queries, but they’re frequently misunderstood. If you want to learn why and when to use pre-aggregations, how to get started, and more tips and tricks, don’t miss this recording of a recent 2-hour interactive workshop. We cover multiple use-cases, step through configuration, and leave lots of time for live Q&A.

**[View the workshop →][4]**


## Help us build Cube Cloud: fully-managed Cube.js apps
![help_us_build_cube_cloud.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/e77bf882-eaf0-4efd-bdbc-2b9b352664f5.png)

[Cube Cloud][5] offers Cube.js hosting with extra features for reliability and performance. It includes all the core features of Cube.js while abstracting away infrastructure concerns including managing the number of instances, memory, high availability, pre-aggregations management, caching, real-time monitoring, and tracing.

We’ll be making Cube Cloud generally available later this year, but first we need your help: we’re searching for design partners to test the pre-release product. You’ll participate in a structured program to collect your feedback, and receive the opportunity to influence our product roadmap and features.

**[Learn more about Cube Cloud and request early access →][6]**




## In case you missed it…

### Cube Dev’s Series A

Last month, Cube Dev raised $15.5M in Series A funding. Our CEO Artyom Keydunov shares what we’ve learned since our seed funding and explains how we intend to use this funding round. **[Our Series A →][8]**


### How FeedMe redesigned their reporting system using Cube.js

FeedMe provides the food and beverage industry with point-of-sale systems and a suite of detailed sales reports, marketing analytics, and inventory management built in JavaScript and powered by Cube. **[Read the User Story →][9]**


### Celebrating Cube.js community accomplishments

Ray Paik, Cube’s Head of Community, reviews eight months of community growth and the multiple ways to get involved in the Cube community. **[Catch his recap →][10]**


## Show us your stuff

What have you built with Cube? We want to see it, help spread the word about your projects, and hook you up with some sweet, sweet Cube merch. [Find us on Slack][24] or [hit us up on Twitter][25].

## Take us to your inbox

Sign up below to subscribe to the Cube.js newsletter via email. Never miss news, releases, and the latest updates from the Cube Dev community.

That's all, folks. Stay healthy!

_With ♥️, the Cube team_


[1]: http://awesome.cube.dev/
[2]: https://cube.dev/blog/introducing-awesome-dataviz-tools/
[3]: https://cube.dev/docs/schema/reference/pre-aggregations
[4]: https://www.youtube.com/watch?v=1_BP3B_wbec
[5]: https://cube.dev/docs/cloud
[6]: https://cube.dev/cloud
[7]: https://forum.cube.dev/t/v0-28-release-notes/248
[8]: https://cube.dev/blog/our-series-a/
[9]: https://cube.dev/blog/feedme-runs-on-cubejs/
[10]: https://cube.dev/blog/growth-of-the-community/
[24]: https://slack.cube.dev/
[25]: https://twitter.com/thecubejs

Summer update: awesome tools, pre-aggregations workshop, release notes…

Less than a year ago, I was writing a blog post [announcing our seed fundraising](https://cube.dev/blog/cube-dev-raises-62M-to-accelerate-Cubejs-development/). I didn't expect to write a successor post so soon! But today we are overjoyed to announce that [Cube Dev has raised $15.5M in Series A funding][bw], led by Decibel.

It’s been a busy eleven months. Since last August, there are 5 times more servers running Cube, and as many new members in our Slack community. The past months have also seen the rollout of powerful new features including [Cube Store](https://cube.dev/blog/introducing-cubestore/), and the emergence of new community features and educational resources.

In announcing this funding round, I want to share what we’ve learned over that time, and explain how we will use the new funding and what we’ve learned to more quickly build the tech stack that powers data applications.

## The new architecture for data applications

The database and cloud data warehouse industry is booming. We indeed have more data than ever. From the Cube community, I know a lot of seed-stage startups dealing with hundreds of millions and even billions of records.

The good news is that the technology now exists to process all of this data. Remember that less than a decade ago, only a privileged few big tech firms could afford the budget needed to run and manage Hadoop clusters on a scale, but since then, cloud data warehouses emerged as Ford's Model T for data. Now, even small companies can afford to store and analyze big data. This enables everyone to build data applications and embed intelligence into their products.

That being said, we still have to build fundamental infrastructure to make it possible—to bring data to life inside the applications. If data is the new oil, we need the tools to refine it into rocket fuel.

![Cube scheme](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/02311426-3a75-4c2f-a128-72f1cc320f4e.png)

Developers need a new architecture to deliver data from warehouses and data lakes to applications at  scale, and there are hard problems to solve in designing this architecture:

* How do we abstract metrics definitions and make them consistent, testable, and version controllable?
* How do we ensure that our API layer supports a large number of users and high volumes of data without growing in complexity?
* How do we manage complex multi-tenancy scenarios and per-end-user custom metrics?
* How do we aggregate and cache the data to process hundreds and even thousands of queries per second?

We’ve spent more than two years tackling these questions with all of you in our amazing Cube community, and we’ve all learned a lot.

Over two years ago, when Pavel and I were playing ping-pong and chatting about open-sourcing Cube.js, I couldn't have imagined where it would lead us. Today’s Cube is so different from what we initially released. Community not only contributed features but pioneered many fundamental architecture decisions we have now in the Cube.

Now, with new funding, we're excited to start a new chapter and to build more and faster.

## Where we go from here

Cube’s mission is to **empower developers to build data applications**. To pursue this mission, here is some of what we have planned:

* **Hiring.** We plan to build a team of 30 people by the end of the year and double our open-source engineering team. This allows us to fix bugs and ship new features faster.
* **Extend Cube Store.** It's been less than 3 months since we released Cube Store, but it's already been adopted by more than half of Cube users. As many of you have noticed, Cube Store provides a significant boost for latency and concurrency on analytics queries like Top-K. We intend to further improve its performance and the ease of configuring and deploying Cube Store and work to make its immense power accessible to every user.
* **New features.** I always enjoy hours spent chatting with the community about what we need to build next. There are many things I know you are excited about. Two frequent requests are **real-time pre-aggregations from Kafka streams and databases**, and **BI connectors**. I'm happy to share that we've already started to work on these, and plan to ship previews later this year. Please DM me on Slack if you are interested in running Cube on top of real-time and streaming data, or connecting Cube to your BI tool.
* **Launch Cube Cloud.** We're also hard at work building Cube Cloud, our fully managed Cube service coming later this year too. Managing infrastructure opens so many new opportunities to build features we’ve always wanted to add to Cube but couldn’t without access to the underlying infrastructure. I can't wait to share it with you all. Cube Cloud will launch to general availability in the autumn, but first we’re working with design partners who can test Cube Cloud with their data, influence feature development, and work hands-on with the Cube Dev team in advance of general availability. Can you help? Don’t miss this opportunity to shape Cube’s next chapter. [Drop us a line](https://cube.dev/cloud/) to learn more.

## Join us!

We’re looking for talented, energetic team members to join Cube Dev both remotely or here in San Francisco. Help advance our mission to create great tools to help developers build modern data applications. [View open positions](https://cube.dev/careers/#positions) and watch this space—we’re growing fast!

[bw]: https://www.businesswire.com/news/home/20210719005008/en/Cube-Dev-Raises-15.5-Million-to-Help-Companies-Build-Applications-with-Cloud-Data-Warehouses

Our Series A

On behalf of the Cube.js team, I'm proud to present an open-source list of awesome data visualization tools that we've built to help you choose the best charting libraries and other tools for your needs. It's available at [awesome.cube.dev](https://awesome.cube.dev) and here's the story behind it.

Have you ever wondered which tool or library to choose to add a chart, a map, or maybe a data grid to your app built with the `%BEST_FRONTEND_FRAMEWORK%` and the `%BEST_PROGRAMMING_LANGUAGE%`?

I know for sure that you have! Every other week someone from the [Cube.js community](https://slack.cube.dev) asks for advice on choosing a dataviz tool. So, I decided to talk to the community and run a [survey](https://dev.to/cubejs/dev-survey-front-end-data-visualization-tools-1f37) that got dozens of replies.

Let me share what I learned.

## Choosing a tool is hard 😬

I discovered that there're two types of criteria that developers evaluate when the need to choose a dataviz tool arise.

**First, you need to check if a particular tool is compatible with the technology stack that you're using,** and by that I mean the front-end framework and the programming language. Obviously, [Recharts](https://recharts.org/en-US/) is a neat tool but there's no way to use it in a Vue app because it's purposefully built for React. Same is true for the language: according to the survey, *2/3 of all developers use TypeScript*, so making sure that the tool has built-in type definitions or something available via [DefinitelyTyped](https://github.com/DefinitelyTyped/DefinitelyTyped) is also crucial.

**It's also nice to check the license.** Discovering that something that you've already built into your app is not open-source and free to use may be painful. In the other case, when you specifically look for a paid tool with enterprise support, checking the license and pricing beforehand is crucial as well. According to the survey, *20 % of all developers use paid tools.*

**Then, you'll see how easy it is to get started, the hard way.** The adoption curve is heavily influenced by the ease of use and available docs and examples. Developers particularly praised the tools that have *readily available "getting started" pages with code examples* as well as the tools with *excessive number of examples.* [D3.js](https://observablehq.com/@d3/gallery) and [ECharts](https://echarts.apache.org/examples/en/index.html) are obvious winners in this category.

**In the end, you might face the sad truth: a tool is rooted in the past.** Your success will depend on the flexibility that a tool can provide for your use cases. And if it can't, soon you'll find yourself checking if there's much activity in the issues and pull requests sections on GitHub. Let's hope you'll find that a tool currently has a vibrant and active community.

## Now, choosing a tool is easy 😎

Equipped with this evidence, we've committed to solve the dataviz tools choice problem once and for all. We've built an open-source list of data visualization tools for software developers that will support you at every step.

In the very beginning, you'll have the chance to pick the front-end framework of your choice and specify if you care about TypeScript support. If you're an open-source aficionado, there's an option for that, too. Also, if you come for something specific (e.g., a data grid), you'll have a filter for that as well.

![Filters](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ylim2ipawsvzbatze6z6.png)

In an instant, you'll see the awesome tools matching your criteria. For those tools that gathered a lot of mentions in the survey results, you'll see badges like "Easy to customize" or "Full-fledged" that you can use to set expectation.

![Cards](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yve18kj4es537noo7s30.png)

Definitely feel free to pick some candidates and deep dive into their pages. You'll be able to build an understanding of what to expect in terms of flexibility and the community dynamics. Checking the last release date and the number of stale issues are also wise things to do.

![Data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t0z9v5w6nv65liyc6bsd.png)

Ready to get started? You'll find links to tutorials and guides that, hopefully, will help you get on track. And if you're stuck, there are a few links for that, too.

As the last but not the least resort, you can always come to the [Cube.js community](https://slack.cube.dev) on Slack. It has thousands of developers building apps for which Cube.js serve as the open-source API layer that delivers data to the dataviz tools on the front-end.

![Getting Started and Getting Help](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dtmpivikvtad3n1zf621.png)

**Now, it's time to visit [awesome.cube.dev](https://awesome.cube.dev) and see it in action!** If you like it, don't hesitate to share a link with your friends. In any case, leave us a comment in Slack: we'd love to know what you think 😇

*P. S. Just like [Cube.js](https://github.com/cube-js/cube.js), awesome.cube.dev is [open-source](https://github.com/cube-js/awesome-tools). We appreciate your issues, suggestions, and pull requests.*

Data Visualization

Awesome dataviz tools for software developers 📊📈


Happy spring from Cube! Here’s some of what we’ve been working on to help you build powerful analytics applications with your data.

## Introducing Cube Store: High concurrency and sub-second latency for any database

![introducing_cube_store.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/50ae85c7-8f96-4435-bde7-1ed9acf890b7.png)

[Cube Store][2] is the custom pre-aggregation storage layer for Cube.js, now available to all Cube users. We strongly recommend using this to store your pre-aggregations.

You may have stored pre-aggregations—pre-computed, partitioned, stored, and refreshed representations of your source data either internally, using the same database as your source data, or externally, in a custom-provisioned PostgreSQL or MySQL instance. However, doing this incurs concurrency and latency costs that limit the performance of analytical APIs.

Cube Store is a drop-in replacement for your existing pre-aggregation storage layer, and it’s optimized for easy and predictable horizontal scaling. Regardless of your data source and data volume, Cube Store provides high concurrency, sub-second latency, and performance optimizations for analytical queries. Give it a look.

**[Learn more about Cube Store →][3]**

## New in the Developer Playground

![new_in_the_developer_playground-2.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/e0e96504-391f-43f5-b944-1dc336bdbcad.png)

Cube’s [Developer Playground][5] has seen a round of quality-of-life improvements, most notably:

- The Security Context for queries can now be modified before making requests;

- Invalid dimensions/measures are now clearly marked;

- Queries must now be explicitly run by clicking the Run button

- You can start a Cube.js project without any database configuration: upon first boot, the Developer Playground will launch a connection wizard;

- Vue code snippets can now be generated;

- Vue-based charts can now be added to the Dynamic Dashboard.

If you haven’t recently tried the Developer Playground to generate and view a schema, build and execute queries, plot results, and explore dashboard application templates, we think you’ll be pleasantly surprised.

**[See full Developer Playground documentation →][6]**

## Introducing the Cube Community Forum: tips, tricks, and swag

![community_forum.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/7867e1d1-2497-466b-a62f-a48aefeb6ac4.png)

Cube’s Head of Community, Ray Paik, recently launched the [Cube.js Community Forum][7] to help Cube users publish your use cases, share your tips & tricks, collaborate on questions, and build a community knowledge base.


Designed to supplement our popular [Slack][9], the forum is a platform where it will be easier to search discussions and collaborate asynchronously.

To encourage everyone to join the conversation, [we’ve even got neat swag][10] to reward pioneers who launch a topic. Drop in and say hello!

**[Visit the Cube.js Community Forum →][8]**

## Tutorial: Multi-Tenant Analytics with Auth0 and Cube.js

![multi-tenant_analytics_with_auth0.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/1617ce05-551e-423c-8c1a-33ce90101509.png)

Here’s a great new tutorial for learning how to secure web applications with industry-standard and proven authentication mechanisms including JSON Web Tokens, JSON Web Keys, and the OAuth 2.0 protocol.

Starting with an openly accessible, insecure analytical app, the tutorial walks through each of the steps required to make a secure, multi-tenant app with role-based access control and an external authentication provider. Start learning!

**[Try the Multi-Tenant Analytics tutorial →][11]**

## We want you: try Cube Cloud

![0d284510-ed12-bf7c-1fb3-e291ebe1ab69.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/e26276ab-f8bf-4854-b952-56d5cabcb163.png)

[Cube Cloud][13], coming mid-2021, will make it possible to deploy fully managed Cube.js apps in the cloud of your choice. We’re currently soliciting pre-launch trial users to kick the tires, find the rough edges, and share their feedback. Could this be you? [Take a look at a demo from earlier this year][14], then get in touch.

**[Learn more about Cube Cloud and join the waitlist →][13]**

## Release notes: Cube.js v0.27

Breaking changes and deprecations: None. Nada. Nothing. Natch. It just works!

**[Read the release notes →][16]**

## In case you missed it…

### Cube ♥️ Simon Data. Simon Data ♥️ Cube.

How does Simon Data address runaway analytics complexity with Cube.js? [We’re glad you asked.…][17]

### Analytic Tools for ClickHouse

Earlier this spring, our Igor Lukanin spoke at a ClickHouse Bay Area virtual session on analytic tools for ClickHouse in particular and data warehouses in general. [Catch the video…][18]

### BigQuery Public Datasets for COVID-19 Impact Research

Explore how to build an analytical application on top of BigQuery using a few public datasets to [visualize COVID-19’s impact on people’s lives][19].

### React Pivot Table with AG Grid and Cube.js

Adding a pivot table to a React app using the world’s best JavaScript grid is even easier than you think—[really, really easy][20].

## Call us, <s>maybe</s> definitely

The next Cube Community Call is July 14 at 9:00a PDT. [Add it to your calendar][22], then [check out past calls][23].

## Show us your stuff

What have you built with Cube? We want to see it, help spread the word about your projects, and hook you up with some sweet, sweet Cube merch. [Find us on Slack][24] or [hit us up on Twitter][25].

## Take us to your inbox

Sign up below to subscribe to the Cube.js newsletter via email. Never miss news, releases, and the latest updates from the Cube Dev community.

That's all, folks. Stay healthy!

_With ♥️, Jon, the—new!—marketing guy, and the Cube team_

[1]: https://cube.dev/blog/introducing-cubestore/
[2]: https://cube.dev/blog/introducing-cubestore/
[3]: https://cube.dev/blog/introducing-cubestore/
[4]: https://cube.dev/docs/dev-tools/dev-playground
[5]: https://cube.dev/docs/dev-tools/dev-playground
[6]: https://cube.dev/docs/dev-tools/dev-playground
[7]: https://cube.dev/blog/introducing-community-forum/
[8]: http://forum.cube.dev/
[9]: https://slack.cube.dev/
[10]: https://twitter.com/thecubejs/status/1385253435637788677
[11]: https://multi-tenant-analytics.cube.dev/
[12]: https://cube.dev/cloud
[13]: https://cube.dev/cloud
[14]: https://youtu.be/mu1_PdCpfCk?t=2277
[15]: https://forum.cube.dev/t/v0-27-release-notes/128
[16]: https://forum.cube.dev/t/v0-27-release-notes/128
[17]: https://cube.dev/blog/simon-data-runs-on-cubejs/
[18]: https://www.youtube.com/watch?v=tZWvJZQ_jDo&t=2362s
[19]: https://bigquery-public-datasets.cube.dev/
[20]: https://react-pivot-table.cube.dev/
[21]: https://cube.dev/community-call/
[22]: https://cube.dev/community-call/
[23]: https://www.youtube.com/playlist?list=PLtdXl_QTQjpb1dHZCM09qKTsgvgqjSvc9
[24]: https://slack.cube.dev/
[25]: https://twitter.com/thecubejs

Spring 2021 newsletter: Cube Store! Vue in the Playground! A new forum!

Today, with the generally available release of Cube Store, Cube.js is fully ready to provide operational analytics for any SQL-compliant database, data warehouse, or query engine.

You can use open-source [Cube.js](https://github.com/cube-js/cube.js) and [Cube Store](https://github.com/cube-js/cube.js/tree/master/rust) to build an analytical API that will respond to highly concurrent requests from your application with sub-second latency regardless of data volume or workload. Even if you have a billion-scale dataset and thousands of concurrent users, Cube.js and Cube Store will get you covered.

Cube Store, the custom pre-aggregation storage layer for Cube.js, is now generally available and strongly recommended for use in production. Let's explore the motivation and design decisions behind Cube Store and learn more about the benefits of using Cube Store for new and existing Cube.js users.

## Why Cube Store?

Cube.js is an open-source analytical API platform that allows developers to build an analytical API and a semantic layer based on data schema over any SQL-compliant data store. In the simplest scenario, Cube.js uses in-memory cache and query queue to provide better performance than the data store is able to deliver. However, it's usually the least scalable and cost-effective solution, so it's not recommended for production.

![cube store-04-3-03.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/e8dbf9eb-5ce2-4b3f-ac44-2954ef4f6824.png)

**In a real-world scenario, Cube.js speeds up analytical queries using pre-aggregations.** These are the condensed representations of source data that are pre-computed, partitioned, stored, and refreshed in a way that allows for super-fast execution of select queries.

Historically, pre-aggregations were stored *internally*, alongside the source data in a database (e.g., PostgreSQL or MySQL), or *externally*, in a custom-provisioned instance of PostgreSQL or MySQL, as the only feasible option for read-only or cost-ineffective data sources (e.g., AWS Athena or BigQuery). Usually, they will be renewed asyncronously by a dedicated refresh worker instance. It's an acceptable solution, however, the pre-aggregation database becomes a bottleneck and limits the scalability of the analytical API.

![cube store-04-2-03.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/ac22387e-7c1c-4c70-9a35-430f789a7cfb.png)

**Pre-aggregations in traditional databases often won't allow for high concurrency and low latency of the analytical API.** They also were the source of frustration due to poor performance in important use cases (e.g., with high cardinality of pre-aggregated data, with partitioned data, with joins across several databases), unsupported features (e.g., approximate distinct counts via HyperLogLog algorithm), or implementation details (e.g., support for long table names or support for various SQL types).

**Cube Store is designed to resolve these issues and provide a performant pre-aggregation storage layer for Cube.js.** It's a drop-in replacement for any currently used pre-aggregation storage and the go-to solution for new Cube.js apps. Being a distributed data store itself, Cube Store can be scaled horizontally with ease and predictability. 

![cube store-04-1-03.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/4f6d753e-77d2-4d46-964f-8d2512ff9a18.png)

**From now on, Cube Store is the recommended pre-aggregation storage layer for production use.** It doesn't have the limitations of other databases used to store pre-aggregations. On top of that, Cube Store provides guaranteed high concurrency and sub-second latency, performance optimizations for analytical queries, and additional features such as data federation via cross-database joins. You'll get these advantages of Cube Store regardless of your data source and the data volume.

![cube store-01-03.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/67ac0250-4bda-4028-ae56-ec23324bcebf.png)

## How Cube Store works

Cube Store uses a well-known and proven distributed query engine architecture. In every Cube Store cluster:

- a single *router* node handles incoming connections, manages database metadata, builds query plans, and orchestrates their execution
- multiple *worker* nodes ingest warmed up data and execute queries in parallel
- a local or cloud-based blob storage keeps pre-aggregated data in columnar format

![cube store-03-03.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/db0e1aeb-3101-4280-b4a4-902e21bcd9a0.png)

**Cube Store was designed with performance in mind.** It's written entirely in Rust to leverage low-level optimizations, effectively making Rust the top-1 language by lines of code in Cube.js codebase. Cube Store allows for almost unlimited horizontal scalability by increasing the number of worker nodes. It stores pre-aggregated data in Parquet format which was deliberately chosen for its efficiency and performance.

**Cube Store is based on well-known open-source building blocks.** Together with [Parquet](https://en.wikipedia.org/wiki/Apache_Parquet), Cube Store uses [Apache Arrow](https://github.com/apache/arrow) for its efficient in-memory data structures and [DataFusion](https://github.com/apache/arrow-datafusion) as a query execution framework.

**Cube Store has been used in [Cube Cloud](https://cube.dev/cloud/) for more than 6 months.** A few select early access users have been exposed to Cube Store in Cube Cloud and experienced substantial performance improvements thanks to Cube Store.

## How to use Cube Store

Cube Store is ready to be used while developing new Cube.js applications and also as a drop-in replacement for the pre-aggregation storage layer in existing Cube.js apps.

**Development.** When Cube.js is run in [development mode](https://cube.dev/docs/configuration/overview#development-mode), a single-node in-process instance of Cube Store is enabled. It requires zero configuration and allows to instantly use the pre-aggregations. All you need is to mark them as *external*:

```jsx
cube(`Cube`, {
  sql: `select * from table`,

  preAggregations: {
    main: {
      type: `rollup`,
      external: true,

      // ...
    },
  },
});
```

Cube.js will build pre-aggregations and store them in Cube Store. You can review the console output of Cube.js to see if queries are served using pre-aggregations. You can also see matched rollups on the Cache tab in Cube.js Developer Playground available at [localhost:4000](http://localhost:4000).

**Production.** A Cube Store cluster should be run in production and consist of a single router instance and several worker instances. There's an official `cubejs/cubestore` [Docker image](https://hub.docker.com/r/cubejs/cubestore) as well as a [Docker Compose template](https://cube.dev/docs/caching/using-pre-aggregations#running-in-production) you can use as a starting point.

Just like Cube.js, Cube Store can be configured via [environment variables](https://cube.dev/docs/reference/environment-variables#cube-store), e.g., Cube Store nodes assume the worker role when `CUBESTORE_WORKER_PORT` is set and ingest pre-aggregated data from the blob storage specified with `CUBESTORE_GCS_BUCKET` or `CUBESTORE_S3_BUCKET`.

**Benchmarks.** Depending on the currently used pre-aggregation storage and the data volume, after switching to Cube Store, you can expect your API performance to stay at least at the same level or increase substantially. You can review and use a [benchmark template](https://github.com/cube-js/cube.js/tree/master/examples/cubestore-benchmark) leveraging [k6](https://k6.io), an open-source load testing tool, to do your own measurements prior to switching to Cube Store.

![cube store-02-03.png](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/13a9bde1-5ddb-4b09-a7b1-d683acaaa42e.png)

## Next steps

**Here at the Cube.js team, we wholeheartedly encourage you to use Cube Store in development and consider migrating your production deployments to Cube Store as well.**

Please review the [release notes](https://forum.cube.dev/t/v0-27-release-notes/128) and [documentation](https://cube.dev/docs/caching/using-pre-aggregations#pre-aggregations-storage), share your experience in the *#cube-store* channel in [Slack](https://slack.cube.dev) or post to [Discourse](https://forum.cube.dev), and don't hesitate to file Cube Store-related issues on [GitHub](https://github.com/cube-js/cube.js/issues).

Also, feel free to sign up for the waiting list of [Cube Cloud](https://cube.dev/cloud/) which provides a fully managed Cube Store cluster for all deployments. Cube Cloud is currently in early access but will be generally available later this year.

Introducing Cube Store: High concurrency and sub-second latency for any database

Cube.js was open-sourced in March 2019, and ever since then we've been adding more features to support diverse use cases and provide better developer experience. However, as the pace of Cube.js development accelerates, we have reached a point where it's time to do some house cleaning and improve how things are done while being transparent with our plans.

As part of this effort, we're very pleased to announce our new [Deprecation Policy](https://github.com/cube-js/cube.js/blob/master/DEPRECATION.md), which can be found in the Cube.js repository on GitHub. In this blog post, we'd like to explain the deprecation policy and highlight some of the features that are planned to be deprecated.

# Deprecation Policy

A **feature to be deprecated** can really be anything, e.g., an obsolete configuration option, the support for an undesirable architecture pattern, or an ability to work with a certain runtime. We're going to apply the same policy to all deprecated features regardless of their kind or scale.

In order to deprecate a feature, we'll follow this procedure:
* a feature is marked as **deprecated** within the [documentation](https://cube.dev/docs) and in the [release notes](https://github.com/cube-js/cube.js/blob/master/CHANGELOG.md)
* a relevant entry is added to the [`DEPRECATION.md`](https://github.com/cube-js/cube.js/blob/master/DEPRECATION.md) file on GitHub with a deprecation date or release clearly specified and migration instructions provided
* the feature remains in Cube.js for **at least one stable release** and usually longer, as specified 
* the feature will most probably be removed from Cube.js after specified date or release, so all users are expected to migrate prior to this happening

# Deprecated Features

As of December 2020, these notable features are marked as deprecated:

- The [`contextToDataSourceId` configuration option](https://github.com/cube-js/cube.js/blob/master/DEPRECATION.md#contexttodatasourceid) has been replaced by `contextToOrchestratorId`. It was removed in v0.25.0, released in December 2020
- The [`CUBEJS_ENABLE_TLS` configuration option](https://github.com/cube-js/cube.js/blob/master/DEPRECATION.md#cubejs_enable_tls) will be removed in favor of using load balancers or reverse proxies. It will be removed in v0.26.0, expected in January 2021
- [Support for Node.js 8](https://github.com/cube-js/cube.js/blob/master/DEPRECATION.md#nodejs-8) due to it reaching end-of-life. It will be removed in v0.26.0, expected in January 2021
- [Embedding Cube.js within Express](https://github.com/cube-js/cube.js/blob/master/DEPRECATION.md#embedding-cubejs-within-express) in favor of stand-alone microservice deployments. It will be removed in June 2021

Please refer to the [Deprecation Policy](https://github.com/cube-js/cube.js/blob/master/DEPRECATION.md) on GitHub for detailed info.

As always, we appreciate all the feedback we get from our community. Feel free to share on [Slack](https://slack.cube.dev), [GitHub](https://github.com/cube-js/cube.js/issues), or [Stack Overflow](https://stackoverflow.com/questions/tagged/cube.js).

Hassan Khan

Announcing Cube.js Deprecation Policy

We're thrilled to share that Cube.js is releasing an official Docker image and amplifying its support for the microservices architecture.

Read more to explore the positive effects of this change on the scalability of Cube.js applications, the migration path to Docker, and the plans for the Express support.

## Motivation for the Change

Historically, Cube.js has been being embedded in Express applications running on Node.js. It makes a lot of sense since Cube.js is written mostly in JavaScript and provides API endpoints that work great alongside other Express routes.

However, while the amount of Cube.js production deployments continues to grow, we kept exploring how developers approach scalability and infrastructure management issues. We learned that Cube.js should be scaled differently than Express applications and deployed as fully isolated microservices that don't share any resources with other parts of the application.

Frankly, we never designed Cube.js to share the event loop with other parts of Express applications running on Node.js. As Cube.js gets more features, it becomes even heavier on the computational side. We've observed intermittent lags appearing under load in a lot of embedded Cube.js deployments which was never the case for the microservice deployments. And while every embedded use case is different, it narrows our ability to track down such issues and work around them on a case-by-case basis.

Although small to medium deployments are able to work as a part of Express applications, this configuration adds unnecessary complexity to the infrastructure, interferes with the best practices of DevOps, and often leads to suboptimal performance for [pre-aggregations](https://cube.dev/docs/caching#pre-aggregations) which are the ultimate feature of Cube.js allowing to serve complex analytical queries with low latency.

![cubejs-docker-1.1-2.png](https://media.graphcms.com/lOKd4eslRSiYVG7Ws84Q)

**We decided that embedding into Express will no longer be a recommended way to run and deploy Cube.js. As a more reliable and scalable alternative, we encourage developers to use Cube.js as a standalone microservice.** Serverless is also a good way to deploy Cube.js applications, and we have no plans to discontinue using it. We will keep the support for Express for the next six months, until June 2021, and provide support with the transition to microservices.

## Cube.js in the Microservices Architecture

We have released several features to enhance the support for using Cube.js in the microservices architecture. Since version 0.23, you're welcome to use these feature, and full details are available in the [changelog](https://github.com/cube-js/cube.js/blob/master/CHANGELOG.md) and our [public roadmap](https://github.com/cube-js/cube.js/projects/6#card-48286488).

**Deployment.** Cube.js now has an [official Docker image](https://hub.docker.com/r/cubejs/cube) based on Alpine Linux.

Running Cube.js in containers makes it easier to deploy and manage installations in on-premise and cloud-based environments, such as Kubernetes, Amazon ECS, Google Compute Engine, or [Cube Cloud](https://cube.dev/cloud/). It also provides a straightforward way to run a dedicated [refresh worker](https://cube.dev/docs/deployment/production-checklist#set-up-refresh-worker) instance which is the best practice for keeping pre-aggregations up-to-date in the background. See the [Deployment Guide](https://cube.dev/docs/deployment/guide) and the [Production Checklist](https://cube.dev/docs/deployment/production-checklist#set-up-refresh-worker) in our documentation to learn more.

Even more importantly, official Docker images make it easier to maintain Cube.js deployments with minimal engineering effort needed to update to the latest version of Cube.js.

**Development.** Running `npx cubejs-cli create new-application` will create a Cube.js application using the new `docker` template which provides several new features:

- the `cube.js` file for configuration options which is more concise than the former `index.js` file (see the [Configuration Overview](https://cube.dev/docs/configuration/overview) in our documentation to learn more)

```js
// Cube.js configuration options: https://cube.dev/docs/config

module.exports = {
  // Cube.js options go here, none by default
  // Options set via environment variables take precedence
};
```

- the `docker-compose.yml` file for running Cube.js installations as microservices with Docker (see the [Getting Started with Docker](https://cube.dev/docs/getting-started-docker) in our documentation to learn more)

```yaml
services:
  cube:
    image: cubejs/cube:latest
    ports:
      - 4000:4000  # Cube.js API
      - 3000:3000  # Developer Playground
    env_file: .env
    volumes:
      - ./dashboard-app:/cube/conf/dashboard-app
      - ./cube.js:/cube/conf/cube.js
      - ./schema:/cube/conf/schema
```

While you still can run Cube.js via `npm run dev` thanks to the existing `package.json` file, now you can also run Cube.js in Docker without npm or Node.js being present on your machine. It makes Cube.js more appealing to developers and engineering teams using different languages and ecosystems.

## What's Next

We encourage you to plan the migration and move existing Cube.js applications to the microservices architecture and Docker. Use this [migration guide](https://cube.dev/docs/configuration/overview#migration-from-express-to-docker-template) to update the source code and the [Deployment Guide](https://cube.dev/docs/deployment/guide) for the advice on popular deployment environments.

If you have any questions or need assistance with the migration, please reach out to the Cube.js team and the community on [Slack](https://slack.cube.dev) or file an [issue on GitHub](https://github.com/cube-js/cube.js/issues).

Cube loves Docker

I’m excited to share that Cube Dev, the company behind Cube.js, has raised a $6.2M round led by Bain Capital Ventures with Eniac Ventures, Uncorrelated Ventures, Innovation Endeavors, Betaworks, Overtime.vc, Slack Fund, and other amazing investors ([TechCrunch](https://techcrunch.com/2020/09/29/cube-js-raises-6-5m-for-its-open-source-data-platform/)).

This round marks a new chapter in the story of Cube.js, which started back in 2019 and will continue to evolve for years to come. In this post, I want to talk about what it means for Cube.js and its fantastic community.

## How We Got Here

Pavel and I open-sourced Cube.js in March 2019. Within months, thousands of developers picked it up and started building applications on top of Cube.js. All of them gave us feedback, spread the word, and participated in the [Cube.js community on Slack](https://slack.cube.dev).

Today, Cube.js has been deployed to more than 70,000 servers in companies worldwide. Businesses trust Cube.js to power mission-critical applications in industries such as healthcare, finance, telecom, software, ad tech, and more. It’s used as a data access layer for complex reporting systems, marketing, and sales tracking, financial budgeting, inventory demand planning, and other analytics software.

![weinrich-quote-3.png](https://media.graphcms.com/rBRp6ZUQdCkf5vweaWLk)

But what I'm most excited about is the fantastic community of developers using and contributing to Cube.js. We have more than 100 contributors on GitHub and thousands of community members, almost 2,000 in our Slack alone. Today, I’d like to take a moment and say thank you to everyone who has been helping us along the way: to our users, contributors, and community members. To all of you: thank you for placing your trust in us and believing in our mission.

Maintaining the growth of such magnitude and accommodating the needs of a diverse spectrum of users takes effort and resources. That's why we established Cube Dev, Inc. and raised an investment round.

## Why We Raised

In the early days, Pavel and I were able to provide advice and support to every new company starting development with Cube.js. Today, there are thousands of companies already running Cube.js in production and even more actively developing new applications. A team of two isn’t enough to support this growth anymore.

Neither is it enough to make the kinds of changes we’ve long wanted to implement, such as: bring our documentation and overall developer experience to a new level, make external pre-aggregations work across any type and volume of data, support real-time streaming platforms as data sources, and more.

**We’ve raised money to build Cube.js faster and keep up with the growth.** Today, we're already a team of 9 full-time engineers (and growing!) working on Cube.js and new complementary products and integrations at full speed.

On the frontend and UI layer, we're building more integrations with frameworks, UI kits, and charting libraries while keeping Cube.js visualization-agnostic. We also have big plans for the data management layer. We're working on a better way to build and maintain pre-aggregations, regardless of the underlying data sources. We're optimizing for low-latency querying of aggregated data, fast ingestion, and horizontal scalability.

Not only do we want to evolve Cube.js further as a product, but we also want to ensure it's available to all development teams. So, we need to establish a company to fund the development of Cube.js while keeping it open-source.

## What's Next for Cube Dev

At Cube Dev, our mission is to build infrastructure for developing modern analytical applications.

As developers ourselves, we believe that core developer tools must be open-source. That's why we're committed to releasing all the core features as open-source under Apache 2.0 license for the server code and MIT for the frontend.

While keeping Cube.js free and open-source, we are also working on a new commercial offering - Cube Cloud. Our recent learnings showed that even in the cloud era deploying highly reliable and available Cube.js applications is hard. So, we want to focus our commercial offering on helping developers to solve these problems. Cube Cloud will provide additional tools to develop, deploy, and scale Cube.js applications.

Cube Cloud is currently in closed beta, which will expand soon, and it includes a free tier for development and simple productions. So, if you haven't got the chance yet, [please sign up to try Cube Cloud out](https://cube.dev/cloud/).

## Thanks for Your Support

Thank you to every one of you whose invaluable contributions have helped us get where we are today. I’m excited to continue building Cube.js with you in the coming years!

As always, feel free to join [our Slack community](https://slack.cube.dev/) to share your feedback, ask questions about Cube.js, or just chat with us. See you there!

Also, if you feel the passion for solving hard data problems and want to shape the next generation of analytical applications, we want to hear from you! We're hiring across multiple roles. [Check out our careers page for more details.](https://cube.dev/careers)

Cube Dev raises $6.2M to accelerate Cube development

For the last couple of years, we’ve been working on [Cube.js](https://github.com/cube-js/cube.js), an analytics framework built specifically for **customization and embedding**. There are a lot of great tools data engineers can use to build internal data infrastructure. But there is a lack of tools for software engineers who are building production, customer-facing applications and need to embed analytics features into these applications.

The major requirements we had were to make it easily scale to large datasets while allowing full UI customization on the frontend. It also had to be dependencies free and especially not require wiring up a complex infrastructure of data pipeline jobs. 

It has been in production in multiple companies for more than a year already with petabyte-sized datasets. Feeling confident that Cube.js is doing its job, we open sourced it 4 months ago for a broader audience to use. In this article, I want to share a detailed description of Cube.js’s architecture and why we designed it this way. After reading it, I hope you try out Cube.js for your own analytics applications!

The schema below shows how Cube.js is typically deployed and embedded into the existing application architecture. The Cube.js backend microservice is connected to one or multiple databases, taking care of database queues, data schema, caching, security, and API gateway. The client loads aggregated data from the backend, processes it, and sends it to the visualization library of your choice.

![cubejs-design-1.png](https://media.graphcms.com/ULBZdqRRCegNesWEm1x6)

Below I’ll put a detailed description of what exactly happens in both the backend and the client, but first I’d like to highlight the most important decisions we made when designing Cube.js.

## Data Schema

ORMs are quite ubiquitous in software development, but when it comes to analytics, it usually ends up with tons of SQL scripts and templates. The idea behind data schema is to take the best practices of ORM and apply them to analytics use cases. We’ve grabbed an good-old idea of multidimensional analysis with measures and dimensions as abstraction entities and de-facto created a [ROLAP](https://en.wikipedia.org/wiki/Online_analytical_processing#Relational_OLAP_(ROLAP)) (Relational OLAP) engine, which transforms measures and dimensions into SQL code. 

The biggest thing about the data schema is that we made it fully dynamic. A data schema in Cube.js is not an XML/JSON-like static thing but is a JavaScript code, which means you can dynamically generate it. You can even load your measures or dimension definitions from a database or over API during the runtime. 

Having the schema in place is vital to providing abstracted and flexible query language for API clients. No one wants to send SQL code or SQL snippet IDs over the API, so the query language is eventually developed in every case of such a system. That is why we made Cube.js come with it already and backed it by OLAP best practices.

![cubejs-design-2.png](https://media.graphcms.com/FXt3N9PXS7G5Y5AHak6i)

## In Database Pre-Aggregations

Although we have a usual last-mile in-memory cache, pre-aggregations make a big difference when it comes to performance, especially for dynamic queries. Cube.js can create reusable aggregate tables in your warehouse, which are extremely fast to read. One aggregate table usually serves multiple queries. Cube.js also takes care of refreshing the aggregate tables when new underlying data comes in. 

Cube.js can also automatically calculate and build the required aggregate tables based on the patterns of requested measures and dimensions. We use principles of the data cubes lattice for this and we’ll cover the math behind it in future blog posts. Additionally, you can store aggregate tables in a separate warehouse, e.g. MySQL, when all the raw data could be in BigQuery. That would give you a sub-second response, which is not possible even on small datasets with BigQuery due to its design. 

Pre-aggregations are essential for scaling. You can think about it as the “T” in the “ETL.” But the transformation happens inside your warehouse, fully orchestrated by Cube.js.

![cubejs-design-3.png](https://media.graphcms.com/TUQfUdKUSquHGUGQExi2)

## Visualizations Agnostic 

This one falls into the category “saying no to the feature is a feature itself.” There are tons of good and mature visualization libraries, so the best we could do is not to invent a new one, but to make sure we play nice with all of them. Although Cube.js doesn’t render anything, it provides a set of useful helpers to post-process data after it is loaded from the backend. It includes things like pivot operations and filling in missing dates. I’ll talk about it later in more detail when describing the components of the Cube.js frontend client.


In the rest of this post, I’ll cover the components of the backend and the frontend. 

## Cube.js Backend

The backend itself is a Node.js application, which can be configured via environment variables or by writing some Javascript code for more complex use cases. It also needs a data schema, which is a JavaScript code describing how your measures and dimensions should be mapped into SQL. The schema also contains rules for caching, security, and pre-aggregations. The backend is usually deployed as a microservice in your cluster. It has a connection to the required databases and exposes an API either externally directly to clients or internally if you have some proxy for clients. 

We’ve designed Cube.js as a modular framework with 4 main components. Usually, they are all used together, but you can use whatever you need in your specific use case. Below I’ll outline each component of the backend and what problems it solves.

### Schema Compiler
_[Schema Compiler on Github](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-schema-compiler)_

Schema Compiler compiles a data schema, which is a JavaScript code and based on it and the incoming query generates a SQL code. The SQL code is then sent to Query Orchestrator to be executed against the database. The data schema allows the creation of well-organized and reusable data models. Since it is JavaScript, you can dynamically create all required definitions, extract common pieces into helpers, and in general apply the best engineering practices to organize the data according to business definitions.

A schema is also a place where row-level security is defined. The user context can be passed with every request to Cube.js and propagated to the schema level. In the schema, you can use the user context to restrict access for specific users only to specific data.

Pre-aggregations, a widely used Cube.js feature for big data sets, are also defined in the schema. In that case, Schema Compiler generates not a single query, but a list of dependent queries to build pre-aggregations first and then the final query to fetch the data.


### Query Orchestrator

_[Query Orchestrator on Github](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-query-orchestrator)_

Query Orchestrator’s job is to ensure that the databases are not overloaded and that multi-stage queries are executed and refreshed in the correct order. To do that, it maintains query execution queues for pre-aggregations and data queries. The queues are idempotent, meaning that if multiple identical queries come in, only one will run against the database. The queries are executed by database drivers. As of today, Cube.js supports [more than ten native database drivers](https://cube.dev/docs/connecting-to-the-database) and a generic JDBC driver.

For multi-stage queries, which is always the case when using pre-aggregations, the query itself consists of multiple pre-aggregations and the final query to fetch the data. Orchestrator makes sure that all the required aggregate tables are fresh and exist before the final query. If the aggregate table does not exist or is outdated, it will schedule a query to create or update it.

![cubejs-design-4.png](https://media.graphcms.com/vsx9XzQp6KkA5Z6u2sVQ)

The background refresh of the aggregate tables is abstracted as the async process and could run either inside the standalone application or in the serverless mode. It also optionally allows you to extract the background processing into multi-tenant clusters.

### API Gateway
_[API Gateway on Github](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-api-gateway)_

API Gateway defines API endpoints to execute queries, load the metadata, and to inspect SQL generated by the schema compiler. The Gateway implements a long-polling idempotent API to load query results. It makes it tolerant of connectivity issues and guarantees the response without request time frame limitations.

The Gateway is responsible for authentication and authorization as well. By default, the security in Cube.js is implemented with JWT tokens. Every request is signed with a JWT token optionally containing information about security context to be passed to the data schema. The default security model can be overridden with custom middleware if you want to load your security context from a database or specific microservice.

### Server Core
_[Server Core on Github](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-server-core)_


Server Core wires all the above components together and exposes a single configuration entry point.

Server Core can be embedded into existing Node.js applications. To launch Cube.js as a standalone application, you need to use the server package; for serverless mode—the serverless package.

## Cube.js Client

_[Cube.js Client Core on Github](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-client-core)_

The Cube.js client is a JavaScript library that works with Cube.js API and post-processes query results. The Core client is a vanilla JavaScript library, which you can run in a browser or on a mobile device if you are using tools like React Native. Additionally, we ship packages for [React](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-react), [Angular](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-client-ngx), and [Vue.js](https://github.com/cube-js/cube.js/tree/master/packages/cubejs-client-vue) to make it easy to integrate the Cube.js client into these frameworks.

The Cube.js client abstracts the transport layer of loading data from the Cube.js backend and handles loading and error states as well. Once the data is loaded, the client provides a set of helper methods to post-process data. The client provides a pivot interface for displaying data in charts or tables; it also has methods for metadata manipulation and some utilities, like filling in missing dates.

We intentionally left out the visualization part. We had this idea from day one—to build a framework that takes care of everything except visualizations. It seems to be the best combination of power and customization for end users.

## Ecosystem

Those were the core components of both the Cube.js backend and frontend. But as a community evolving around the framework, I’m happy to see how the ecosystem grows. The ecosystem of integrations, tutorials, and examples is as important as the tool itself. I’m so grateful for all of the contributions we’ve had from our amazing open-source community already—database drivers, frontend frameworks’ bindings, tutorials, and code samples. 

If you are interested in Cube.js and want to contribute—I’d love to see you in our [Slack community](https://slack.cube.dev/). It is an ideal place to get started with Cube.js in general and start contributing as well.