
Based in Malaysia, [FeedMe](https://feedmepos.com/) was founded in 2019 and provides a Point of Sale (POS) system for the food and beverage industry across South East Asia. In addition to POS equipment that you see in restaurants, FeedMe provides reporting to their customers. The reporting ranges from sales reports, marketing analytics, inventory management, and others.  

FeedMe wanted to provide a robust reporting system that allowed users to generate their own custom reports. However, their previous solution that was based on SQL queries and JavaScript to process data across MongoDB, CouchDB, and BigQuery was becoming increasingly difficult to maintain and scale. FeedMe uses CouchDB in their client POS systems that can work either online or offline. CouchDB data is then fed into the BigQuery data lake for reporting.  FeedMe’s backend systems are written in JavaScript, so they wanted to use JavaScript to process data from MongoDB and BigQuery.  

## Taking a phased approach to deploy Cube.js

The FeedMe team found Cube.js through Google searches as they were looking for a solution that can help them develop a data schema standard.  They soon learned that Cube.js could help simplify their data processing and decided to take a phased approach to deploy Cube.js. 

In the first phase, they only focused on migrating part of their data schema using Cube.js’ [Dynamic Schema Creation](https://cube.dev/docs/schema/dynamic-schema-creation#top). They found this process to be relatively straightforward and spent about two weeks to complete this first phase.

In the next phase, the team spent another three weeks migrating the rest of their data schema and then implementing more advanced features such as [pre-aggregations](https://cube.dev/docs/schema/reference/pre-aggregations) and [Cube Store](https://cube.dev/docs/deployment/overview#cube-store). The FeedMe team liked the more flexible caching mechanism in Cube.js since they can customize [refreshKey](https://cube.dev/docs/schema/reference/pre-aggregations#refresh-key) and [rollup](https://cube.dev/docs/schema/reference/pre-aggregations#parameters-type-rollup) definitions. With Cube.js pre-aggregations and Cube Store, FeedMe’s BigQuery cost is down by about ⅔. (Before Cube.js, they could only rely on BigQuery’s built-in cache.)

Also, without pre-aggregations, it could take several minutes for their customers to query several months of sales data which wasn’t ideal.  When the data is cached and pre-aggregated, a similar query would take less than 10 seconds. 

## How FeedMe is using Cube.js

At FeedMe, Cube.js is deployed via Docker along with Kubernetes, and they use [Chart.js](https://awesome.cube.dev/tools/chartjs) for visualizations. You will find more details in the architectural diagram below.

![Architecture Diagram Screenshot](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/cd35818c-31b5-4fcf-9e69-866de2f670b1.png)

Cube.js has been in production at FeedMe since May 2021, and below is a sample screenshot of a sales dashboard. 

![Sales Dashboard Screenshot](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/469f653f-9dcf-432c-b628-54adbf313764.png)

The following screenshot shows a dashboard called the daily Close Up summary.

![Close Up Summary Screenshot](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/cdfbbc45-fdaf-466d-aa39-e27314be13f1.png)

## Working with the Cube.js community

The FeedMe team used the [Cube.js documentation](https://cube.dev/docs/) as an onboarding tool to get familiar with Cube.js and its features in the early days. They also liked the collaborative nature of the Cube.js community, especially on [Slack](https://slack.cube.dev/), as they were navigating different features such as [rollupJoin](https://cube.dev/docs/schema/reference/pre-aggregations#rollup-join), pre-aggregations, and Cube Store. 

They also provided feedback to the Cube Dev team when they discovered an [issue](https://github.com/cube-js/cube.js/issues/3081) with pre-aggregations. Enhancements to the rollupJoin and pre-aggregations will further streamline how the FeedMe team provides custom reports for their users while maintaining their BigQuery costs. 


## Interested in joining FeedMe's success with Cube.js?

Explore Cube.js [examples & tutorials](https://cube.dev/docs/examples) and [get started](https://cube.dev/docs/getting-started) today. To jumpstart your efforts, please join us on [Discourse](https://forum.cube.dev/) & [Slack](https://slack.cube.dev/), follow us on [Twitter](https://twitter.com/thecubejs), and get engaged with the growing Cube.js community.


Ray Paik

Inspiring stories about companies building analytical solutions for end-users with Cube.

Company news and product release notes for prominent feature launches.

News & Product Updates

Community news, accomplishments, and programs.

Community Updates

Mastering Cube by digging into advanced concepts and dissecting core features.

Deep Dives

User Stories

Upcoming workshops and community calls with Cube creators and community members.

Events

Company news, product updates, and engineering deep dives.

All Things Cube

Serverless data warehouse of Google Cloud Platform.

Petabyte-scale data warehouse of Amazon Web Services.

Amazon Redshift

Google BigQuery

Cloud-agnostic data platform supporting AWS, GCP, and Azure.

Snowflake

Popular cloud-based scalable data stores optimized for BI and analytics.

Cloud Data Warehouses (CDW)

Source-available document-oriented data store.

MongoDB

Key-value and document-based data stores.

NoSQL & Document Databases

How FeedMe used Cube.js to redesign their reporting system

Artyom Keydunov

Declarative component-based JavaScript library originally created by Facebook.

React

Web framework for React web applications by the Vercel team providing SSR & SSG.

Next.js

Approachable, performant, and versatile JavaScript framework.

Fully extensible web developer's platform.

Angular

Popular front-end frameworks for developing modern web applications.

Front-end Frameworks

This guide shows how to build a full-stack real-time dashboard with only open-source tools—from the database to the visualizations. You’ll learn how to set up a database, seed it with data, build an API endpoint on top of it, and then load and update charts on the frontend via WebSockets in real time.

Real-Time Dashboard with Open-Source Tools

For a pretty decent amount of time, doing analytics with MongoDB required additional overhead compared to modern SQL RDBMS and Data Warehouses associated with aggregation pipeline and MapReduce practices. While this approach allowed one to craft advanced custom tailored aggregation algorithms it required additional knowledge of how to build and maintain it. 

To fill this gap, MongoDB released the MongoDB connector for BI, which acts as a MySQL server on top of your MongoDB data. Under the hood it bridges existing aggregation mechanisms to the MySQL protocol, allowing standard MySQL clients to connect and issue SQL queries.

In this short 30-minutes tutorial we’ll setup the MongoDB connector for your local MongoDB instance and spin up a Node.js application that provides an analytics API, query cache, and orchestration using the Cube.js analytics framework. Cube.js can be used as a standalone server or it can be embedded as part of an existing Node.js web application. You can learn more about it [here](https://github.com/cube-js/cube.js).

## Setting up the MongoDB connector for BI
To install the MongoDB connector for BI locally you can use either [quickstart guide](https://docs.mongodb.com/bi-connector/master/local-quickstart/) or one of the [platform dependent installation guides](https://docs.mongodb.com/bi-connector/master/installation/).

Please make sure you use the MongoDB version that supports the MongoDB connector for BI. In this tutorial we use 4.0.5.

If you don’t have a local MongoDB instance please download it [here](https://www.mongodb.com/download-center/community).
The BI Connector can be downloaded [here](https://www.mongodb.com/download-center/bi-connector).

After the BI connector has been installed please start a `mongod` instance first. If you use the downloaded installation it can be started from its home directory like so:

```
$ bin/mongod
```

The BI connector itself can be started the same way:

```
$ bin/mongosqld
```

Please note that `mongosqld` resides in another `bin` directory. 
If everything works correctly, you should see a success log message in your shell for the `mongosqld` process:

```
[initandlisten] waiting for connections at 127.0.0.1:3307
```

If you’re using the MongoDB Atlas you can use [this guide](https://docs.atlas.mongodb.com/bi-connection/#bi-connection) to enable BI connector.

## Importing test dataset

You can skip this step if you already have data in your DB that can be analyzed. Otherwise you can use the zip code test dataset from MongoDB to populate your DB with some test data.

Download [zips.json](http://media.mongodb.org/zips.json) and run `mongoimport` from the MongoDB home directory:
```
$ bin/mongoimport --db test --collection zips --file <path/to/zips.json>
```

Please make sure to restart the MongoDB BI connector instance in order to generate an up-to-date MySQL schema from the just added collection.

## Spinning up a Cube.js application

We’ll use Cube.js to make analytic queries to our MongoDB instance. To install its CLI run:

```
$ npm install -g cubejs-cli
```

To create a new Cube.js application with MongoBI driver run:

```
$ cubejs create mongo-tutorial -d mongobi
```

Go to the just created `mongo-tutorial` directory and edit the just created `.env` file: replace placeholders with your MongoDB BI connector credentials.

By default it should be something like: 

```
CUBEJS_DB_HOST=localhost
CUBEJS_DB_NAME=test
CUBEJS_DB_PORT=3307
CUBEJS_DB_TYPE=mongobi
CUBEJS_API_SECRET=941ed7ad8a49bec1b3f87a9f48bff2a5e549e946fc22fcc5f18c3a17bf62c64ed3398e99b271cd66d4521a6fd7caa4bfd268dfccea59ffd9c67de660f3967338
```

Now generate Cube.js schema files for the `zips` collection from the test data set or for your own collection:

```
$ cd mongo-tutorial
$ cubejs generate -t zips
```

In order to start a Cube.js dev server you’ll also need a locally running Redis instance, which is used for cache and query queue orchestration. You can download it and run it using [these instructions](https://redis.io/download).

If everything went smoothly, you’re able to run the Cube.js dev server:

```
$ npm run dev
```

If the server started successfully, you can now open `http://localhost:4000` and navigate to the Cube.js dev environment. There you should see a working example of a pie chart.

## Building a Dashboard

First, let’s generate our app with Cube.js templates. Navigate to the Dashboard App tab and select “Create your Own” with React and Ant Design. 

![Create custom app](https://cubedev-blog-images.s3.us-east-2.amazonaws.com/c62e3426-3350-427a-b73f-6c6f6b897092.png)

The Cube.js dev environment example contains all essential client pieces to build an analytics dashboard. Let’s modify it a little bit so it looks more like a dashboard and uses the zips collection.

Replace the contents of `index.js` in your dev environment code sandbox:

```javascript
import React from "react";
import ReactDOM from "react-dom";

import "antd/dist/antd.css";
import { Row, Col, Card, Layout, Spin } from "antd";

import cubejs from "@cubejs-client/core";
import { QueryRenderer } from "@cubejs-client/react";
import { Chart, Axis, Tooltip, Geom, Coord, Legend } from "bizcharts";

const renderChart = resultSet => (
  <Chart height={400} data={resultSet.chartPivot()} forceFit>
    <Coord type="theta" radius={0.75} />
    <Axis name="Zips.count" />
    <Legend position="bottom" name="category" />
    <Tooltip showTitle={false} />
    <Geom type="intervalStack" position="Zips.count" color="x" />
  </Chart>
);

const cubejsApi = cubejs(
  "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1NDkzMDk3NzMsImV4cCI6MTU0OTM5NjE3M30.eXEdfUa_ek2V9MlGTpBMOd_AFfs8laaZj8ZsuM1wqqo",
  { apiUrl: "http://localhost:4000/cubejs-api/v1" }
);

const { Header, Footer, Sider, Content } = Layout;

const App = () => (
  <Layout>
    <Header>
      <h2 style={{ color: "#fff" }}>MongoDB Dashboard</h2>
    </Header>
    <Content style={{ padding: "25px", margin: "25px" }}>
      <Row type="flex" justify="space-around" align="top" gutter={24}>
        <Col span={24} lg={12}>
          <Card title="Zip count by state" style={{ marginBottom: "24px" }}>
            <QueryRenderer
              query={{ measures: ["Zips.count"], dimensions: ["Zips.state"] }}
              cubejsApi={cubejsApi}
              render={({ resultSet, error }) =>
               (resultSet && renderChart(resultSet)) ||
(error && error.toString()) || <Spin />
              }
              />
          </Card>
        </Col>
      </Row>
    </Content>
  </Layout>
);

const rootElement = document.getElementById("root");
ReactDOM.render(<App />, rootElement);

```

In the development environment, Cube.js doesn't enforce the use of the token to authorize queries, so you can use any string for your token here. You can learn more about using and generating tokens in the production environment [here in the docs](https://github.com/cube-js/cube.js#security).

If everything worked well, you should see the following dashboard:

![](https://media.graphcms.com/oORb3acEQ7ahiKaQ4onH)

## Why Cube.js?
So why is using Cube.js better than hitting SQL queries to MongoDB directly? Cube.js solves a plethora of different problems every production-ready analytic application needs to solve: analytic SQL generation, query results caching and execution orchestration, data pre-aggregation, security, API for query results fetch, and visualization. 

These features allow you to build production grade analytics applications that are able to handle thousands of concurrent users and billions of data points. They also allow you to do analytics on a production MongoDB read replica or even MongoDB main node due to their ability to reduce the amount of actual queries issued to a MongoDB instance. Cube.js schemas also allow you to model everything from simple counts to funnels and cohort retention analysis. You can learn more about it [here](https://statsbot.co/docs/getting-started-cubejs/).

## Performance considerations
In order to be able to handle a massive amount of data, Cube.js heavily relies on pre-aggregations. As of now, the MongoDB BI Connector doesn’t support Create Table as Statement, which is required to materialize query results right in your database and create pre-aggregations. If you need to analyze well over 100M of data points in MongoDB please consider using [Presto](https://prestodb.github.io/) with the [MongoDB Connector](https://prestodb.github.io/docs/current/connector/mongodb.html), which is also supported by Cube.js.

NoSQL & Document Databases

MongoDB

How FeedMe used Cube.js to redesign their reporting system

Real-Time Dashboard with Open-Source Tools

Building MongoDB Dashboard using Node.js