Cyndx tools serve some of the largest financial services companies worldwide to help them identify emerging companies, growth opportunities, and other investment opportunities. Along with their AI-driven algorithms, Cyndx uses data from over 12 million companies and more than 1 million acquisitions, capital raises, investments, and investors data in their database to provide predictive analytics and help its clients identify target lists in a fraction of the time of traditional workflows.
Recently, Cyndx decided to expand its product and develop functionality to explore the data analytics that would allow its end users to dig deeper than their existing platform. The Cyndx team evaluated several commercial business intelligence (BI) products in the past. Still, most of the solutions required a lot of custom work for frontend/design and integrating with their AI. After one of the Cyndx analysts discovered Cube.js, they put together a demo within a day that looked like the Cyndx platform.
How Cyndx is using Cube.js
On its frontend, Cyndx uses the React framework with movable/resizable components. They also use visualization components from Nivo, a data visualization library built on top of D3.js. An extended version of the Cube.js Developer Playground is used for creating the individual visualizations.
On the backend, the Cube.js API instance is hosted on a Google Kubernetes Engine (GKE) cluster. Cube.js is making calls to Cyndx BigQuery data warehouse. For performance, BigQuery caching has been key, but they also do database clustering and partition keys.
Further, they are leveraging a custom backend which is used to store permissions and custom dashboard configurations on a per user basis. The permissions gate access to the dashboards while the dashboard configurations are retrieved on page load to give a user a personalized experience.
Testing at Cyndx
To ensure the quality of the data being released, Cyndx runs numerous tests both on the frontend visualizations and on the backend data. Tools such as Datadog are used for browser testing to ensure functionality like drill-downs/links are working and to monitor long-running queries that may take too long.
Additionally, Cyndx also have around 1,000 tests on data itself using dbt before data gets promoted to production. The Cyndx team has also been active in the dbt community to help build the underlying data structure going into Cube.js and reducing the overhead on Cube.js by standardizing what tables look like.
Self-hosting and custom data visualizations
For Cyndx, it was essential to be able to manage their own security, and the Cyndx team liked the fact that they can self-host Cube.js rather than deal with third-party servers.
Other key benefits of Cube.js were the ability to integrate directly with the React framework and using Cyndx's own CSS so that new features such as plotting looked like the rest of the Cyndx platform. The screenshots below show the Cyndx platform (with mock data) where Cube.js is used.
Cyndx team and plans
To implement Cube.js, Cyndx had a dedicated frontend engineer and analyst as well as a part-time backend engineer and designer. Cube.js Developer Playground has been instrumental for the team and is on everyone's computer. The team was also able to get responsive community support from Cube.js Slack.
The Cyndx team successfully went live with Cube.js in October 2020, and they are looking forward to several improvements in Cube.js that can help enhance their platform. For example, they default to quarterly (vs. year-over-year) views at Cyndx, so a better way to address granularity in Cube.js would be beneficial. They would also like to see more custom functions such as "calculated fields" and enhancements in pulling in data from REST APIs (e.g., Salesforce API).
Future: Self-serve analytics
In the future, Cyndx is planning on rolling out self-serve analytics to its end-users. Clients have expressed a strong interest in digging deeper and customizing the analytics for their organizations. Many investors focus on specific regions and/or specific segments of the market, and they would like the analytics to reflect what is relevant for them. This type of customization is not possible without self-serve analytics as it is not possible to enumerate all of the use cases that Cyndx's end-users may be looking for.
This personalization of the analytics will be done through a multi-phase project in which Cyndx first extends the Cube.js Developer Playground so that it looks and feels like their web app. The ability to make such customizations was a crucial component in their selection of Cube.js for their backend modeling tool. Many other enterprise offerings for analytics will allow customizing the visuals, but the user interface for building out visuals was often not editable.
Through the use of the customized playground, the end-users will be able to develop dashboards customized to both the user's individual needs and jointly edit dashboards necessary for their organization's workflows. This type of permission logic is easily extensible on top of Cube.js backend and was an additional reason for selecting the tool.