For many years already, we live in the data-driven world where accessing the data and deriving insights from it efficiently is paramount. This year, we experience an explosion of interest to artificial intelligence (AI) and large language models (LLMs) in particular, fueled by the latest developments in the technology and vast perspectives unfolding.

[LangChain](https://www.langchain.com), an extensive toolkit for working with LLMs, has become one of the most important building blocks for developers of AI experiences. At Cube, we know that our semantic layer is also an important building block for AI applications because Cube not only centralizes metrics calculations but also serves an [antidote to AI hallucinations](https://cube.dev/blog/semantic-layer-the-backbone-of-ai-powered-data-experiences).

## Semantic document loader

Today, we're happy to present Cube's integration with LangChain. It comes as the [document loader](https://python.langchain.com/docs/integrations/document_loaders/cube_semantic) that is intended to be used to populate a vector database with embeddings derived from the data model of the semantic layer. Later, this vector database can be queried to find best-matching entities of the semantic layer. This is useful to match free-form input, e.g., queries in a natural language, with the views and their members in the data model.

![diagram](https://ucarecdn.com/32e98c8b-a920-4620-a8d2-05d57618db8e/)

We're also providing an chat-based demo application (see [source code](https://github.com/cube-js/cube/tree/master/examples/langchain) on GitHub) with example [OpenAI prompts](https://github.com/cube-js/cube/blob/master/examples/langchain/utils.py#L48,L87) for constructing queries to Cube's [SQL API](https://cube.dev/docs/product/apis-integrations/sql-api). If you wish to create an [AI-powered conversational interface](https://cube.dev/blog/conversational-interface-for-semantic-layer) for the semantic layer, functioning similar to [Delphi](https://www.delphihq.com), these prompts can be a good starting point.

## Chat-based demo app, dissected

Here's what you can build with the all-new document loader for LangChain, a vector database, an LLM by [OpenAI](https://openai.com), and [Streamlit](https://streamlit.io) for the user interface:

<video width="100%" controls><source src="https://ucarecdn.com/5500be7a-ab78-4de8-abc9-02df575ad3e0/video.mp4" type="video/mp4"/></video>

See how the tables, columns, aggregations, and filters in the SQL query generated by an LLM match the human input. Check the [README file on GitHub](https://github.com/cube-js/cube/blob/master/examples/langchain/README.md) for pointers on running this demo application on your machine or skim through the following highlights.

**Ingesting metadata from Cube and populating the vector database.** The `ingest_cube_meta` function in the `ingest.py` file loads the data model from Cube using the all-new `CubeSemanticLoader`. Note that only [views](https://cube.dev/docs/product/data-modeling/reference/view) are loaded as they are considered the ["facade" of the data model](https://cube.dev/blog/complementing-data-graph-with-views). Loaded documents are then embedded and saved in the FAISS vector store, which is subsequently pickled for later use.

```python
def ingest_cube_meta():
    ...
    loader = CubeSemanticLoader(api_url, api_token)
    documents = loader.load()
    ...
    with open("vectorstore.pkl", "wb") as f:
        pickle.dump(vectorstore, f)
```

**LLM setup.** In the `main.py` file, dependencies are imported and the environment variables are loaded. The OpenAI model (`llm`) is initialized with the provided API key.

```python
import ...
load_dotenv()
llm = OpenAI(
    temperature=0, 
    openai_api_key=os.environ.get("OPENAI_API_KEY"),
    verbose=True
)
```

**User input and vector store initialization.** In the same file, Streamlit primitives are utilized to get user input:

```python
question = st.text_input(
    "Your question: ",
    placeholder="Ask me anything ...",
    key="input"
)
if st.button("Submit", type="primary"):
    check_input(question)
    vectorstore = init_vectorstore()
```

**Querying the vector store.** The vector store is queried for documents similar to the user's question. The best match's table name is extracted and taken as the best guess to try to create a prompt using the columns from vectorstore:

```python
docs = vectorstore.similarity_search(question)
    # take the first document as the best guess
    table_name = docs[0].metadata["table_name"]

    # Columns
    columns_question = "All available columns"
    column_docs = vectorstore.similarity_search(
        columns_question,
        filter=dict(table_name=table_name),
        k=15
    )
```

**Building the prompt and calling OpenAI.*** The OpenAI large language model is called with the constructed prompt, and the response is parsed to extract the SQL query and any associated filters:

```python
# Construct the prompt
prompt = CUBE_SQL_API_PROMPT.format(
    input_question=question,
    table_info=table_name,
    columns_info=columns,
    top_k=1000,
    no_answer_text=_NO_ANSWER_TEXT,
)
llm_answer = llm(prompt + PROMPT_POSTFIX)
```

![](https://ucarecdn.com/1252fc98-75db-4887-be82-227b60d477a9/)

## Wrapping up

Cube's integration with LangChain provides a seamless interface for querying data using natural language. By abstracting the complexities of SQL and leveraging the power of LLMs, it provides the builders of AI experiences with a user-friendly and error-prone approach to data access.

<Quote authorName="Josh Patterson" authorAvatar="https://media.licdn.com/dms/image/C4E03AQEdLbPJvBTgBg/profile-displayphoto-shrink_800_800/0/1517680131555?e=1697673600&v=beta&t=CVQuMtVDphwPh1WwLOb_OJ0qBHZqHEJmDCcM6GJIYzo" authorRole="CEO at Patterson Consulting">

It is essential for enterprises to leverage internal knowledge correctly with the power of LLM’s reasoning. Cube’s semantic layer integration with LangChain is a great example of how most product companies will eventually write smart integrations for LLMs to better power these reasoning engines.

</Quote>

Our partners at Patterson Consulting have been hard at work building with Cube, LangChain and LLMs. They have written a wonderful ebook as a primer to [Building Enterprise Applications with LLMs](https://pattersonconsultingtn.com/content/intro_to_llms_ebook.html).

**You can try the LangChain integration with a Cube Core or [Cube Cloud deployment](https://cubecloud.dev/auth/signup) today.** Join our Slack community at [slack.cube.dev](https://slack.cube.dev) to share your thoughts and opinions about this update.

Mikhail Nitsenko

Artificial Intelligence

Powerful tooling for building AI experiences with LLMs and the semantic layer

Introducing the LangChain integration

## The AI-powered data experience

We’ve long seen that the data experiences landscape has been very diverse. From dozens of general-purpose BIs to hundreds of vertical purpose-built analytical solutions, data consumers have many, many options to visualize and consume data.

The evolution of front-end technologies made it possible to embed quality analytics experiences directly into many software products, further accelerating the proliferation of data products and experiences. 

And now, with the arrival of large language models, we are living through another step change in technology that will enable many new features and even in the advent of an entirely new class of products across multiple use cases and domains—_including_ data. 

LLMs are taking the data consumption layer to the next level with **AI-powered data experiences** ranging from chatbots answering questions about your business data to AI agents making actions based on the signals and anomalies in data. 

![](https://ucarecdn.com/5c8bd295-4317-4820-beea-532f8c9ba391/)

## Semantic layer gives context to LLMs

LLMs are indeed a step change, but inevitably, as with every technology, it comes with its own limitations. **[LLMs hallucinate](https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)); the [garbage in, garbage out](https://en.wikipedia.org/wiki/Garbage_in,_garbage_out) problem has never been more of a problem.** Let’s think about it like this: when it’s hard for humans to comprehend inconsistent and disorganized data, LLM will simply compound on that confusion to produce wrong answers.  

We can’t feed LLM with database schema and expect it to generate the correct SQL. To operate correctly and execute trustworthy actions, __it needs to have enough context and semantics__ about the data it consumes; it must understand the metrics, dimensions, entities, and relational aspects of the data by which it's powered. Basically—LLM needs a semantic layer.

The semantic layer organizes data into meaningful business definitions and then allows for querying these definitions—rather than querying the database directly. 

The ‘querying’ application is equally important as that of ‘definitions’ because it enforces LLM to query data through the semantic layer, ensuring the correctness of the queries and returned data. With that, the semantic layer becomes a solution for the LLM hallucination problem.

![](https://ucarecdn.com/c79d8e5a-ad67-43b3-ac4f-b3d323c0fb65/)

Moreover, the combination of LLMs and semantic layers can enable a new generation of AI-powered data experiences. At Cube, we’ve already witnessed many organizations build custom in-house LLM-powered applications, and startups, like [Delphi](https://cube.dev/blog/conversational-interface-for-semantic-layer), build out-of-the-box solutions on top of Cube’s semantic layer ([demo here](https://cube.dev/events/unleash-the-power-of-ai)). 

On the edge of this developmental forefront, we see Cube being an integral part of the modern AI tech stack as it sits on top of data warehouses, providing context to AI agents and acting as an interface to query data.

![](https://ucarecdn.com/ea4d11af-b3f9-43b9-b02e-b8fa8056c4b1/)

Cube’s data model provides structure and definitions that are used as a context for LLM to understand data and generate correct queries. LLM doesn’t need to navigate complex joins and metrics calculations because Cube abstracts those and provides a simple interface that operates on the business-level terminology, instead of SQL table and column names. This simplification helps LLM to be less error-prone and avoid hallucinations. 

For example, an AI-based application would first read Cube’s meta API endpoint, downloading all the definitions of the semantic layer and storing them as embeddings in a vector database. Later, when a user sends a query, these embeddings would be used in the prompt to LLM to provide additional context. LLM would then respond with a generated query to Cube, and the application would execute it. This process can be chained and repeated multiple times to answer complicated questions or create summary reports. 

### Performance

In regards to response times—when working on complicated queries and tasks, the AI system may need to query the semantic layer multiple times, applying different filters. 

So, to ensure reasonable performance, these queries must be cached and not always pushed down to the underlying data warehouses. Cube provides a relational cache engine to build pre-aggregations on top of raw data and implements aggregate awareness to route queries to these aggregates when possible.

### Security

And, finally, security and access control should never be an afterthought when building AI-based applications. As we mentioned above, generating raw SQL and executing it in a data warehouse may lead to wrong results. 

However, AI poses an additional risk here: since it cannot be controlled and may generate arbitrary SQL, direct access between AI and raw data stores can also be a significant security vulnerability. Instead, generating SQL through the semantic layer can ensure granular access control policies are in place. 

### And more...

We have a lot of exciting integrations with the AI ecosystem in store and can’t wait to share them with you. Meanwhile, if you are working on an AI-powered application with Cube – [let’s talk](https://cube.dev/contact)! 


Artyom Keydunov

—and the antidote to AI Hallucinations.

Semantic Layer: The Backbone of AI-powered Data Experiences

Today, we’re happy to present the newest addition to the plethora of [data tools](https://cube.dev/integrations) that work with Cube, the semantic layer for building data apps. [Delphi](https://www.delphihq.com) provides a way to interact with Cube in natural language and delivers insights straight to business users in Slack. 

![Diagram: Cube and Delphi](https://ucarecdn.com/ed0d51c4-2ff7-40cc-891f-32badcc57a51/)

Powered by the newest large language model (LLM) technology from OpenAI, Delphi brings the accessibility of self-service analytics to the next level and enables a compelling use case for semantic layers. See Cube and Delphi in action:

<iframe width="790" height="447" src="https://www.youtube.com/embed/FotEaaf20gY" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen allow="fullscreen;"></iframe>

Learn more about Delphi and [semantic superiority](https://davidsj.substack.com/p/semantic-superiority-part-4) from David Jayatillake, co-founder and CEO at Delphi Labs:

## Natural language interface for data

Our mission for Delphi is to solve the whole workflow, from question to answer, with data. Today, this is broken—self-serve BI tools haven’t delivered on this promise to date, and data teams are bombarded with ad hoc requests at a scale greater than ever before as businesses move to operate with data.

When we ask data leaders in the space what % of their organization is capable of self-serving with data, the answer is usually between 10 to 25%, with only 10% being very competent. It’s been three years since “dashboards are dead,” but unfortunately they’re still being used as the primary way to serve data. In my experience, no stakeholder ever wanted a dashboard. They wanted an answer, an explanation, a prediction, data to use elsewhere operationally, a graph or a slide to make a specific point.
This is the problem space we aim to solve with Delphi.

There are many other entrants who are trying to offer a natural language interface to data in this LLM gold rush. However, almost all of them are pointing the LLM at a database schema and getting it to generate SQL. We believe this is the wrong approach: every time a query is answered using this method, the LLM generates a minimal semantic layer on the fly in order to derive meaning from the data. As humans created the data structures and their meaning in the first place, why introduce a probabilistic element into the meaning of your data?

We believe that working with semantic layers curated by data folks is the best approach. It synthesizes the information already baked into the semantic layer, truly enabling safe and accurate answers. It also allows us to say that we can’t answer the question—a very normal outcome when asking a data team a question; LLMs connected directly to a data store will simply guess, as they don’t have a concept of a world and what does, and doesn’t, exist in it, which a semantic layer offers.

This is why we’re so excited to work with semantic layers like Cube—we’re already blown away by the interest we’ve had from the Cube community and hope to continue to work with you all!

As I’ve written in my [substack](https://davidsj.substack.com/archive), I believe the future of semantic layers is standalone. It’s what the people want.

## What’s next?

Here's what an early adopter of Delphi and Cube thinks:

> Getting started with Delphi on Cube was a breeze and we really quickly had a conversational AI interface to our data warehouse. The accuracy was great and some of the Delphi features really helped explain our data and ensure we knew how we were querying it. I'd highly recommend it for it's wow factor and enabling moving towards a data driven culture.

Interested? Implement your [semantic layer](https://cube.dev/use-cases/semantic-layer) with Cube by [getting started with Cube Cloud](https://cubecloud.dev/auth/signup) today or by [contacting us](https://cube.dev/contact). Also, [sign up for Delphi](https://docs.google.com/forms/d/e/1FAIpQLSc7nJc1Zeu5kEHcKrghbTPy3MSxJLOBsge0kj4Iiz5aqX6zJg/viewform) and see it in action at [slack.cube.dev](https://slack.cube.dev) in the `#random-delphi-demo` channel:

![Delphi in Slack](https://ucarecdn.com/ddf30e1e-c179-4fda-8aa8-2f75a05f808e/)


Artificial Intelligence

Introducing the LangChain integration

Semantic Layer: The Backbone of AI-powered Data Experiences

AI-powered conversational interface for the semantic layer

Get Cube updates to your inbox

Upgrade your data stack today