Understanding the Backend of Mindbreeze InSpire AI Chat

Let's take a closer look at what happens in the backend of Mindbreeze InSpire when we ask questions in the AI Chat.

Example Search Query

To demonstrate, we search for "What are the deployment options of Mindbreeze InSpire?"  in our chat.

Exploring the Process in the Mindbreeze Management Center

Now, let's switch over to the Mindbreeze Management Center to explore the process further.

First, we open App.Telemetry by navigating to Reporting → Telemetry Details. I prefer opening App.Telemetry in a separate tab.

Mindbreeze Insight Services Section

The key section to focus on is the Mindbreeze Insight Services section. This section contains all the logpools for the configured Mindbreeze Insight Services within our Mindbreeze InSpire setup.

For our use case, we need to analyze the Insight Services LLM logpool as the RAG utilizes large language models (LLMs).

Accessing and Analyzing Logs

Let's open the Insight Services LLM logpool. To access the logpool, we click View Telemetry Data, which allows us to analyze the details efficiently.

Next, we select the latest logpool entry representing our most recent request in the chat.

We then preview the details of the selected entry by clicking on the eyeball icon.

Key Information in Log Details

In the details, we can see key information such as:

• Request start time
• Duration
• Service in use
• Model being utilized
• API user
• URL of the large language model

Additionally, we can see the number of generated tokens and the time it took for the first token to be produced.

Critical Log Entries

Our most critical logs are:

• User Prompt: The question or request entered into the chat.
• LLM Prompt: The final prompt sent to the large language model (LLM), enriched with relevant data retrieved through the retrieval augmented generation pipeline (RAG).
• Generated Text: The response returned to the user.

For our example, the User Prompt is: "What are the deployment options for Mindbreeze InSpire?"

By understanding these logs, we gain valuable insights into how Mindbreeze InSpire processes and generates AI-powered responses.