JoinDiscord

 

llamaindex: Build LLM applications with LlamaIndex and monitor the data with Argilla.

This integration allows the user to include the feedback loop that Argilla offers into the LlamaIndex ecosystem. It’s based on a callback handler to be run within the LlamaIndex workflow.

LlamaIndex is a specialized data framework tailored for supporting LLM-driven application development. It provides a sophisticated structure enabling developers to seamlessly integrate various data sources with large language models. These sources encompass diverse file formats like PDFs and PowerPoints, popular applications such as Notion and Slack, and databases like Postgres and MongoDB. Through a range of connectors, the framework streamlines data ingestion, facilitating smooth interactions with LLMs. Additionally, LlamaIndex offers an efficient interface for data retrieval and queries. This functionality allows developers to input LLM prompts and receive context-rich, knowledge-enhanced outputs.

In essence, LlamaIndex acts as an intermediary for managing interactions with an LLM by constructing an index from input data. This index is then leveraged to answer queries related to the provided data. LlamaIndex is flexible, capable of generating different types of indexes—vector, tree, list, or keyword indexes—tailored to specific requirements.

LlamaIndex offers a wide array of tools that facilitate the processes of data ingestion, retrievals, structuring, and integration with diverse application frameworks. Don’t hesitate to check out both LlamaIndex and Argilla

Running Argilla

For this tutorial, you will need to have an Argilla server running. There are two main options for deploying and running Argilla:

Deploy Argilla on Hugging Face Spaces: If you want to run tutorials with external notebooks (e.g., Google Colab) and you have an account on Hugging Face, you can deploy Argilla on Spaces with a few clicks:

For details about configuring your deployment, check the official Hugging Face Hub guide.

Launch Argilla using Argilla’s quickstart Docker image: This is the recommended option if you want Argilla running on your local machine. Note that this option will only let you run the tutorial locally and not with an external notebook service.

For more information on deployment options, please check the Deployment section of the documentation.

Tip

This tutorial is a Jupyter Notebook. There are two options to run it:

• Use the Open in Colab button at the top of this page. This option allows you to run the notebook directly on Google Colab. Don’t forget to change the runtime type to GPU for faster model training and inference.

• Download the .ipynb file by clicking on the View source link at the top of the page. This option allows you to download the notebook and run it on your local machine or on a Jupyter notebook tool of your choice.

Getting started

You first need to install argilla and argilla-llama-index as follows:

%pip install argilla-llama-index

Usage

It requires just a simple step to log your data into Argilla within your LlamaIndex workflow. We just need to call the handler before starting production with your LLM. We will use GPT3.5 from OpenAI as our LLM. For this, you will need a valid API key from OpenAI. You can have more info and get one via this link. After you get your API key, the easiest way to import it is through an environment variable, or via getpass().

import os

openai_api_key = os.getenv("OPENAI_API_KEY")

Let’s now write all the necessary imports and initialize the Argilla client.

from llama_index.core import VectorStoreIndex, ServiceContext, SimpleDirectoryReader, set_global_handler
from llama_index.llms.openai import OpenAI

import argilla as rg

rg.init(
    api_url="http://localhost:6900",
    api_key="owner.apikey",
    workspace="admin"
)

Now, we will set up an Argilla global handler for Llama Index. By doing so, we ensure that the predictions that we obtain using Llama Index is automatically uploaded to the Argilla client we initialized before Within the handler, we need to provide the dataset name that we will use. If the dataset does not exist, it will be created with the given name. You can also set the API KEY, API URL, and the workspace name. You can learn more about the variables that controls Argilla initialization here.

set_global_handler("argilla", dataset_name="query_model")

Let’s now create the llm instance, using GPT-3.5 from OpenAI.

llm = OpenAI(model="gpt-3.5-turbo", temperature=0.8, openai_api_key=openai_api_key)

With the code snippet below, you can create a basic workflow with LlamaIndex. You will also need a txt file as the data source within a folder data. We have an example .txt file ready for you inside that folder, obtained from the Llama Index documentation.

service_context = ServiceContext.from_defaults(llm=llm)
docs = SimpleDirectoryReader("../../data").load_data()
index = VectorStoreIndex.from_documents(docs, service_context=service_context)
query_engine = index.as_query_engine()

Now, let’s run the query_engine to have a response from the model.

response = query_engine.query("What did the author do growing up?")
response

As the global handler was set, an Argilla dataset is created, with the dataset name that we introduced as parameter. Everything predicted using the query function will be automatically logged in this dataset, with information about the steps to produce the prediction. The prompt given and the response are logged, as you can see on this example of Argilla’s UI: