When integrating AI/LLM functionality into applications, interacting with different platforms can be beneficial. One simple way to achieve this is by making API calls to the platforms. By sending queries and processing the returned results within the app, functionality can be greatly extended.
The following code snippet demonstrates an API call to ChatGPT-4 using the openai package in Python. The code is available in the file APICall.py.Â
When network access is limited - whether due to connectivity issues or data privacy concerns - it can be useful to run an LLM locally. Hugging Face provides access to a variety of models, including TinyLlama, a compact model with 1.1 billion parameters.
Thanks to its small size, TinyLlama is quick to install and can run efficiently on regular laptops and desktop computers.
A code snippet for running an LLM locally can be found here: ReadMe.txt and tinyllamaQuery.py.
Note: The model is downloaded the first time the program runs. After that, it can be used offline without an internet connection - just unplug and run.
Using contexts can improve LLM queries. By finding an appropriate context to the query that is to be posted to a LLM, the quality of the answers can be improved. The files in this projects tries this out in detail. First, a two documents, the October 2024 GFSR of the IMF and the Financial Stability Report 2024:2 of the Riksbank, are used to extract text. This text is then split into smaller pieces and the query that is to be sent to the LLM is matched against these smaller pieces. The best-fitting pieces are then passed as context, together with the query, to two different LLMs. The first LLM is GPT2, which is locally run. The other LLM is ChatGPT4 which is accessed through an OpenAI API call. The files that are used are described below.
RAGQuery.py: The main file for the RAG-based query.
TextUtilities.py: A class (PDFReader) for reading PDF, cleaning, and converting into text. Also methods for splitting texts into chunks are found in the file.
Contextualize.py: Method for embedding text chunks frpm background text and query and find the closest matches to construct the context that will be passed together with the query.
GPT2Query.py: Method for querying GPT2.
ChatGPT4Query.py: Method for querying ChatGPT4 through OpenAIs API. An API key/token is needed to access the model.