PrivateGPT: The Most Powerful and Secure AI Language Model

PrivateGPT is a unique and revolutionary service that enables you to construct your own personalized GPT-3 models without requiring any coding or technical knowledge. In this essay, We will define PrivateGPT, explain how it works, and why you should use it for personal or corporate purposes.

What is PrivateGPT?

Ask questions to your documents without an internet connection, using the power of LLMs. 100% private, no data leaves your execution environment at any point. You can ingest documents and ask questions without an internet connection.

PrivateGPT is a term that refers to different products or solutions that use generative AI models, such as ChatGPT, in a way that protects the privacy of the users and their data. For example, PrivateGPT by Private AI is a tool that redacts sensitive information from user prompts before sending them to ChatGPT, and then restores the information in the answer. Another example is PrivateGPT by privategpt.io, which is a chatbot that connects to various sources of organizational data, such as Notion, JIRA, Slack, GitHub, etc., and provides answers to questions based on the knowledge from those sources. A third example is privateGPT.py by imartinez, which is a script that uses a local language model based on GPT4All-J to interact with documents stored in a local vector store. These are some of the ways that PrivateGPT can be used to leverage the power of generative AI while ensuring data privacy and security.

Environment Setup

To prepare your environment for running the code above, first install the following requirements:

pip install -r requirements.txt

Rename example.env to .env and edit the variables appropriately.

MODEL_TYPE: supports LlamaCpp or GPT4All
PERSIST_DIRECTORY: is the folder you want your vectorstore in
LLAMA_EMBEDDINGS_MODEL: Path to your LlamaCpp supported embeddings model
MODEL_PATH: Path to your GPT4All or LlamaCpp supported LLM
MODEL_N_CTX: Maximum token limit for both embeddings and LLM models

Then, download the two models and save them in the following directory (be sure to change your .env with the model paths):

• LLM: default to ggml-gpt4all-j-v1.3-groovy.bin. If you prefer a different GPT4All-J compatible model, just download it and reference it in your .env file.
• Embedding: default to ggml-model-q4_0.bin. If you prefer a different compatible Embeddings model, just download it and reference it in your .env file.

Test dataset

This repo uses a state of the union transcript as an example.

Instructions for ingesting your own Dataset

Put any and all of your .txt, .pdf, or .csv files into the source_documents directory.

Run the following command to ingest all the data.

python ingest.py

It will create a db folder in which the local vectorstore will be stored. Depending on the size of your papers, this will take some time. You can upload as many documents as you wish, and they will all be stored in the local embeddings database. If you wish to start from scratch, remove the db folder.

Please keep in mind that no data leaves your local environment throughout the ingest process. You can consume without an internet connection.

Locally, ask questions about your documents!

To ask a question, use the following command:

python privateGPT.py

And then wait for the script to ask for your feedback.

> Enter a query:

Hit enter. Wait 20-30 seconds (depending on your system) as the LLM model absorbs the prompt and prepares the response. It will then publish the response as well as the four sources it utilized as context from your documents; you can then ask another question without re-running the script; simply wait for the prompt again.

It’s worth noting that you could disable your internet connection and the script inference would still function. No data leaves your immediate surroundings.

To end the script, type exit.

How does it work?

Using the correct local models and the capability of LangChain, you can run the full pipeline locally, without allowing any data to leave your environment, and with reasonable speed.

• ingest.py parses the document and creates embeddings locally using LlamaCppEmbeddings using LangChain tools. The result is then saved in a local vector database using the Chroma vector store.
• privateGPT.py understands questions and generates answers using a local LLM based on GPT4All-J or LlamaCpp. The context for the answers is extracted from the local vector store using a similarity search to find the correct piece of context from the documents.
• GPT4All-J wrapper was introduced in LangChain 0.0.162.

This article is to help you learn about PrivateGPT. We trust that it has been helpful to you. Please feel free to share your thoughts and feedback in the comment section below.