Microsoft Guidance is a prompt programming language for the future generation that allows developers to control modern language models more effectively and efficiently than traditional prompting or chaining. Guidance programs enable developers to interleave generation, prompting, and logical control into a single continuous flow that corresponds to how the language model processes the text. As a result, the outcomes are more precise and consistent, as well as a more intuitive and productive development experience.
Table of Contents
Microsoft Guidance
Guidance is an effective method for improving the control and efficiency of current language models (LLMs) such as GPT-3 and GPT-4. It enables you to integrate generation, prompting, and logical control into a continuous flow that corresponds to how the language model processes text.
You might also like
The use of basic output structures such as Chain of Thought (CoT) and its variants such as ART (Adaptive Response Technology) and Auto-CoT is a common type of guiding. These structures have been shown to enhance the performance of LLMs. Guidance can enable more richer structures with the advent of more powerful LLMs like GPT-4, making it easier and more cost-effective to deal with them.
Installation
pip install guidanceHow to use the “Microsoft guidance” package in a Jupyter/VSCode Notebook for live streaming of complex templates and generations. Here is the code:
- Importing the “guidance” package:
import guidance- Setting the default language model:
guidance.llm = guidance.llms.OpenAI("text-davinci-003")This line specifies the language model that will be utilized to execute the guidance programs. It’s set to OpenAI’s “text-davinci-003” model in this example. You can modify the model to meet your needs.
- Defining a guidance program:
program = guidance("""Tweak this proverb to apply to model instructions instead.
{{proverb}}
- {{book}} {{chapter}}:{{verse}}
UPDATED
Where there is no guidance{{gen 'rewrite' stop="\\n-"}}
- GPT {{gen 'chapter'}}:{{gen 'verse'}}""")This code block uses a multi-line string to define a guidance program. Variable interpolation is used with double curly braces (e.g., proverb), as well as logical control flow with if statements and generations using gen commands. The program’s goal is to model instructions after a proverb.
- Executing the program:
executed_program = program(
proverb="Where there is no guidance, a people falls,\nbut in an abundance of counselors there is safety.",
book="Proverbs",
chapter=11,
verse=14
)This code block executes the defined guidance program by calling the program object as a function and passing in the desired values for the variables. The executed program is stored in the executed_program variable.
- Accessing generated variables:
executed_program["rewrite"]After executing the program, you can access the generated variables by accessing the keys of the executed_program dictionary. In this case, the generated variable with the key “rewrite” will contain the resulting text.
By using the “Guidance” package, you can iteratively develop and refine your prompts in a notebook environment, combining template-like syntax with language model generation and logical control flow.
Chat Dialog Development with Guidance
How to use the “Microsoft Guidance” package in a Jupyter/VSCode Notebook for interactive chat dialog development with chat models like GPT-4 or Vicuna. Here is the code:
- Connecting to a chat model:
gpt4 = guidance.llms.OpenAI("gpt-4")This line establishes a connection to a chat model, in this case, GPT-4. You can use other chat models like Vicuna by instantiating the corresponding guidance.llms.transformers.Vicuna object.
- Defining a guidance program for chat dialog:
experts = guidance('''
{{#system~}}
You are a helpful and terse assistant.
{{~/system}}
{{#user~}}
I want a response to the following question:
{{query}}
Name 3 world-class experts (past or present) who would be great at answering this?
Don't answer the question yet.
{{~/user}}
{{#assistant~}}
{{gen 'expert_names' temperature=0 max_tokens=300}}
{{~/assistant}}
{{#user~}}
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
{{~/user}}
{{#assistant~}}
{{gen 'answer' temperature=0 max_tokens=500}}
{{~/assistant}}
''', llm=gpt4)This code block defines a guidance program for the chat dialog. The program includes different role tags like {{#system}}, {{#user}}, and {{#assistant}} to represent the different participants in the conversation. Within each role tag, you can specify the desired instructions and use {{gen}} commands to generate responses from the chat model.
- Executing the chat dialog:
experts(query='How can I be more productive?')To initiate the chat dialog, you call the experts object as a function and pass in the user query using the query parameter. This will execute the Microsoft Guidance program and generate a response from the chat model based on the defined instructions.
By using the “guidance” package, you can interactively develop chat dialogs that combine rich templating, logical control, and chat model responses. This allows for dynamic and interactive conversations with the language model.
Also, Read Mojo programming language exclusive for AI.
Agents
You may use the await command within a loop to pause and resume execution of a partially performed Microsoft Guidance program to simulate a dialogue between two agents using the “guidance” package. Here’s an example:
import guidance
import re
guidance.llm = guidance.llms.OpenAI("gpt-4")
role_simulator = guidance('''
{{#system~}}
You are a helpful assistant
{{~/system}}
{{#user~}}
You will answer the user as {{role}} in the following conversation. At every step, I will provide you with the user input, as well as a comment reminding you of your instructions. Never talk about the fact that you are an AI, even if the user asks you. Always answer as {{role}}.
{{#if first_question}}You can also start the conversation.{{/if}}
{{~/user}}
{{~! The assistant either starts the conversation or not, depending on if this is the first or second agent }}
{{#assistant~}}
Ok, I will follow these instructions.
{{#if first_question}}Let me start the conversation now:
{{role}}: {{first_question}}{{/if}}
{{~/assistant}}
{{~! Then the conversation unrolls }}
{{~#geneach 'conversation' stop=False}}
{{#user~}}
User: {{set 'this.input' (await 'input')}}
Comment: Remember, answer as a {{role}}. Start your utterance with {{role}}:
{{~/user}}
{{#assistant~}}
{{gen 'this.response' temperature=0 max_tokens=300}}
{{~/assistant}}
{{~/geneach}}''')
republican = role_simulator(role='Republican', await_missing=True)
democrat = role_simulator(role='Democrat', await_missing=True)
first_question = '''What do you think is the best way to stop inflation?'''
republican = republican(input=first_question, first_question=None)
democrat = democrat(input=republican["conversation"][-2]["response"].strip('Republican: '), first_question=first_question)
for i in range(2):
republican = republican(input=democrat["conversation"][-2]["response"].replace('Democrat: ', ''))
democrat = democrat(input=republican["conversation"][-2]["response"].replace('Republican: ', ''))
print('Democrat: ' + first_question)
for x in democrat['conversation'][:-1]:
print('Republican:', x['input'])
print()
print(x['response'])Generation
Basic generation
The gen tag is used to generate text. You can use whatever arguments are supported by the underlying model. Executing a prompt calls the generation prompt:
import guidance
# Set the default llm. Could also pass a different one as argument to guidance(), with guidance(llm=...)
guidance.llm = guidance.llms.OpenAI("text-davinci-003")
prompt = guidance('''The best thing about the beach is {{~gen 'best' temperature=0.7 max_tokens=7}}''')
prompt = prompt()
promptguidance caches all OpenAI generations with the same arguments. If you want to flush the cache, you can call guidance.llms.OpenAI.cache.clear().
Hidden generation
You can generate text without displaying it or using it in the subsequent generations using the hidden tag, either in a block or in a gen tag:
prompt = guidance('''{{#block hidden=True}}Generate a response to the following email:
{{email}}.
Response:{{gen "response"}}{{/block}}
I will show you an email and a response, and you will tell me if it's offensive.
Email: {{email}}.
Response: {{response}}
Is the response above offensive in any way? Please answer with a single word, either "Yes" or "No".
Answer:{{#select "answer" logprobs='logprobs'}} Yes{{or}} No{{/select}}''')
prompt = prompt(email='I hate tacos')
promptFeatures of Guidance
- Simple Syntax: Microsoft Guidance has a basic and accessible syntax based on Handlebars templating. This syntax makes it simple to specify and alter the produced text’s output structure.
- Rich Output Structure: Microsoft Guidance enables the design of complicated output structures involving numerous generations, selects, conditionals, tool usage, and more. This adaptability allows for fine-grained control over the output text’s content and layout.
- Streaming in Jupyter/VSCode Notebooks: Microsoft Guidance provides a playground-like experience for interactive and iterative model creation in Jupyter/VSCode Notebooks. This feature makes it easier to experiment with and explore the model’s possibilities.
- Caching for Smart Seed-Based Generation: Microsoft Guidance includes clever caching methods that enhance the generation process. It minimizes duplicate computations and speeds up subsequent requests for comparable outputs by caching previously created text based on seed inputs.
- Role-Based Chat Model Support: Microsoft Guidance is designed to interact seamlessly with role-based chat models such as ChatGPT. This connection allows for context-aware discussions and improves the model’s capacity to recognize and respond to various roles or personas.
- Simple Integration with Hugging Face Models: Microsoft Guidance may be simply connected with Hugging Face model hub models. It includes a number of acceleration techniques to increase speed over ordinary prompting, token healing to optimize prompt borders, and regex pattern guides to impose certain formats or limits on the output text.
Overall, Microsoft Guidance offers a complete framework for regulating and optimizing the output of current language models, making it simpler to effectively and economically use their capabilities.
This article is to help you learn Microsoft Guidance. We trust that it has been helpful to you. Please feel free to share your thoughts and feedback in the comment section below.
