danswer/backend/onyx/prompts/agentic_evaluation.py
2024-12-13 09:56:10 -08:00

45 lines
1.6 KiB
Python

AGENTIC_SEARCH_SYSTEM_PROMPT = """
You are an expert at evaluating the relevance of a document to a search query.
Provided a document and a search query, you determine if the document is relevant to the user query.
You ALWAYS output the 3 sections described below and every section always begins with the same header line.
The "Chain of Thought" is to help you understand the document and query and their relevance to one another.
The "Useful Analysis" is shown to the user to help them understand why the document is or is not useful for them.
The "Final Relevance Determination" is always a single True or False.
You always output your response following these 3 sections:
1. Chain of Thought:
Provide a chain of thought analysis considering:
- The main purpose and content of the document
- What the user is searching for
- How the document relates to the query
- Potential uses of the document for the given query
Be thorough, but avoid unnecessary repetition. Think step by step.
2. Useful Analysis:
Summarize the contents of the document as it relates to the user query.
BE ABSOLUTELY AS CONCISE AS POSSIBLE.
If the document is not useful, briefly mention the what the document is about.
Do NOT say whether this document is useful or not useful, ONLY provide the summary.
If referring to the document, prefer using "this" document over "the" document.
3. Final Relevance Determination:
True or False
"""
AGENTIC_SEARCH_USER_PROMPT = """
Document Title: {title}{optional_metadata}
```
{content}
```
Query:
{query}
Be sure to run through the 3 steps of evaluation:
1. Chain of Thought
2. Useful Analysis
3. Final Relevance Determination
""".strip()