63 Commits

Author SHA1 Message Date
9dbe12cea8 Feat: Search Eval Testing Overhaul (provide ground truth, categorize query, etc.) (#4739)
* fix: autoflake & import order

* docs: readme

* fix: mypy

* feat: eval

* docs: readme

* fix: oops forgot to remove comment

* fix: typo

* fix: rename var

* updated default config

* fix: config issue

* oops

* fix: black

* fix: eval and config

* feat: non tool calling query mod
2025-05-21 19:25:10 +00:00
cac03c07f7 v1 answer refactor (#4721)
* v1 answer refactor

* fix tests

* good catch, tests

* more cleanup
2025-05-20 23:34:27 +00:00
d64f479c9f feat: error handling & optimization (#4722) 2025-05-19 20:27:22 +00:00
30d9ce1310 feat: search quality eval (#4720)
* fix: import order

* test examples

* fix: import

* wip: reranker based eval

* fix: import order

* feat: adjuted score

* fix: mypy

* fix: suggestions

* sorry cvs, you must go

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: mypy

* fix: suggestions

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-05-15 23:44:33 +00:00
953a4e3793 v1 file connector with metadata (#4552) 2025-04-17 23:02:34 +00:00
426883bbf5 Feature/agentic buffered (#4231)
* rename agent test script to prevent pytest autodiscovery

* first cut

* fix log message

* fix up typing

* add a sample test

---------

Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2025-03-10 15:48:42 +00:00
6687d5d499 major Agent Search Updates (#3994) 2025-02-14 19:40:21 +00:00
118e8afbef reworked config to have logical structure 2025-02-03 20:10:51 -08:00
0ccd83e809 deep_search_a and agent_a_config renaming 2025-02-03 20:10:51 -08:00
bd3b1943c4 WIP PR comments 2025-02-03 20:10:50 -08:00
4fd6e36c2f second clean commit 2025-02-03 20:07:57 -08:00
21ec5ed795 welcome to onyx 2024-12-13 09:56:10 -08:00
aa1c4c635a Combining Search and Chat Backend (#3273)
* k

* k

* fix slack issues

* rebase

* k
2024-12-03 22:37:14 +00:00
62a4aa10db Refactor Search (#3233) 2024-11-23 13:42:54 -08:00
089c734f63 disabled llm when skip_gen_ai_answer_question set (#2687)
* disabled llm when skip_gen_ai_answer_question set

* added unit test

* typing
2024-10-06 18:10:02 +00:00
50c17438d5 Litellm bump (#2195)
* ran bump-pydantic

* replace root_validator with model_validator

* mostly working. some alternate assistant error. changed root_validator and typing_extensions

* working generation chat. changed type

* replacing .dict with .model_dump

* argument needed to bring model_dump up to parity with dict()

* fix a fewremaining issues -- working with llama and gpt

* updating requirements file

* more requirement updates

* more requirement updates

* fix to make search work

* return type fix:

* half way tpyes change

* fixes for mypy and pydantic:

* endpoint fix

* fix pydantic protected namespaces

* it works!

* removed unecessary None initializations

* better logging

* changed default values to empty lists

* mypy fixes

* fixed array defaulting

---------

Co-authored-by: hagen-danswer <hagen@danswer.ai>
2024-08-28 00:00:27 +00:00
61b5bd569b Reworked chunking to support mega chunks (#2032) 2024-08-14 22:18:53 -07:00
c7e5b11c63 EE Connector Deletion Bugfix + Refactor (#2042)
---------

Co-authored-by: Weves <chrisweaver101@gmail.com>
2024-08-11 20:33:07 -07:00
0261d689dc Various Admin Page + User Flow Improvements (#1987) 2024-08-03 18:09:46 -07:00
5307d38472 Fixed tokenizer logic (#1986) 2024-07-31 09:59:45 -07:00
3aa0e0124b Add new admin page (#1947)
* add admin page

* credential + typing fix

* rebase fix

* on add, cleaner buttons

* functional G + Ddrive

* organized auth sections

* update types and remove logs

* ccs -> connectors

* validated formik

* update styling + connector-handling logic

* udpate colors

* separate out hooks + util functions

* update to adhere to rest standards

* remove "todos"

* rebase

* copy + formatting + sidebar

* update statuses + configuration possibilities

* update interfaces to be clearer

* update indexing status page

* formatting

* address backend security + comments

* update font

* fix form routing

* fix hydration error

* add statuses, fix bugs, etc. (squash)

* fix color (squash)

* squash

* add functionality to sidebar

* disblae buttons if deleting

* add color

* minor copy + formatting updates
- on modify credential, close
- update copy for deletion of connectors

* fix build error

* copy

---------

Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-07-28 20:57:43 -07:00
6ff8e6c0ea Improve eval pipeline qol (#1908) 2024-07-23 17:16:34 -07:00
a0b46c60c6 Switched eval api target back to oneshotqa (#1902) 2024-07-22 20:55:18 -07:00
6c88c0156c Added file upload retry logic (#1889) 2024-07-22 13:13:22 -07:00
2ba24b1734 Reenable Search Pipeline (#1886) 2024-07-21 10:33:29 -07:00
44820b4909 k 2024-07-21 10:27:57 -07:00
e95bfa0e0b Suffix Test (#1880) 2024-07-20 15:54:55 -07:00
4848b5f1de Suffix Edits (#1878) 2024-07-20 13:59:14 -07:00
38290057f2 Search Eval (#1873) 2024-07-19 16:48:58 -07:00
87fadb07ea COMPLETE USER EXPERIENCE OVERHAUL (#1822) 2024-07-17 19:44:21 -07:00
e93de602c3 Use SHA instead of branch and save more data (#1850) 2024-07-17 14:56:24 -07:00
b83f435bb0 Catch dropped eval questions and added multiprocessing (#1849) 2024-07-16 12:33:02 -07:00
a1e638a73d Improved eval logging and stability (#1843) 2024-07-15 14:58:45 -07:00
1b864a00e4 Added support for multiple Eval Pipeline UIs (#1830) 2024-07-14 15:16:20 -07:00
991ee79e47 some qol improvements for search pipeline (#1809) 2024-07-11 17:42:11 -07:00
39982e5fdc Info propagating to allow Chunk Merging (#1794) 2024-07-09 18:15:07 -07:00
f0888f2f61 Eval Script Incremental Write (#1784) 2024-07-06 15:43:40 -07:00
d35d7ee833 Evaluation Pipeline Touchup (#1783) 2024-07-06 13:17:05 -07:00
ac14369716 Added search quality testing pipeline (#1774) 2024-07-06 11:51:50 -07:00
415960564d Fix fast models 2024-06-29 15:19:09 -07:00
680482bd06 Metadata filter for document search API (#53) 2024-06-25 15:07:56 -07:00
123ec4342a Relari (#1687)
Also includes some bugfixes
2024-06-22 18:52:48 -07:00
b723627e0c Ability to pass through headers to LLM call 2024-06-10 13:18:31 -07:00
a17060af5a Provide Additional Context for Chunk Options in APIs (#1330) 2024-04-14 18:32:22 -07:00
f135ba9c0c Rework LLM answering flow 2024-03-25 13:34:03 -07:00
1ba74ee4df Refactor search pipeline 2024-03-25 13:34:03 -07:00
6768c24723 Default LLM Update (#1042) 2024-02-05 01:25:51 -08:00
bb2b517124 Relari Test Script (#1033) 2024-02-02 09:50:48 -08:00
4b45164496 Background Index Attempt Creation (#1010) 2024-01-28 23:14:20 -08:00
cf4ede2130 Embedding Models Table (#1006) 2024-01-26 18:40:53 -08:00