80 Commits

Author SHA1 Message Date
Chris Weaver
bb7e1d6e55
Add integration tests for document set syncing (#1904) 2024-08-06 18:00:19 -07:00
rkuo-danswer
fcc4c30ead
don't skip the start of the json answer value (#2067) 2024-08-06 23:59:13 +00:00
pablodanswer
0261d689dc
Various Admin Page + User Flow Improvements (#1987) 2024-08-03 18:09:46 -07:00
Weves
51731ad0dd Fix issue where large docs/batches break openai embedding 2024-08-02 01:07:09 -07:00
hagen-danswer
5307d38472
Fixed tokenizer logic (#1986) 2024-07-31 09:59:45 -07:00
Yuhong Sun
036d5c737e
No Null Embeddings (#1982) 2024-07-30 19:54:49 -07:00
hagen-danswer
3938a053aa
Rework tokenizer (#1957) 2024-07-29 23:01:49 -07:00
rkuo-danswer
4a0a927a64
fix removed parameter in MediaWikiConnector (#1970) 2024-07-29 18:47:30 +00:00
pablodanswer
3aa0e0124b
Add new admin page (#1947)
* add admin page

* credential + typing fix

* rebase fix

* on add, cleaner buttons

* functional G + Ddrive

* organized auth sections

* update types and remove logs

* ccs -> connectors

* validated formik

* update styling + connector-handling logic

* udpate colors

* separate out hooks + util functions

* update to adhere to rest standards

* remove "todos"

* rebase

* copy + formatting + sidebar

* update statuses + configuration possibilities

* update interfaces to be clearer

* update indexing status page

* formatting

* address backend security + comments

* update font

* fix form routing

* fix hydration error

* add statuses, fix bugs, etc. (squash)

* fix color (squash)

* squash

* add functionality to sidebar

* disblae buttons if deleting

* add color

* minor copy + formatting updates
- on modify credential, close
- update copy for deletion of connectors

* fix build error

* copy

---------

Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-07-28 20:57:43 -07:00
Yuhong Sun
9651ea828b
Handling Metadata by Vector and Keyword (#1909) 2024-07-24 11:05:56 -07:00
hagen-danswer
6ff8e6c0ea
Improve eval pipeline qol (#1908) 2024-07-23 17:16:34 -07:00
hagen-danswer
a0b46c60c6
Switched eval api target back to oneshotqa (#1902) 2024-07-22 20:55:18 -07:00
hagen-danswer
6c88c0156c
Added file upload retry logic (#1889) 2024-07-22 13:13:22 -07:00
Yuhong Sun
2ba24b1734
Reenable Search Pipeline (#1886) 2024-07-21 10:33:29 -07:00
Yuhong Sun
44820b4909 k 2024-07-21 10:27:57 -07:00
Yuhong Sun
e95bfa0e0b
Suffix Test (#1880) 2024-07-20 15:54:55 -07:00
Yuhong Sun
4848b5f1de
Suffix Edits (#1878) 2024-07-20 13:59:14 -07:00
Yuhong Sun
38290057f2
Search Eval (#1873) 2024-07-19 16:48:58 -07:00
pablodanswer
87fadb07ea
COMPLETE USER EXPERIENCE OVERHAUL (#1822) 2024-07-17 19:44:21 -07:00
hagen-danswer
e93de602c3
Use SHA instead of branch and save more data (#1850) 2024-07-17 14:56:24 -07:00
hagen-danswer
b83f435bb0
Catch dropped eval questions and added multiprocessing (#1849) 2024-07-16 12:33:02 -07:00
hagen-danswer
a1e638a73d
Improved eval logging and stability (#1843) 2024-07-15 14:58:45 -07:00
Weves
0d52e99bd4 Improve confluence rate limiting 2024-07-14 16:40:45 -07:00
hagen-danswer
1b864a00e4
Added support for multiple Eval Pipeline UIs (#1830) 2024-07-14 15:16:20 -07:00
Yuhong Sun
e90c66c1b6
Include Titles in Chunks (#1817) 2024-07-12 09:42:24 -07:00
hagen-danswer
991ee79e47
some qol improvements for search pipeline (#1809) 2024-07-11 17:42:11 -07:00
Yuhong Sun
08c6e821e7
Merge Sections Logic (#1801) 2024-07-10 20:14:02 -07:00
pablodanswer
09a11b5e1a
Fix citations + unit tests (#1760) 2024-07-10 10:05:20 -07:00
Yuhong Sun
39982e5fdc
Info propagating to allow Chunk Merging (#1794) 2024-07-09 18:15:07 -07:00
Yuhong Sun
f0888f2f61
Eval Script Incremental Write (#1784) 2024-07-06 15:43:40 -07:00
Yuhong Sun
d35d7ee833
Evaluation Pipeline Touchup (#1783) 2024-07-06 13:17:05 -07:00
hagen-danswer
ac14369716
Added search quality testing pipeline (#1774) 2024-07-06 11:51:50 -07:00
Weves
415960564d Fix fast models 2024-06-29 15:19:09 -07:00
Yuhong Sun
680482bd06 Metadata filter for document search API (#53) 2024-06-25 15:07:56 -07:00
Yuhong Sun
123ec4342a
Relari (#1687)
Also includes some bugfixes
2024-06-22 18:52:48 -07:00
Weves
b723627e0c Ability to pass through headers to LLM call 2024-06-10 13:18:31 -07:00
Weves
97d058b8b2 Fix mypy for mediawiki tests 2024-05-25 17:16:47 -07:00
Andrew Sansom
94018e83b0
Add MediaWiki and Wikipedia Connectors (#1250)
* Add MediaWikiConnector first draft

* Add MediaWikiConnector first draft

* Add MediaWikiConnector first draft

* Add MediaWikiConnector sections for each document

* Add MediaWikiConnector to constants and factory

* Integrate MediaWikiConnector with connectors page

* Unit tests + bug fixes

* Allow adding multiple mediawikiconnectors

* add wikipedia connector

* add wikipedia connector to factory

* improve docstrings of mediawiki connector backend

* improve docstrings of mediawiki connector backend

* move wikipedia and mediawiki icon locations in admin page

* undo accidental commit of modified docker compose yaml
2024-05-24 08:51:20 -07:00
Yuhong Sun
546815dc8c
Consolidate File Processing (#1449) 2024-05-11 23:11:22 -07:00
Yuhong Sun
a17060af5a
Provide Additional Context for Chunk Options in APIs (#1330) 2024-04-14 18:32:22 -07:00
Weves
f135ba9c0c Rework LLM answering flow 2024-03-25 13:34:03 -07:00
Weves
1ba74ee4df Refactor search pipeline 2024-03-25 13:34:03 -07:00
Yuhong Sun
6768c24723
Default LLM Update (#1042) 2024-02-05 01:25:51 -08:00
Yuhong Sun
bb2b517124
Relari Test Script (#1033) 2024-02-02 09:50:48 -08:00
Yuhong Sun
4b45164496
Background Index Attempt Creation (#1010) 2024-01-28 23:14:20 -08:00
Itay
a4053501d0
CI: adding prettier to pre-commit (#1009) 2024-01-28 13:03:39 -08:00
Itay
0ce992e22e
CI: Run Python tests (#1001) 2024-01-28 12:59:51 -08:00
Yuhong Sun
cf4ede2130
Embedding Models Table (#1006) 2024-01-26 18:40:53 -08:00
Jeremi Joslin
d07647c597
Fix typo in gmail test connector (#981) 2024-01-24 12:01:26 -08:00
Itay
692fdb4597
Gmail Connector (#946)
---------

Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-01-22 16:25:10 -08:00