608 Commits

Author SHA1 Message Date
Weves
9cff294a71 Increase retries for google drive connector 2023-11-30 03:03:26 -08:00
Weves
e983aaeca7 Add more logging on existing jobs 2023-11-30 02:58:37 -08:00
Yuhong Sun
fda89ac810
Expert Recommendation Heuristic Only () 2023-11-29 15:53:57 -08:00
Yuhong Sun
006fd4c438
Ingestion API now always updates regardless of document updated_at () 2023-11-29 02:08:50 -08:00
Weves
c64c25b2e1 Fix temp file deletion 2023-11-29 02:00:20 -08:00
Yuhong Sun
c2727a3f19
Custom OpenAI Model Server () 2023-11-29 01:41:56 -08:00
Chris Weaver
37daf4f3e4
Remove AI Thoughts by default ()
- Removes AI Thoughts by default - only shows when validation fails
- Removes punctuation "words" from queries in addition to stopwords (Vespa ignores punctuation anyways)
- Fixes Vespa deletion script for larger doc counts
2023-11-29 01:00:53 -08:00
Yuhong Sun
fcb7f6fcc0
Accept files with character issues () 2023-11-28 22:43:58 -08:00
Yuhong Sun
187b94a7d8
Blurb Key Error () 2023-11-28 16:09:33 -08:00
Weves
30225fd4c5 Fix filter hiding 2023-11-28 04:13:11 -08:00
Weves
eab4fe83a0 Remove Slack bot personas from web UI 2023-11-28 02:53:18 -08:00
Chris Weaver
78d1ae0379
Customizable personas ()
Also includes a small fix to LLM filtering when combined with reranking
2023-11-28 00:57:48 -08:00
Yuhong Sun
87beb1f4d1
Log LLM details on server start () 2023-11-27 21:32:48 -08:00
Yuhong Sun
05c2b7d34e
Update LLM related Libs () 2023-11-26 19:54:16 -08:00
Yuhong Sun
39d09a162a
Danswer APIs Document Ingestion Endpoint () 2023-11-26 19:09:22 -08:00
Yuhong Sun
d291fea020
Turn off Reranking for Streaming Flows () 2023-11-26 16:45:23 -08:00
Yuhong Sun
2665bff78e
Option to turn off LLM for eval script () 2023-11-26 15:31:03 -08:00
Yuhong Sun
65d38ac8c3
Slack to respect LLM chunk filter settings () 2023-11-26 01:06:12 -08:00
Yuhong Sun
8391d89bea
Fix Indexing Concurrency () 2023-11-25 21:40:36 -08:00
Yuhong Sun
ac2ed31726
Indexing Jobs to have shorter lived DB sessions () 2023-11-24 21:38:16 -08:00
Chris Weaver
47f947b045
Use torch.multiprocessing + enable SimpleJobClient by default () 2023-11-24 18:29:28 -08:00
Weves
3cec854c5c Allow different model servers for different models / indexing jobs 2023-11-23 23:39:03 -08:00
Weves
26c6651a03 Improve LLM answer parsing 2023-11-23 15:03:35 -08:00
Yuhong Sun
13001ede98
Search Regression Test and Save/Load State updates () 2023-11-23 00:00:30 -08:00
Yuhong Sun
fda377a2fa
Regression Script for Search quality () 2023-11-22 19:33:28 -08:00
Yuhong Sun
bdfb894507
Slack Role Override () 2023-11-22 17:47:18 -08:00
Weves
35c3511daa Increase Vespa timeout 2023-11-22 01:42:59 -08:00
Chris Weaver
c1e19d0d93
Add selected docs in UI + rework the backend flow a bit()
Changes the flow so that the selected docs are sent over in a separate packet rather than as part of the initial packet for the streaming QA endpoint.
2023-11-21 19:46:12 -08:00
mattboret
e78aefb408
Add script to analyse the sources selection ()
---------

Co-authored-by: Matthieu Boret <matthieu.boret@fr.clara.net>
2023-11-21 18:35:26 -08:00
Bryan Peterson
aa2e859b46
add missing dependencies in model_server dockerfile ()
Thanks for catching this! Super helpful!
2023-11-21 17:59:28 -08:00
Yuhong Sun
c0c8ae6c08
Minor Tuning for Filters () 2023-11-21 15:47:58 -08:00
Weves
1225c663eb Add new env variable to compose file 2023-11-20 21:40:54 -08:00
Weves
e052d607d5 Add option to log Vespa timing info 2023-11-20 21:37:22 -08:00
Yuhong Sun
8e5e11a554
Add md files to File Connector () 2023-11-20 19:56:06 -08:00
Yuhong Sun
57f0323f52
NLP Model Warmup Reworked () 2023-11-20 17:28:23 -08:00
Weves
6e9f31d1e9 Fix ResourceLogger blocking main thread 2023-11-20 16:46:18 -08:00
Weves
eeb844e35e Fix bug with Google Drive shortcut error case 2023-11-20 16:34:07 -08:00
Sid Ravinutala
d6a84ab413 fix for url parsing google site 2023-11-20 16:08:43 -08:00
Yuhong Sun
0cc3d65839
Add option to run a faster/cheaper LLM for secondary flows () 2023-11-19 17:48:42 -08:00
Weves
df37387146 Fix a couple bugs with google sites link finding 2023-11-19 15:35:54 -08:00
Yuhong Sun
f72825cd46
Provide Metadata to the LLM () 2023-11-19 12:28:45 -08:00
Yuhong Sun
6fb07d20cc
Multilingual Query Expansion () 2023-11-19 10:55:55 -08:00
Chris Weaver
b258ec1bed
Adjust checks for removal from existing_jobs dict + add more logging + only one scheduled job for a connector at a time () 2023-11-19 02:03:17 -08:00
Yuhong Sun
4fd55b8928
Fix GPT4All () 2023-11-18 21:21:02 -08:00
Yuhong Sun
fa0d19cc8c
LLM Chunk Filtering () 2023-11-18 17:12:24 -08:00
Weves
d5916e420c Fix duplicated query event for 'answer_qa_query_stream' and missing llm_answer in 'answer_qa_query' 2023-11-17 21:10:23 -08:00
Weves
ae72cd56f8 Add a bit more logging in indexing pipeline 2023-11-16 12:00:19 -08:00
Yuhong Sun
be5ef77896
Optional Anonymous Telemetry () 2023-11-16 09:22:36 -08:00
Weves
0ed8f14015 Improve Vespa filtering performance 2023-11-15 14:30:12 -08:00
Weves
a03e443541 Add root_page_id option for Notion connector 2023-11-15 12:46:41 -08:00