562 Commits

Author	SHA1	Message	Date
Yuhong Sun	02095e9281	Restructure APIs (#803)	2023-12-02 14:48:08 -08:00
Yuhong Sun	8954a04602	Reorder Tables for cleaner extending (#800)	2023-12-01 17:46:13 -08:00
Yuhong Sun	8020db9e9a	Update connector interface with optional Owners information (#798)	2023-11-30 23:08:16 -08:00
Yuhong Sun	17c2f06338	Add more metadata options for File connector (#797)	2023-11-30 13:24:22 -08:00
Weves	9cff294a71	Increase retries for google drive connector	2023-11-30 03:03:26 -08:00
Weves	e983aaeca7	Add more logging on existing jobs	2023-11-30 02:58:37 -08:00
Yuhong Sun	fda89ac810	Expert Recommendation Heuristic Only (#791)	2023-11-29 15:53:57 -08:00
Yuhong Sun	006fd4c438	Ingestion API now always updates regardless of document updated_at (#786)	2023-11-29 02:08:50 -08:00
Weves	c64c25b2e1	Fix temp file deletion	2023-11-29 02:00:20 -08:00
Yuhong Sun	c2727a3f19	Custom OpenAI Model Server (#782)	2023-11-29 01:41:56 -08:00
Chris Weaver	37daf4f3e4	Remove AI Thoughts by default (#783) ... - Removes AI Thoughts by default - only shows when validation fails - Removes punctuation "words" from queries in addition to stopwords (Vespa ignores punctuation anyways) - Fixes Vespa deletion script for larger doc counts	2023-11-29 01:00:53 -08:00
Yuhong Sun	fcb7f6fcc0	Accept files with character issues (#781)	2023-11-28 22:43:58 -08:00
Yuhong Sun	187b94a7d8	Blurb Key Error (#778)	2023-11-28 16:09:33 -08:00
Weves	30225fd4c5	Fix filter hiding	2023-11-28 04:13:11 -08:00
Weves	eab4fe83a0	Remove Slack bot personas from web UI	2023-11-28 02:53:18 -08:00
Chris Weaver	78d1ae0379	Customizable personas (#772) ... Also includes a small fix to LLM filtering when combined with reranking	2023-11-28 00:57:48 -08:00
Yuhong Sun	87beb1f4d1	Log LLM details on server start (#773)	2023-11-27 21:32:48 -08:00
Yuhong Sun	05c2b7d34e	Update LLM related Libs (#771)	2023-11-26 19:54:16 -08:00
Yuhong Sun	39d09a162a	Danswer APIs Document Ingestion Endpoint (#716)	2023-11-26 19:09:22 -08:00
Yuhong Sun	d291fea020	Turn off Reranking for Streaming Flows (#770)	2023-11-26 16:45:23 -08:00
Yuhong Sun	2665bff78e	Option to turn off LLM for eval script (#769)	2023-11-26 15:31:03 -08:00
Yuhong Sun	65d38ac8c3	Slack to respect LLM chunk filter settings (#768)	2023-11-26 01:06:12 -08:00
Yuhong Sun	8391d89bea	Fix Indexing Concurrency (#767)	2023-11-25 21:40:36 -08:00
Yuhong Sun	ac2ed31726	Indexing Jobs to have shorter lived DB sessions (#766)	2023-11-24 21:38:16 -08:00
Chris Weaver	47f947b045	Use torch.multiprocessing + enable SimpleJobClient by default (#765)	2023-11-24 18:29:28 -08:00
Weves	3cec854c5c	Allow different model servers for different models / indexing jobs	2023-11-23 23:39:03 -08:00
Weves	26c6651a03	Improve LLM answer parsing	2023-11-23 15:03:35 -08:00
Yuhong Sun	13001ede98	Search Regression Test and Save/Load State updates (#761)	2023-11-23 00:00:30 -08:00
Yuhong Sun	fda377a2fa	Regression Script for Search quality (#760)	2023-11-22 19:33:28 -08:00
Yuhong Sun	bdfb894507	Slack Role Override (#755)	2023-11-22 17:47:18 -08:00
Weves	35c3511daa	Increase Vespa timeout	2023-11-22 01:42:59 -08:00
Chris Weaver	c1e19d0d93	Add selected docs in UI + rework the backend flow a bit(#754) ... Changes the flow so that the selected docs are sent over in a separate packet rather than as part of the initial packet for the streaming QA endpoint.	2023-11-21 19:46:12 -08:00
mattboret	e78aefb408	Add script to analyse the sources selection (#721) ... --------- Co-authored-by: Matthieu Boret <matthieu.boret@fr.clara.net>	2023-11-21 18:35:26 -08:00
Bryan Peterson	aa2e859b46	add missing dependencies in model_server dockerfile (#752) ... Thanks for catching this! Super helpful!	2023-11-21 17:59:28 -08:00
Yuhong Sun	c0c8ae6c08	Minor Tuning for Filters (#753)	2023-11-21 15:47:58 -08:00
Weves	1225c663eb	Add new env variable to compose file	2023-11-20 21:40:54 -08:00
Weves	e052d607d5	Add option to log Vespa timing info	2023-11-20 21:37:22 -08:00
Yuhong Sun	8e5e11a554	Add md files to File Connector (#749)	2023-11-20 19:56:06 -08:00
Yuhong Sun	57f0323f52	NLP Model Warmup Reworked (#748)	2023-11-20 17:28:23 -08:00
Weves	6e9f31d1e9	Fix ResourceLogger blocking main thread	2023-11-20 16:46:18 -08:00
Weves	eeb844e35e	Fix bug with Google Drive shortcut error case	2023-11-20 16:34:07 -08:00
Sid Ravinutala	d6a84ab413	fix for url parsing google site	2023-11-20 16:08:43 -08:00
Yuhong Sun	0cc3d65839	Add option to run a faster/cheaper LLM for secondary flows (#742)	2023-11-19 17:48:42 -08:00
Weves	df37387146	Fix a couple bugs with google sites link finding	2023-11-19 15:35:54 -08:00
Yuhong Sun	f72825cd46	Provide Metadata to the LLM (#740)	2023-11-19 12:28:45 -08:00
Yuhong Sun	6fb07d20cc	Multilingual Query Expansion (#737)	2023-11-19 10:55:55 -08:00
Chris Weaver	b258ec1bed	Adjust checks for removal from existing_jobs dict + add more logging + only one scheduled job for a connector at a time (#739)	2023-11-19 02:03:17 -08:00
Yuhong Sun	4fd55b8928	Fix GPT4All (#738)	2023-11-18 21:21:02 -08:00
Yuhong Sun	fa0d19cc8c	LLM Chunk Filtering (#735)	2023-11-18 17:12:24 -08:00
Weves	d5916e420c	Fix duplicated query event for 'answer_qa_query_stream' and missing llm_answer in 'answer_qa_query'	2023-11-17 21:10:23 -08:00