678 Commits

Author SHA1 Message Date
Yuhong Sun
c0c8ae6c08
Minor Tuning for Filters (#753) 2023-11-21 15:47:58 -08:00
Weves
1225c663eb Add new env variable to compose file 2023-11-20 21:40:54 -08:00
Weves
e052d607d5 Add option to log Vespa timing info 2023-11-20 21:37:22 -08:00
Yuhong Sun
8e5e11a554
Add md files to File Connector (#749) 2023-11-20 19:56:06 -08:00
Yuhong Sun
57f0323f52
NLP Model Warmup Reworked (#748) 2023-11-20 17:28:23 -08:00
Weves
6e9f31d1e9 Fix ResourceLogger blocking main thread 2023-11-20 16:46:18 -08:00
Weves
eeb844e35e Fix bug with Google Drive shortcut error case 2023-11-20 16:34:07 -08:00
Sid Ravinutala
d6a84ab413 fix for url parsing google site 2023-11-20 16:08:43 -08:00
Yuhong Sun
0cc3d65839
Add option to run a faster/cheaper LLM for secondary flows (#742) 2023-11-19 17:48:42 -08:00
Weves
df37387146 Fix a couple bugs with google sites link finding 2023-11-19 15:35:54 -08:00
Yuhong Sun
f72825cd46
Provide Metadata to the LLM (#740) 2023-11-19 12:28:45 -08:00
Yuhong Sun
6fb07d20cc
Multilingual Query Expansion (#737) 2023-11-19 10:55:55 -08:00
Chris Weaver
b258ec1bed
Adjust checks for removal from existing_jobs dict + add more logging + only one scheduled job for a connector at a time (#739) 2023-11-19 02:03:17 -08:00
Yuhong Sun
4fd55b8928
Fix GPT4All (#738) 2023-11-18 21:21:02 -08:00
Yuhong Sun
fa0d19cc8c
LLM Chunk Filtering (#735) 2023-11-18 17:12:24 -08:00
Weves
d5916e420c Fix duplicated query event for 'answer_qa_query_stream' and missing llm_answer in 'answer_qa_query' 2023-11-17 21:10:23 -08:00
Weves
ae72cd56f8 Add a bit more logging in indexing pipeline 2023-11-16 12:00:19 -08:00
Yuhong Sun
be5ef77896
Optional Anonymous Telemetry (#727) 2023-11-16 09:22:36 -08:00
Weves
0ed8f14015 Improve Vespa filtering performance 2023-11-15 14:30:12 -08:00
Weves
a03e443541 Add root_page_id option for Notion connector 2023-11-15 12:46:41 -08:00
Weves
81cbcbb403 Fix connector deletion bug 2023-11-14 09:07:59 -08:00
Yuhong Sun
69644b266e
Hybrid Search Alpha Parameter (#714) 2023-11-09 17:11:10 -08:00
Yuhong Sun
5a4820c55f
Skip Index on Docs with no newer updated at (#713) 2023-11-09 16:27:32 -08:00
Weves
a5d69bb392 Add back end time to Gong 2023-11-09 14:03:46 -08:00
Weves
23ee45c033 Enhance document explorer 2023-11-09 00:58:51 -08:00
Yuhong Sun
31bfd015ae
Request Tracker Connector (#709)
Contributed by Evan! Thanks for the contribution!

- Minor linting and rebasing done by Yuhong, everything else from Evan

---------

Co-authored-by: Evan Sarmiento <e.sarmiento@soax.com>
Co-authored-by: Evan <esarmien@fas.harvard.edu>
2023-11-07 16:55:10 -08:00
Yuhong Sun
0125d8a0f6
Source Filter Extraction (#708) 2023-11-07 14:21:04 -08:00
Yuhong Sun
4f64444f0f
Fix Version from Tag not picked up (#705) 2023-11-06 20:01:20 -08:00
Weves
abf9cc3248 Add timeout to all Notion calls 2023-11-06 19:29:42 -08:00
Chris Weaver
f5bf2e6374
Fix experimental checkpointing + move check for disabled connector to the start of the batch (#703) 2023-11-06 17:14:31 -08:00
Yuhong Sun
7433dddac3
Model Server (#695)
Provides the ability to pull out the NLP models into a separate model server which can then be hosted on a GPU instance if desired.
2023-11-06 16:36:09 -08:00
Weves
fe938b6fc6 Add experimental checkpointing 2023-11-04 14:51:28 -07:00
dependabot[bot]
2db029672b
Bump pypdf from 3.16.4 to 3.17.0 in /backend/requirements (#667)
Bumps [pypdf](https://github.com/py-pdf/pypdf) from 3.16.4 to 3.17.0.
- [Release notes](https://github.com/py-pdf/pypdf/releases)
- [Changelog](https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md)
- [Commits](https://github.com/py-pdf/pypdf/compare/3.16.4...3.17.0)

---
updated-dependencies:
- dependency-name: pypdf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-03 18:54:29 -07:00
Yuhong Sun
602f9c4a0a
Default Version to 0.2-dev (#690) 2023-11-03 18:37:01 -07:00
Bradley
551705ad62
Implemented Danswer versioning system. (#649)
* Web & API server versioning system. Displayed on UI.

* Remove some debugging code.

* Integrated backend version into GitHub Action & Docker build workflow using env variables.

* Fixed web container environment variable name.

* Revise Dockerfiles for GitHub Actions workflow.

* Added system information page to admin panel with version info. Updated github workflows to include tagged version, and corresponding changes in the dockerfiles and codebases for web&backend to use env variables if present. Changed to 'dev' naming scheme if no env var is present to indicate local setup. Removed version from admin panel header.

* Added missing systeminfo dir to remote repo.
2023-11-03 18:02:39 -07:00
Weves
d9581ce0ae Fix Notion recursive search for non-shared database 2023-11-03 15:46:23 -07:00
Yuhong Sun
e27800d501 Formatting 2023-11-02 23:31:19 -07:00
Yuhong Sun
927dffecb5
Prompt Layer Rework (#688) 2023-11-02 23:26:47 -07:00
Weves
68b23b6339 Enable database reading in recursive notion crawl 2023-11-02 23:14:54 -07:00
Weves
174f54473e Fix notion recursive search for blocks with children 2023-11-02 22:21:55 -07:00
Weves
329824ab22 Address issue with links for Google Sites connector 2023-11-02 22:01:08 -07:00
Yuhong Sun
b0f76b97ef
Guru and Productboard Time Updated (#683) 2023-11-02 14:27:06 -07:00
Weves
e8786e1a20 Small formatting fixes 2023-11-01 21:46:23 -07:00
Bryan Peterson
44e3dcb19f
support for zendesk help center (#661) 2023-11-01 21:11:56 -07:00
Weves
e8f778ccb5 Improve index attempt display 2023-11-01 18:33:54 -07:00
Weves
d9adee168b Add simple job client to try and get rid of some of the flakiness / weirdness that we are seeing with Dask 2023-11-01 17:43:58 -07:00
Yuhong Sun
73b653d324
More Cleanup and Deduplication (#675) 2023-11-01 16:03:48 -07:00
Weves
9cd0c197e7 Fix frozen jobs 2023-11-01 14:30:51 -07:00
Weves
0b07d615b1 Add env variable to control Gong start time 2023-11-01 14:09:13 -07:00
Weves
5c9c70dffb Remove more native enums 2023-11-01 12:51:33 -07:00