Commit Graph

1965 Commits

Author SHA1 Message Date
pablodanswer
fb99b30930 close to functioning 2024-09-19 19:18:57 -07:00
pablodanswer
ba6296bc81 update some types 2024-09-19 17:47:13 -07:00
pablodanswer
9abde19e44 update interfaces to standardize 2024-09-19 17:41:57 -07:00
pablodanswer
3cc99cf79a squash 2024-09-19 16:07:03 -07:00
pablodanswer
14871a62ad new llm providers 2024-09-19 13:06:11 -07:00
pablodanswer
fe5df3c8ee update integration test models 2024-09-19 12:54:27 -07:00
pablodanswer
3f0895682c additional clarity for llm provider creation / updates 2024-09-14 11:33:21 -07:00
pablodanswer
3b70a94fcd update typing 2024-09-14 11:00:13 -07:00
pablodanswer
074342165b new model structure concept 2024-09-13 20:38:03 -07:00
pablodanswer
7b91beb3b2 quick update to some naming 2024-09-13 16:52:25 -07:00
pablodanswer
4b2e4ca159 update deployment 2024-09-13 16:40:57 -07:00
pablodanswer
001bbb89cc obfuscate api keys 2024-09-13 16:39:54 -07:00
pablodanswer
648c2531f9 Add custom tool chat session / message ID dynamic prompting (#2404)
* add custom tool chat session / message ID dynamic prompting

* update some formatting

* code organization + remove unnecessary card

* remove log

* update for clarity
2024-09-13 18:42:21 +00:00
pablodanswer
fc98c560a4 Add fix for logging (#2431) v0.5.18 2024-09-13 11:27:20 -07:00
pablodanswer
566f44fcd6 Minor update to llm image ability tracking (#2423)
* minor update to llm image ability tracking

* quick robustification
2024-09-13 17:24:51 +00:00
rkuo-danswer
2fe49e5efb add ssl testing for redis against a cloud instance (#2422) 2024-09-13 10:28:04 -07:00
rkuo-danswer
f58acd4e2a Add redis to helm chart (#2390) 2024-09-13 10:26:51 -07:00
pablodanswer
53008a0271 update multipass indeixng server default 2024-09-13 10:24:26 -07:00
pablodanswer
13278663d9 Update refresh + robustify embeddings (#2420)
* update refresh + robustify embeddings

* squash
2024-09-13 14:26:33 +00:00
pablodanswer
31ca6857fb Custom Refresh on Client Side (#2376) 2024-09-13 00:04:03 -07:00
pablodanswer
6dd91414be delete chat session immediately 2024-09-13 00:02:43 -07:00
rkuo-danswer
140c34e59e ephemeral behavior for redis (#2373)
* ephemeral behavior for redis

* notes for redis command line consistency
2024-09-13 04:48:50 +00:00
rkuo-danswer
da8e68b320 reformat celery logging to match danswer style logging across services (#2409)
* reformat celery logging to match danswer style logging across services

* mypy fixes

* handle logfile argument
2024-09-13 01:51:51 +00:00
hagen-danswer
e9a616e579 Added search_doc_ids to the simple api to allow for skipping search (#2421)
* Added search_doc_ids to the simple api to allow for skipping search

* comment

* fixed behaviour
2024-09-12 23:22:41 +00:00
pablodanswer
cb2169f2a3 Warm up reranker on model switch (#2408)
* warm up reranker on model switch

* properly type

* fix issue

* Update search_settings.py
2024-09-12 22:12:17 +00:00
pablodanswer
79aa5dd6e0 add a tiny bit of clarity to index doc counts (#2414) 2024-09-12 21:59:10 +00:00
hagen-danswer
604ebafe6c simple apis now cited/context doc indices (#2419)
* simple apis now cited/context doc indices

* minor fixes
2024-09-12 21:29:24 +00:00
pablodanswer
a2d775efbd Reformatted tailwind config (#2417)
* reformatted tailwind config

* minor update
2024-09-12 19:41:11 +00:00
rkuo-danswer
641690e3f7 fix enabling ssl in connection pool (#2418) 2024-09-12 19:18:04 +00:00
rkuo-danswer
eebf98e3a6 fix setting redis_scheme (#2416) 2024-09-12 18:07:38 +00:00
rkuo-danswer
4bc4da29f5 add SSL parameter support for redis (#2389)
* add SSL parameter support for redis

* add ssl support to redis pool
2024-09-12 16:18:11 +00:00
pablodanswer
7af572d0e7 display only failed (#2413) v0.5.17 2024-09-12 16:01:17 +00:00
pablodanswer
58bdf9d684 Add connector deletion failure message (#2392) v0.5.16 2024-09-11 22:38:15 -07:00
pablodanswer
f69922fff7 Add environment variable for setting vespa search threads (#2400) 2024-09-11 22:37:38 -07:00
pablodanswer
d4d37c9cdd add bedrock models (#2405) 2024-09-12 04:34:43 +00:00
Yuhong Sun
2654df49fd Update CONTRIBUTING.md 2024-09-11 19:17:23 -07:00
pablodanswer
aee5fcd4e0 Add env variables for overriding embedding batch size (#2395)
* add env variabels for overriding

* proper ports

* proper overrides
2024-09-12 00:51:45 +00:00
pablodanswer
2c77dd241b Add error table to re-indexing (#2388)
* add error table to re-indexing

* robustify

* update with proper comment

* add popup

* update typo
2024-09-11 22:55:55 +00:00
pablodanswer
d90c90dd92 simplify unnecessary display logic (#2406) 2024-09-11 21:35:50 +00:00
pablodanswer
2c971cf774 add claude image-support 2024-09-11 13:31:27 -07:00
trial-danswer
eab55bdd85 Misc clarifications for CONTRIBUTING.md (#2401)
* Reorder and clarify dependency installation instructions

* Clarify instructions for local development with Docker external deps vs full Docker stack

* Final words at the end of the local setup process

---------

Co-authored-by: danswer-trial <danswer-trial@danswer-trials-MacBook-Pro.local>
2024-09-11 19:16:37 +00:00
rkuo-danswer
f4f2fb5943 Bugfix/connector deletion test (#2402)
* fixes a bug with deleting connectors and foreign keys

* test foreign key handling on deletion
2024-09-11 12:04:27 -07:00
rkuo-danswer
71f2f1a90a fixes a bug with deleting connectors and foreign keys (#2398) 2024-09-11 12:03:51 -07:00
hagen-danswer
74a2271422 Added HARD_DELETE_CHATS to environment variables (#2397) 2024-09-11 18:08:29 +00:00
trial-danswer
d42fb6ce34 Add link to macOS contributions doc for installing Python 3.11 (#2396)
Co-authored-by: danswer-trial <danswer-trial@danswer-trials-MacBook-Pro.local>
2024-09-11 17:45:52 +00:00
pablodanswer
0d749ebd46 add ccpair id to logging (#2391) 2024-09-11 01:27:03 +00:00
pablodanswer
9f6e8bd124 Improve Dev Experience (#2347)
* clean interfaces + improve dex experience

* update formatting

* update ports

* ports

* remove some number of unnecessary lines

* remove unnecssary isPublicGroupSelector checks in all spots

* add comment

* update building
2024-09-10 20:49:04 +00:00
pablodanswer
3a2a6abed4 Add basic virtualization (#2370)
* add basic virtualization

* functioning perfectly

* squash

* change ports

* remove some comments

* remove comment

* update buffering clarity
2024-09-10 19:06:04 +00:00
pablodanswer
07f49a384f Update spread order (#2386)
* update spread

* update
2024-09-10 18:04:47 +00:00
rkuo-danswer
f1c5e80f17 Feature/background processing (#2275)
* first cut at redis

* some new helper functions for the db

* ignore kombu tables in alembic migrations (used by celery)

* multiline commands for readability, add vespa_metadata_sync queue to worker

* typo fix

* fix returning tuple fields

* add constants

* fix _get_access_for_document

* docstrings!

* fix double function declaration and typing

* fix type hinting

* add a global redis pool

* Add get_document function

* use task_logger in various celery tasks

* add celeryconfig.py to simplify configuration. Will be used in a subsequent commit

* Add celery redis helper. used in a subsequent PR

* kombu warning getting spammy since celery is not self managing its queue in Postgres any more

* add last_modified and last_synced to documents

* fix task naming convention

* use celeryconfig.py

* the big one. adds queues and tasks, updates functions to use the queues with priorities, etc

* change vespa index log line to debug

* mypy fixes

* update alembic migration

* fix fence ordering, rename to "monitor", fix fetch_versioned_implementation call

* mypy

* switch to monotonic time

* fix startup dependencies on redis

* rebase alembic migration

* kombu cleanup - fail silently

* mypy

* add redis_host environment override

* update REDIS_HOST env var in docker-compose.dev.yml

* update the rest of the docker files

* harden indexing-status endpoint against db changes happening in the background.  Needs further improvement but OK for now.

* allow no task syncs to run because we create certain objects with no entries but initially marked as out of date

* add back writing to vespa on indexing

* update contributing guide

* backporting fixes from background_deletion

* renaming cache to cache_volume

* add redis password to various deployments

* try setting up pr testing for helm

* fix indent

* hopefully this release version actually exists

* fix command line option to --chart-dirs

* fetch-depth 0

* edit values.yaml

* try setting ct working directory

* bypass testing only on change for now

* move files and lint them

* update helm testing

* some issues suggest using --config works

* add vespa repo

* add postgresql repo

* increase timeout

* try amd64 runner

* fix redis password reference

* add comment to helm chart testing workflow

* rename helm testing workflow to disable it

* adding clarifying comments

* address code review

* missed a file

* remove commented warning ... just not needed

---------

Co-authored-by: Richard Kuo <rkuo@rkuo.com>
2024-09-10 16:28:19 +00:00