784 Commits

Author SHA1 Message Date
hj-danswer
974f85da66
Migrate standard answers implementations to ee/ ()
* Migrate standard answers implementations to ee/

* renaming

* Clean up slackbot non-ee standard answers import

* Move backend api/manage/standard_answer route to ee

* Move standard answers web UI to ee

* Hide standard answer controls in bot edit page

* Kwargs for fetch_versioned_implementation

* Add docstring explaining return types for handle_standard_answers

* Consolidate blocks into ee/handle_standard_answers

---------

Co-authored-by: Hyeong Joon Suh <hyeongjoonsuh@Hyeongs-MacBook-Pro.local>
Co-authored-by: danswer-trial <danswer-trial@danswer-trials-MacBook-Pro.local>
2024-09-14 01:57:03 +00:00
hj-danswer
3cb00de6d4
Support regex in standard answers ()
* Support regex in standard answers

* fix mypy

* Add match_regex boolean column to StandardAnswer

* Add match_regex flag and validation to Pydantic models

* GET /manage/admin/standard-answer: add match_regex to create_standard_answer

* PATCH /manage/admin/standard-answer/🆔 add match_regex to update_standard_answer

* Add "Match Regex" toggle to standard answer form

* Decode error pattern in case it's bytes

* Refactor regex support to use match_regex flag instead of supplemental tuple

* Better error handling for invalid regexes

* Show "match regex" in table and style keywords appropriately

* Fix stale UI copy for non-"match_regex" branch

* Fix stale docstring in find_matching_standard_answers

* Update down_revision to reflect most recent migration

* Update UI copy

* Initial implementation of match group display

* Fix pydantic StandardAnswer vs SQLAlchemy StandardAnswer model usage

* Update docstring return type

* Fix missing key prop

---------

Co-authored-by: Hyeong Joon Suh <hyeongjoonsuh@Hyeongs-MacBook-Pro.local>
Co-authored-by: danswer-trial <danswer-trial@danswer-trials-MacBook-Pro.local>
2024-09-14 00:07:42 +00:00
Chris Weaver
da6e46ae75
Slack flow improvements () 2024-09-13 16:56:45 -07:00
pablodanswer
648c2531f9
Add custom tool chat session / message ID dynamic prompting ()
* add custom tool chat session / message ID dynamic prompting

* update some formatting

* code organization + remove unnecessary card

* remove log

* update for clarity
2024-09-13 18:42:21 +00:00
pablodanswer
566f44fcd6
Minor update to llm image ability tracking ()
* minor update to llm image ability tracking

* quick robustification
2024-09-13 17:24:51 +00:00
pablodanswer
13278663d9
Update refresh + robustify embeddings ()
* update refresh + robustify embeddings

* squash
2024-09-13 14:26:33 +00:00
pablodanswer
31ca6857fb
Custom Refresh on Client Side () 2024-09-13 00:04:03 -07:00
pablodanswer
6dd91414be delete chat session immediately 2024-09-13 00:02:43 -07:00
pablodanswer
cb2169f2a3
Warm up reranker on model switch ()
* warm up reranker on model switch

* properly type

* fix issue

* Update search_settings.py
2024-09-12 22:12:17 +00:00
pablodanswer
79aa5dd6e0
add a tiny bit of clarity to index doc counts () 2024-09-12 21:59:10 +00:00
pablodanswer
a2d775efbd
Reformatted tailwind config ()
* reformatted tailwind config

* minor update
2024-09-12 19:41:11 +00:00
pablodanswer
58bdf9d684
Add connector deletion failure message () 2024-09-11 22:38:15 -07:00
pablodanswer
d4d37c9cdd
add bedrock models () 2024-09-12 04:34:43 +00:00
pablodanswer
2c77dd241b
Add error table to re-indexing ()
* add error table to re-indexing

* robustify

* update with proper comment

* add popup

* update typo
2024-09-11 22:55:55 +00:00
pablodanswer
d90c90dd92
simplify unnecessary display logic () 2024-09-11 21:35:50 +00:00
pablodanswer
2c971cf774 add claude image-support 2024-09-11 13:31:27 -07:00
pablodanswer
9f6e8bd124
Improve Dev Experience ()
* clean interfaces + improve dex experience

* update formatting

* update ports

* ports

* remove some number of unnecessary lines

* remove unnecssary isPublicGroupSelector checks in all spots

* add comment

* update building
2024-09-10 20:49:04 +00:00
pablodanswer
3a2a6abed4
Add basic virtualization ()
* add basic virtualization

* functioning perfectly

* squash

* change ports

* remove some comments

* remove comment

* update buffering clarity
2024-09-10 19:06:04 +00:00
pablodanswer
07f49a384f
Update spread order ()
* update spread

* update
2024-09-10 18:04:47 +00:00
rkuo-danswer
f1c5e80f17
Feature/background processing ()
* first cut at redis

* some new helper functions for the db

* ignore kombu tables in alembic migrations (used by celery)

* multiline commands for readability, add vespa_metadata_sync queue to worker

* typo fix

* fix returning tuple fields

* add constants

* fix _get_access_for_document

* docstrings!

* fix double function declaration and typing

* fix type hinting

* add a global redis pool

* Add get_document function

* use task_logger in various celery tasks

* add celeryconfig.py to simplify configuration. Will be used in a subsequent commit

* Add celery redis helper. used in a subsequent PR

* kombu warning getting spammy since celery is not self managing its queue in Postgres any more

* add last_modified and last_synced to documents

* fix task naming convention

* use celeryconfig.py

* the big one. adds queues and tasks, updates functions to use the queues with priorities, etc

* change vespa index log line to debug

* mypy fixes

* update alembic migration

* fix fence ordering, rename to "monitor", fix fetch_versioned_implementation call

* mypy

* switch to monotonic time

* fix startup dependencies on redis

* rebase alembic migration

* kombu cleanup - fail silently

* mypy

* add redis_host environment override

* update REDIS_HOST env var in docker-compose.dev.yml

* update the rest of the docker files

* harden indexing-status endpoint against db changes happening in the background.  Needs further improvement but OK for now.

* allow no task syncs to run because we create certain objects with no entries but initially marked as out of date

* add back writing to vespa on indexing

* update contributing guide

* backporting fixes from background_deletion

* renaming cache to cache_volume

* add redis password to various deployments

* try setting up pr testing for helm

* fix indent

* hopefully this release version actually exists

* fix command line option to --chart-dirs

* fetch-depth 0

* edit values.yaml

* try setting ct working directory

* bypass testing only on change for now

* move files and lint them

* update helm testing

* some issues suggest using --config works

* add vespa repo

* add postgresql repo

* increase timeout

* try amd64 runner

* fix redis password reference

* add comment to helm chart testing workflow

* rename helm testing workflow to disable it

* adding clarifying comments

* address code review

* missed a file

* remove commented warning ... just not needed

---------

Co-authored-by: Richard Kuo <rkuo@rkuo.com>
2024-09-10 16:28:19 +00:00
pablodanswer
b7ad810d83
Prevent spam search () 2024-09-10 08:44:50 -07:00
pablodanswer
99b28643f7
show groups if they exist for user () 2024-09-10 15:14:30 +00:00
pablodanswer
e563746730
Consent screen ()
* update

* add consent popup

* rm
2024-09-10 02:40:32 +00:00
James Jordan
4558351801
Zendesk tickets () 2024-09-09 16:36:53 -07:00
pablodanswer
dbd56f946f
address pablo's nits () 2024-09-09 14:44:27 -07:00
hj-danswer
e4e4765c60
Add user when they interact outside of UI (e.g. Slack bot) ()
* Add user when they interact outside of UI (e.g. Slack bot)

* fix mypy errors

* don't use user manager to avoid async messiness

* fix email is none scenario

* fix mypy

* make code slightly clearer

* PR comments

* get slack email in generate button as well

* fix alembic migration

* update name to be more descriptive

---------

Co-authored-by: Hyeong Joon Suh <hyeongjoonsuh@Hyeongs-MacBook-Pro.local>
2024-09-09 20:21:31 +00:00
pablodanswer
3a9b964d5c
Add Litellm Rerank proxy ()
* add ability ot set reranking litellm proxy

* add fully functional rerank litellm cards

* minor formatting enforcement

* remove logs
2024-09-09 15:57:01 +00:00
Chris Weaver
be4b6189d2
Fix streaming auth locally () 2024-09-08 14:01:26 -07:00
pablodanswer
ace041415a
Clearer onboarding + Provider Updates () 2024-09-08 13:35:20 -07:00
pablodanswer
1555ac9dab
More explicit credential creation flow ()
* more explcit drive credential creation flow

* remove logs

* update naming

* fix user-contributed formatting

* fix (^) v2
2024-09-08 12:09:23 -07:00
dependabot[bot]
57c1deb8b8
Bump braces from 3.0.2 to 3.0.3 in /web ()
Bumps [braces](https://github.com/micromatch/braces) from 3.0.2 to 3.0.3.
- [Changelog](https://github.com/micromatch/braces/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/braces/compare/3.0.2...3.0.3)

---
updated-dependencies:
- dependency-name: braces
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-07 21:06:34 -07:00
pablodanswer
7d4f8ef4e8
Minor Confluence Fixes for Robustification ()
* add connector config

* update confluence connector
2024-09-08 01:39:49 +00:00
pablodanswer
350482e53e
Squash misc UX bugs () 2024-09-07 14:26:14 -07:00
pablodanswer
fb3d7330fa
minor QOL improvement on first chat () 2024-09-07 14:25:05 -07:00
pablodanswer
491f3254a5 regeneration - don't remove human message unnecessarily 2024-09-06 15:38:02 -07:00
pablodanswer
5abf67fbf0
PDF metadata + list defaults ()
* validate web list

* update pdf extraction of metadat

* remove pdf + log

* stricter type enforcing

* fix up indexing widths

* minor formatting

* add list case

* check for empty metadata
2024-09-06 21:21:24 +00:00
pablodanswer
aeb6060854
Add ability to delete users ()
* add ability to delete users

* fix tiny build issue

* Add comments
2024-09-06 17:37:04 +00:00
hagen-danswer
8977b1b5fc
Paginate connector page ()
* Added pagination to individual connector pages

* I cooked

* Gordon Ramsay in this b

* meepe

* properly calculated max chunk and switch dict to array

* chunks -> batches

* increased max page size

* renmaed var
2024-09-06 17:00:25 +00:00
pablodanswer
69c0419146
Updated refreshing ()
* clean up + add environment variables

* remove log

* update

* update api settings

* somewhat cleaner refresh functionality

* fully functional

* update settings

* validated

* remove random logs

* remove unneeded paramter + log

* move to ee + remove comments

* Cleanup unused

---------

Co-authored-by: Weves <chrisweaver101@gmail.com>
2024-09-06 04:36:55 +00:00
pablodanswer
2bd3833c55
Update search settings + chat/search handling ()
* validate web list

* update search settings + chat/search handling

* remove accidentally added search manager

* minor build fix

* push from local
2024-09-06 00:07:39 +00:00
pablodanswer
04f83eb1e1
Proper popover behavior, no showing queries with no docs, + bubbles () 2024-09-04 21:26:19 -07:00
pablodanswer
420aabc963
Update UX () 2024-09-04 18:45:52 -07:00
pablodanswer
34ba3181ff
Update auth for litellm proxy ()
* update for auth

* validated embedding model names

* remove embedding provider

* remove logs

* add ability to delete search setting

* add abiility to delete models + more streamlined API endpoints

* remove upsert

* minor typing fix

* add connector utils
2024-09-04 20:59:07 +00:00
pablodanswer
0ff1a023cd
Minor search setting clarity ()
* minor search setting clarity

* 5433

* squash

* remove logs
2024-09-03 20:48:34 -07:00
Yuhong Sun
d68d281e1c
Slight copy update () 2024-09-03 20:14:03 -07:00
pablodanswer
f96bd12ab8
prevent accidental submission () 2024-09-03 16:44:54 -07:00
pablodanswer
32359d2dff
Add user dropdown seed-able list ()
* add user dropdown seedable list

* minor cleanup

* fix build issue

* minor type update

* remove log

* quick update to divider logic (squash)

* tiny icon updates
2024-09-03 19:24:50 +00:00
Chris Weaver
5da6d792de
Add ingestion as a "Source" for the FE + improve typing () 2024-09-03 12:34:31 -07:00
pablodanswer
5b1f3c8d4e
Formatting nits ()
* stream in all cases

* update code block

* code formatting nits

* proper ports

* proper ports

* remove unnecessary lines
2024-09-03 16:05:02 +00:00
hagen-danswer
a3b1b1db38
fixed doc set table () 2024-09-03 15:36:07 +00:00