1400 Commits

Author SHA1 Message Date
Weves
39c946536c Fix deletion due to foreign key issue 2024-09-02 17:56:43 -07:00
pablodanswer
6afcaafe54
Continue Generating ()
* add stop reason

* add initial propagation

* add continue generating full functionality

* proper continue across chat session

* add new look

* propagate proper types

* fix typing

* cleaner continue generating functionality

* update types

* remove unused imports

* proper infodump

* temp

* add standardized stream handling

* validateing chosen tool args

* properly handle tools

* proper ports

* remove logs + build

* minor typing fix

* fix more minor typing issues

* add stashed reversion for tool call chunks

* ignore model dump types

* remove stop stream

* fix typing
2024-09-02 22:49:56 +00:00
Yuhong Sun
812ca69949
Vespa Degraded Handling () 2024-09-02 15:53:37 -07:00
rkuo-danswer
abe01144ca
Update CONTRIBUTING.md () 2024-09-02 15:30:18 -07:00
Yuhong Sun
d988a3e736
Productboard Minor Fix () 2024-09-02 14:46:35 -07:00
pablodanswer
2b14afe878
Add proper typing such that tests pass mypy ()
* add proper typing such that tests pass mypy

* nit (squash)

* minor update
2024-09-02 21:03:53 +00:00
Chris Weaver
033ec0b6b1
Remove unused env variables () 2024-09-02 20:29:14 +00:00
Weves
0027f161d7 Fix revisions 2024-09-02 11:13:55 -07:00
Yuhong Sun
32e551b69c
Vespa Log No Response () 2024-09-02 09:14:28 -07:00
pablodanswer
299cb5035c
Add litellm proxy embeddings ()
* add litellm proxy

* formatting

* move `api_url` to cloud provider + nits

* remove log

* typing

* quick tuyping fix

* update LiteLLM selection logic

* remove logs + validate functionality

* rename proxy var

* update path casing

* remove pricing for custom models

* functional values
2024-09-02 09:08:35 -07:00
hagen-danswer
aa84846298
Connector deletion fix ()
---------

Co-authored-by: Weves <chrisweaver101@gmail.com>
2024-09-01 23:32:20 -07:00
pablodanswer
c122be2f6a
More explicit Confluence Connector () 2024-09-01 20:35:29 -07:00
Weves
f871b4c6eb Update default anthropic / bedrock models 2024-09-01 20:35:00 -07:00
hagen-danswer
a96cea2ce0 logging improvements 2024-09-01 16:21:35 -07:00
hagen-danswer
8d443ada5b
Integration tests ()
* initial commit

* almost done

* finished 3 tests

* minor refactor

* built out initial permisison tests

* reworked test_deletion

* removed logging

* all original tests have been converted

* renamed user_groups to user_group

* mypy

* added test for doc set permissions

* unified naming for manager methods

* Refactored models and added new deletion test

* minor additions

* better logging+fixed input variables

* commented out failed tests

* Added readme

* readme update

* Added auth to IT

set auth_type to basic and require_email_verification to false

* Update run-it.yml

* used verify and added to readme

* added api key manager
2024-09-01 22:21:00 +00:00
Yuhong Sun
580848cf8c
mypy () 2024-08-30 18:02:18 -07:00
Yuhong Sun
f01027cfb7
Catch LLM Eval Failures () 2024-08-30 17:42:58 -07:00
pablodanswer
76db4b765a
Detect GPU on startup for default multi-pass indexing value () 2024-08-30 17:38:31 -07:00
pablodanswer
5800c7158e
Add typing to pdf extraction () 2024-08-30 17:16:56 -07:00
Weves
21af852073 Add connector creation docs 2024-08-30 16:43:42 -07:00
pablodanswer
4181124e7a
add metadata to pdf extraction () 2024-08-30 15:14:02 -07:00
pablodanswer
f2e8680955
Account for edge case in indexing times with connectors () 2024-08-30 14:07:07 -07:00
pablodanswer
b952dbef42
Minor search formatting updates () 2024-08-30 14:02:35 -07:00
pablodanswer
183569061b
Minor search UX improvements + Critical connector fixes () 2024-08-30 11:47:52 -07:00
hagen-danswer
1734a4a18c
Added DanswerBot response limit environment variables ()
* Added DanswerBot response limit environment variables

* mypy fix

* changed defaults
2024-08-29 19:25:11 +00:00
rkuo-danswer
766652de14
ignore kombu tables used by celery in alembic () 2024-08-29 18:49:35 +00:00
pablodanswer
00fa36d591
Get accurate model output max ()
* get accurate model output max

* squash

* udpated max default tokens

* rename + use fallbacks

* functional

* remove max tokens

* update naming

* comment out function to prevent mypy issues
2024-08-29 18:01:56 +00:00
pablodanswer
3b596fd6a8
Default rerank API key to None (new Pydantic compatibility) ()
* default to None

* rm
2024-08-28 16:02:06 +00:00
hagen-danswer
e4e67c61ef
Some additional curator polish () 2024-08-28 02:44:24 +00:00
Chris Weaver
8afa53c6bf
Confluence improvements ()
* Confluence improvements

* Improve CONFLUENCE_CONNECTOR_INDEX_ONLY_ACTIVE_PAGES
2024-08-28 02:16:10 +00:00
Yuhong Sun
1e67332078
Remove warning on user signup () 2024-08-27 18:49:05 -07:00
josvdw
50c17438d5
Litellm bump ()
* ran bump-pydantic

* replace root_validator with model_validator

* mostly working. some alternate assistant error. changed root_validator and typing_extensions

* working generation chat. changed type

* replacing .dict with .model_dump

* argument needed to bring model_dump up to parity with dict()

* fix a fewremaining issues -- working with llama and gpt

* updating requirements file

* more requirement updates

* more requirement updates

* fix to make search work

* return type fix:

* half way tpyes change

* fixes for mypy and pydantic:

* endpoint fix

* fix pydantic protected namespaces

* it works!

* removed unecessary None initializations

* better logging

* changed default values to empty lists

* mypy fixes

* fixed array defaulting

---------

Co-authored-by: hagen-danswer <hagen@danswer.ai>
2024-08-28 00:00:27 +00:00
Yuhong Sun
657d2050a5
Confluence Internal Error Handling () 2024-08-27 15:23:02 -07:00
Yuhong Sun
3640d0c550
Better Web Connector Logging () 2024-08-27 15:06:24 -07:00
Chris Weaver
8614cd8934
Handle missing email more gracefully () 2024-08-27 20:29:25 +00:00
pablodanswer
97ba71e1b3
Db search ()
* k

* update enum imports

* add functional types + model swaps

* remove a log

* remove kv

* fully functional + robustified for kv swap

* validated with hosted + cloud

* ensure not updating current search settings when reindexing

* add instance check

* revert back to updating search settings (will need a slight refactor for endpoint)

* protect advanced config override1

* run pretty

* fix typing

* update typing

* remove unnecessary function

* update model name

* clearer interface names

* validated foreign key constaint

* proper migration

* squash

---------

Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-08-27 04:26:51 +00:00
Chris Weaver
a873fc6483
Fix Confluence freezing () 2024-08-26 19:44:01 -07:00
Chris Weaver
c0e1a02e8e
Add it on merge queue ()
* Github action to run integration tests

* Improve

* Fix build

* Add pull

* Fix readiness script

* Add IT runner

* Add IT runner

* Add logs

* update

* Fix

* Fix path

* file path

* test

* fix

* fix

* fix

* test

* network

* fix

* cleanup

* fix

* test

* Fix downgrade

* Add OpenAI API key

* Add VESPA_HOST

* test pulling first

* Add API server host

* Cache tweak

* Fix pull/push settings:

* Stop pushing to latest tag

* test cache change

* test

* test

* test

* remove cache temporarily

* Fix

* Enable EE

* test

* Remove duplicate funcs

* add back build

* Update all

* Fix stop cmd

* Add to merge queue

* Cleanup image tag
2024-08-26 07:20:28 +00:00
hagen-danswer
205c3c3fc8
Combined the get document set endpoints ()
* Combined the get document set endpoints

* removed unused function

* fixed permissioning for document sets
2024-08-25 19:02:27 +00:00
hagen-danswer
c21b0ee3f5
Curator polish ()
* add new user provider hook

* account for additional logic

* add users

* remove is loading

* Curator polish

* useeffect -> provider + effect

* squash

* use use user for user default models

* squash

* Added ability to add users to groups among other things

* final polish

* added connection button to groups

* mypy fix

* Improved document set clarity

* string fixes

---------

Co-authored-by: pablodanswer <pablo@danswer.ai>
2024-08-25 01:10:24 +00:00
hagen-danswer
c1c35b00cb
Fixed slack bot auto filters for document sets () 2024-08-24 18:54:21 +00:00
Weves
7d201f67d4 Fix typing for custom tool response 2024-08-23 13:34:53 -07:00
pablodanswer
c152123ef4
alembic once again () 2024-08-23 05:28:13 +00:00
Chris Weaver
5cb9c17ddf
Add better logging for connectors ()
* Add better logging for connectors

* fix
2024-08-23 03:58:29 +00:00
Chris Weaver
b1302303b2
Add chat_session_id + message_pair_num () 2024-08-22 20:55:21 -07:00
pablodanswer
e89dc67e5d
Update embedding interface ()
* squash

* simplify interface

* some updates to typing

* cloud provider type

* update typing to be even clearer

* push local commit (squash)

* cleaner interfaces

* another quick pass

* squash

* cleaner alembic

* cleaner

* remove trailing whitespace

* add sequence

* quick circle back to double check

* update

* update naming

* update naming
2024-08-23 03:52:02 +00:00
hagen-danswer
c042a19c00
Curator role ()
* Added backend support for curator role

* modal refactor

* finalized first 2 commits

same as before

finally

what was it for

* added credential, cc_pair, and cleanup

mypy is super helpful hahahahahahahahahahahaha

* curator support for personas

* added connector management permission checks

* fixed the connector creation flow

* added document access to curator

* small cleanup added comments and started ui

* groups and assistant editor

* Persona frontend

* Document set frontend

* cleaned up the entire frontend

* alembic fix

* Minor fixes

* credentials section

* some credential updates

* removed logging statements

* fixed try catch

* fixed model name

* made everything happen in one db commit

* Final cleanup

* cleaned up fast code

* mypy/build fixes

* polish

* more token rate limit polish

* fixed weird credential permissions

* Addressed chris feedback

* addressed pablo feedback

* fixed alembic

* removed deduping and caching

* polish!!!!
2024-08-23 01:39:37 +00:00
Chris Weaver
99db27d989
Add metadata for simple doc () 2024-08-22 12:30:28 -07:00
pablodanswer
197b62aed1
Regenerate (branch of stop) ()
* add regenerate

* functional once again post rebase but quite ugly

* validated + cleaner UI

* more robust implementation for first messages

* squash

* remove parameter

* proper margin

* clarify for future programmers

* remove some logs

* self nit pick - smoother ux

* more self-nits

* stroke line cap

* rebase
2024-08-22 19:06:44 +00:00
Yuhong Sun
9d5db05e4b
Add Migration () 2024-08-22 10:44:42 -07:00