32 Commits

Author SHA1 Message Date
pablodanswer
20df20ae51
Multi tenant vespa (#2762)
* add vespa multi tenancy

* k

* formatting

* Billing (#2667)

* k

* data -> control

* nit

* nit: error handling

* auth + app

* nit: color standardization

* nit

* nit: typing

* k

* k

* feat: functional upgrading

* feat: add block for downgrading to seats < active users

* add auth

* remove accomplished todo + prints

* nit

* tiny nit

* nit: centralize security

* add tenant expulsion/gating + invite user -> increment billing seat no.

* add cloud configs

* k

* k

* nit: update

* k

* k

* k

* k

* nit
2024-10-12 23:53:11 +00:00
pablodanswer
f40c5ca9bd
Add tenant context (#2596)
* add proper tenant context to background tasks

* update for new session logic

* remove unnecessary functions

* add additional tenant context

* update ports

* proper format / directory structure

* update ports

* ensure tenant context properly passed to ee bg tasks

* add user provisioning

* nit

* validated for multi tenant

* auth

* nit

* nit

* nit

* nit

* validate pruning

* evaluate integration tests

* at long last, validated celery beat

* nit: minor edge case patched

* minor

* validate update

* nit
2024-10-10 16:34:32 +00:00
pablodanswer
d5b9a6e552
add vespa + embedding timeout env variables (#2689)
* add vespa + embedding timeout env variables

* nit: integration test

* add dangerous override

* k

* add additional clarity

* nit

* nit
2024-10-09 03:20:28 +00:00
pablodanswer
e56fd43ba6
cors update (#2686) 2024-10-05 23:08:28 +00:00
pablodanswer
df464fc54b
Allow for CORS Origin Setting (#2449)
* allow setting of CORS origin

* simplify

* add environment variable + rename

* slightly more efficient

* simplify so mypy doens't complain

* temp

* go back to my preferred formatting
2024-09-16 18:54:36 +00:00
pablodanswer
31ca6857fb
Custom Refresh on Client Side (#2376) 2024-09-13 00:04:03 -07:00
pablodanswer
3a9b964d5c
Add Litellm Rerank proxy (#2346)
* add ability ot set reranking litellm proxy

* add fully functional rerank litellm cards

* minor formatting enforcement

* remove logs
2024-09-09 15:57:01 +00:00
Shukant Pal
362156f97e
Model inference for connector classifier on queries (#2137) 2024-09-08 14:46:00 -07:00
pablodanswer
69c0419146
Updated refreshing (#2327)
* clean up + add environment variables

* remove log

* update

* update api settings

* somewhat cleaner refresh functionality

* fully functional

* update settings

* validated

* remove random logs

* remove unneeded paramter + log

* move to ee + remove comments

* Cleanup unused

---------

Co-authored-by: Weves <chrisweaver101@gmail.com>
2024-09-06 04:36:55 +00:00
pablodanswer
34ba3181ff
Update auth for litellm proxy (#2316)
* update for auth

* validated embedding model names

* remove embedding provider

* remove logs

* add ability to delete search setting

* add abiility to delete models + more streamlined API endpoints

* remove upsert

* minor typing fix

* add connector utils
2024-09-04 20:59:07 +00:00
pablodanswer
299cb5035c
Add litellm proxy embeddings (#2291)
* add litellm proxy

* formatting

* move `api_url` to cloud provider + nits

* remove log

* typing

* quick tuyping fix

* update LiteLLM selection logic

* remove logs + validate functionality

* rename proxy var

* update path casing

* remove pricing for custom models

* functional values
2024-09-02 09:08:35 -07:00
josvdw
50c17438d5
Litellm bump (#2195)
* ran bump-pydantic

* replace root_validator with model_validator

* mostly working. some alternate assistant error. changed root_validator and typing_extensions

* working generation chat. changed type

* replacing .dict with .model_dump

* argument needed to bring model_dump up to parity with dict()

* fix a fewremaining issues -- working with llama and gpt

* updating requirements file

* more requirement updates

* more requirement updates

* fix to make search work

* return type fix:

* half way tpyes change

* fixes for mypy and pydantic:

* endpoint fix

* fix pydantic protected namespaces

* it works!

* removed unecessary None initializations

* better logging

* changed default values to empty lists

* mypy fixes

* fixed array defaulting

---------

Co-authored-by: hagen-danswer <hagen@danswer.ai>
2024-08-28 00:00:27 +00:00
pablodanswer
97ba71e1b3
Db search (#2235)
* k

* update enum imports

* add functional types + model swaps

* remove a log

* remove kv

* fully functional + robustified for kv swap

* validated with hosted + cloud

* ensure not updating current search settings when reindexing

* add instance check

* revert back to updating search settings (will need a slight refactor for endpoint)

* protect advanced config override1

* run pretty

* fix typing

* update typing

* remove unnecessary function

* update model name

* clearer interface names

* validated foreign key constaint

* proper migration

* squash

---------

Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-08-27 04:26:51 +00:00
Weves
c44c22a009 Fix model server 2024-08-19 07:23:24 -07:00
Yuhong Sun
5ab4d94d94
Logging Level Update (#2165) 2024-08-18 21:53:40 -07:00
Yuhong Sun
119aefba88
Add log files to containers (#2164) 2024-08-18 19:18:28 -07:00
pablodanswer
22573aba2a
Improve Search (#2105) 2024-08-16 21:29:15 -07:00
Yuhong Sun
386b229ed3
Cohere Rerank (#2109) 2024-08-11 14:22:42 -07:00
Yuhong Sun
ce666f3320
Propagate Embedding Enum (#2108) 2024-08-11 12:17:54 -07:00
Yuhong Sun
d60fb15ad3
Allowing users to set Search Settings (#2106) 2024-08-10 20:48:58 -07:00
Yuhong Sun
8cd1eda8b1
Rework Rerankers (#2093) 2024-08-08 21:33:49 -07:00
Yuhong Sun
7dcc42aa95
Intent Model Update (#2069) 2024-08-08 20:45:53 -07:00
Weves
51731ad0dd Fix issue where large docs/batches break openai embedding 2024-08-02 01:07:09 -07:00
Yuhong Sun
036d5c737e
No Null Embeddings (#1982) 2024-07-30 19:54:49 -07:00
pablodanswer
48a0d29a5c
Fix empty / reverted embeddings (#1910) 2024-07-23 22:41:31 -07:00
hagen-danswer
eb3e7610fc
Added retries and multithreading for cloud embedding (#1879)
* added retries and multithreading for cloud embedding

* refactored a bit

* cleaned up code

* got the errors to bubble up to the ui correctly

* added exceptin printing

* added requirements

* touchups

---------

Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-07-20 22:10:18 -07:00
Yuhong Sun
bd1e0c5969
Add Enum File (#1842) 2024-07-15 09:13:27 -07:00
Weves
6fe3eeaa48 Fix model serer startup 2024-07-14 23:33:58 -07:00
pablodanswer
e7f81d1688
add third party embedding models (#1818) 2024-07-14 10:19:53 -07:00
Weves
cc9cb202cd Add option to specify custom port for indexing model server 2024-05-21 11:08:10 -07:00
Yuhong Sun
b59912884b
Fix Model Server (#1320) 2024-04-10 23:13:22 -07:00
Yuhong Sun
2db906b7a2
Always Use Model Server (#1306) 2024-04-07 21:25:06 -07:00