rkuo-danswer
870b59a1cc
Bugfix/vertex crash ( #4181 )
...
* Update text embedding model to version 005 and enhance embedding retrieval process
* re
* Fix formatting issues
* Add support for Bedrock reranking provider and AWS credentials handling
* fix: improve AWS key format validation and error messages
* Fix vertex embedding model crash
* feat: add environment template for local development setup
* Add display name for Claude 3.7 Sonnet model
* Add display names for Gemini 2.0 models and update Claude 3.7 Sonnet entry
* Fix ruff errors by ensuring lines are within 130 characters
* revert to currently default onyx browser settings
* add / fix boto requirements
---------
Co-authored-by: ferdinand loesch <f.loesch@sportradar.com>
Co-authored-by: Ferdinand Loesch <ferdinandloesch@me.com>
Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2025-03-05 01:59:46 +00:00
Chris Weaver
f25e1e80f6
Add option to not re-index ( #4157 )
...
* Add option to not re-index
* Add quantizaton / dimensionality override support
* Fix build / ut
2025-03-03 10:54:11 -08:00
pablonyx
a98dcbc7de
Update tenant logic ( #4122 )
...
* k
* k
* k
* quick nit
* nit
2025-02-26 03:53:46 +00:00
pablonyx
47fd4fa233
Strict Tenant ID Enforcement ( #3871 )
...
* strict tenant id enforcement
* k
* k
* nit
* merge
* nit
* k
2025-02-19 00:52:56 +00:00
Chris Weaver
a5d2f0d9ac
Fix airtable connector w/ mt cloud + move telem logic to match new st… ( #3868 )
...
* Fix airtable connector w/ mt cloud + move telem logic to match new standard
* Address Greptile comment
* Small fixes/improvements
* Revert back monitoring frequency
* Small monitoring fix
2025-01-31 16:29:04 -08:00
Richard Kuo
174e5968f8
we didn't want to rename the alt index suffix, reverting
2024-12-31 11:28:11 -08:00
pablodanswer
21ec5ed795
welcome to onyx
2024-12-13 09:56:10 -08:00
Weves
18e63889b7
Change default log level back to info
2024-12-06 10:07:14 -08:00
pablodanswer
03e2789392
Text embedding (PDF, TXT) ( #3113 )
...
* add text embedding
* post rebase cleanup
* fully functional post rebase
* rm logs
* rm '
* quick clean up
* k
2024-12-02 22:43:53 +00:00
pablodanswer
7c841051ed
Cohere ( #3111 )
...
* add cohere default
* finalize
* minor improvement
* update
* update
* update configs
* ensure we properly expose name(space) for slackbot
* update config
* config
2024-11-14 01:58:54 +00:00
pablodanswer
facf1d55a0
Cloud improvements ( #3099 )
...
* add improved cloud configuration
* fix typing
* finalize slackbot improvements
* minor update
* finalized keda
* moderate slackbot switch
* update some configs
* revert
* include reset engine!
2024-11-13 23:52:52 +00:00
pablodanswer
9272d6ebfe
Remove ee ( #3093 )
...
* move api key to non-ee
* finalize previous migration
* move token rate limit to non-ee
* general cleanup
* update
* update
* finalize
* finalize
* ensure callable
* k
2024-11-09 20:51:36 +00:00
pablodanswer
e4bb14d4e1
Super user ( #2944 )
...
* add super user
* nits
2024-11-02 17:29:23 +00:00
pablodanswer
da3c5e3711
Feat: add clean logging for api routes ( #2928 )
...
* feat: add clean logging for api routes
* nit
* `MULTI_TENANT` must be shared config
* nit
2024-10-27 05:15:41 +00:00
pablodanswer
0545fb4443
Multitenant redis update ( #2889 )
...
* add multi tenancy to redis
* rename context var
* k
* args -> kwargs
* minor update to kv interface
* robustify
2024-10-24 02:12:25 +00:00
pablodanswer
14e75bbd24
add default schema config ( #2888 )
...
* add default schema config
* resolve circular import
* k
2024-10-23 23:12:17 +00:00
Chris Weaver
36134021c5
Refactor + add global timeout env variable ( #2844 )
...
* Refactor + add global timeout env variable
* remove model
* mypy
* Remove unused
2024-10-18 18:25:27 +00:00
pablodanswer
61424de531
add sentry ( #2786 )
...
* add sentry
* nit
* nit
* add requirement to ee
* try to ensure sentry is installed in integration tests
2024-10-17 23:20:37 +00:00
pablodanswer
a385234c0e
Parsing ( #2734 )
...
* k
* update chunking limits
* nit
* nit
* clean up types
* nit
* validate
* k
2024-10-16 16:44:19 +00:00
pablodanswer
e022e77b6d
Simpler azure embedding ( #2751 )
...
* functional but janky
* nit
* adapt for azure
* nit
* minor updates
* nits
* nit
* nit
* ensure access to litellm
* k
2024-10-15 23:23:11 +00:00
pablodanswer
20df20ae51
Multi tenant vespa ( #2762 )
...
* add vespa multi tenancy
* k
* formatting
* Billing (#2667 )
* k
* data -> control
* nit
* nit: error handling
* auth + app
* nit: color standardization
* nit
* nit: typing
* k
* k
* feat: functional upgrading
* feat: add block for downgrading to seats < active users
* add auth
* remove accomplished todo + prints
* nit
* tiny nit
* nit: centralize security
* add tenant expulsion/gating + invite user -> increment billing seat no.
* add cloud configs
* k
* k
* nit: update
* k
* k
* k
* k
* nit
2024-10-12 23:53:11 +00:00
pablodanswer
f40c5ca9bd
Add tenant context ( #2596 )
...
* add proper tenant context to background tasks
* update for new session logic
* remove unnecessary functions
* add additional tenant context
* update ports
* proper format / directory structure
* update ports
* ensure tenant context properly passed to ee bg tasks
* add user provisioning
* nit
* validated for multi tenant
* auth
* nit
* nit
* nit
* nit
* validate pruning
* evaluate integration tests
* at long last, validated celery beat
* nit: minor edge case patched
* minor
* validate update
* nit
2024-10-10 16:34:32 +00:00
pablodanswer
d5b9a6e552
add vespa + embedding timeout env variables ( #2689 )
...
* add vespa + embedding timeout env variables
* nit: integration test
* add dangerous override
* k
* add additional clarity
* nit
* nit
2024-10-09 03:20:28 +00:00
pablodanswer
e56fd43ba6
cors update ( #2686 )
2024-10-05 23:08:28 +00:00
pablodanswer
df464fc54b
Allow for CORS Origin Setting ( #2449 )
...
* allow setting of CORS origin
* simplify
* add environment variable + rename
* slightly more efficient
* simplify so mypy doens't complain
* temp
* go back to my preferred formatting
2024-09-16 18:54:36 +00:00
pablodanswer
31ca6857fb
Custom Refresh on Client Side ( #2376 )
2024-09-13 00:04:03 -07:00
pablodanswer
3a9b964d5c
Add Litellm Rerank proxy ( #2346 )
...
* add ability ot set reranking litellm proxy
* add fully functional rerank litellm cards
* minor formatting enforcement
* remove logs
2024-09-09 15:57:01 +00:00
Shukant Pal
362156f97e
Model inference for connector classifier on queries ( #2137 )
2024-09-08 14:46:00 -07:00
pablodanswer
69c0419146
Updated refreshing ( #2327 )
...
* clean up + add environment variables
* remove log
* update
* update api settings
* somewhat cleaner refresh functionality
* fully functional
* update settings
* validated
* remove random logs
* remove unneeded paramter + log
* move to ee + remove comments
* Cleanup unused
---------
Co-authored-by: Weves <chrisweaver101@gmail.com>
2024-09-06 04:36:55 +00:00
pablodanswer
34ba3181ff
Update auth for litellm proxy ( #2316 )
...
* update for auth
* validated embedding model names
* remove embedding provider
* remove logs
* add ability to delete search setting
* add abiility to delete models + more streamlined API endpoints
* remove upsert
* minor typing fix
* add connector utils
2024-09-04 20:59:07 +00:00
pablodanswer
299cb5035c
Add litellm proxy embeddings ( #2291 )
...
* add litellm proxy
* formatting
* move `api_url` to cloud provider + nits
* remove log
* typing
* quick tuyping fix
* update LiteLLM selection logic
* remove logs + validate functionality
* rename proxy var
* update path casing
* remove pricing for custom models
* functional values
2024-09-02 09:08:35 -07:00
josvdw
50c17438d5
Litellm bump ( #2195 )
...
* ran bump-pydantic
* replace root_validator with model_validator
* mostly working. some alternate assistant error. changed root_validator and typing_extensions
* working generation chat. changed type
* replacing .dict with .model_dump
* argument needed to bring model_dump up to parity with dict()
* fix a fewremaining issues -- working with llama and gpt
* updating requirements file
* more requirement updates
* more requirement updates
* fix to make search work
* return type fix:
* half way tpyes change
* fixes for mypy and pydantic:
* endpoint fix
* fix pydantic protected namespaces
* it works!
* removed unecessary None initializations
* better logging
* changed default values to empty lists
* mypy fixes
* fixed array defaulting
---------
Co-authored-by: hagen-danswer <hagen@danswer.ai>
2024-08-28 00:00:27 +00:00
pablodanswer
97ba71e1b3
Db search ( #2235 )
...
* k
* update enum imports
* add functional types + model swaps
* remove a log
* remove kv
* fully functional + robustified for kv swap
* validated with hosted + cloud
* ensure not updating current search settings when reindexing
* add instance check
* revert back to updating search settings (will need a slight refactor for endpoint)
* protect advanced config override1
* run pretty
* fix typing
* update typing
* remove unnecessary function
* update model name
* clearer interface names
* validated foreign key constaint
* proper migration
* squash
---------
Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-08-27 04:26:51 +00:00
Weves
c44c22a009
Fix model server
2024-08-19 07:23:24 -07:00
Yuhong Sun
5ab4d94d94
Logging Level Update ( #2165 )
2024-08-18 21:53:40 -07:00
Yuhong Sun
119aefba88
Add log files to containers ( #2164 )
2024-08-18 19:18:28 -07:00
pablodanswer
22573aba2a
Improve Search ( #2105 )
2024-08-16 21:29:15 -07:00
Yuhong Sun
386b229ed3
Cohere Rerank ( #2109 )
2024-08-11 14:22:42 -07:00
Yuhong Sun
ce666f3320
Propagate Embedding Enum ( #2108 )
2024-08-11 12:17:54 -07:00
Yuhong Sun
d60fb15ad3
Allowing users to set Search Settings ( #2106 )
2024-08-10 20:48:58 -07:00
Yuhong Sun
8cd1eda8b1
Rework Rerankers ( #2093 )
2024-08-08 21:33:49 -07:00
Yuhong Sun
7dcc42aa95
Intent Model Update ( #2069 )
2024-08-08 20:45:53 -07:00
Weves
51731ad0dd
Fix issue where large docs/batches break openai embedding
2024-08-02 01:07:09 -07:00
Yuhong Sun
036d5c737e
No Null Embeddings ( #1982 )
2024-07-30 19:54:49 -07:00
pablodanswer
48a0d29a5c
Fix empty / reverted embeddings ( #1910 )
2024-07-23 22:41:31 -07:00
hagen-danswer
eb3e7610fc
Added retries and multithreading for cloud embedding ( #1879 )
...
* added retries and multithreading for cloud embedding
* refactored a bit
* cleaned up code
* got the errors to bubble up to the ui correctly
* added exceptin printing
* added requirements
* touchups
---------
Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
2024-07-20 22:10:18 -07:00
Yuhong Sun
bd1e0c5969
Add Enum File ( #1842 )
2024-07-15 09:13:27 -07:00
Weves
6fe3eeaa48
Fix model serer startup
2024-07-14 23:33:58 -07:00
pablodanswer
e7f81d1688
add third party embedding models ( #1818 )
2024-07-14 10:19:53 -07:00
Weves
cc9cb202cd
Add option to specify custom port for indexing model server
2024-05-21 11:08:10 -07:00
