* refactor salesforce sqlite db access
* more refactoring
* refactor again
* refactor again
* rename object
* add finalizer to ensure db connection is always closed
* avoid unnecessarily nesting connections and commit regularly when possible
* remove db usage from csv download
* dead code
* hide deprecation warning in ddtrace
* remove unused param
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* debug script + slight refactor of db class
* better comments
* move setup logger
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
* friendlier handling of slack channel retrieval
* retry on downgrade_postgres deadlock
* fix comment
* text
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* Convert the model_names and display_model_names into a set instead
* Update backend/alembic/versions/7a70b7664e37_add_model_configuration_table.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Return default value instead of throwing error
* Add default parameter
* Move logic around
* Use dummy value for max_input_tokens in testing flow
* Remove unnecessary assignment
* tool to generate vespa schema variations for our cloud
* extraneous assign
* use a real templating system instead of search/replace
* fix float
* maybe this should be double
* remove redundant var
* template the other files
* try a spawned process
* move the wrapper
* fix args
* increase timeout
* run multitenant reset operations out of process as well
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
* add emails to retry with on 403
* attempted fix for connector test
* CW comments
* connector test fix
* test fixes and continue on 403
* fix tests
* fix tests
* fix concurrency tests
* fix integration tests with llmprovider eager loading
* Add multi text array field
* Add multiple values to model configuration for a custom LLM provider
* Fix reference to old field name
* Add migration
* Update all instances of model_names / display_model_names to use new schema migration
* Update background task
* Update endpoints to not throw errors
* Add test
* Update backend/alembic/versions/7a70b7664e37_add_models_configuration_table.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update backend/onyx/background/celery/tasks/llm_model_update/tasks.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix list comprehension nits
* Update web/src/components/admin/connectors/Field.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update web/src/app/admin/configuration/llm/interfaces.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Implement greptile recommendations
* Update backend/onyx/db/llm.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update backend/onyx/server/manage/llm/api.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update backend/onyx/background/celery/tasks/llm_model_update/tasks.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update backend/onyx/db/llm.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix more greptile suggestions
* Run formatter again
* Update backend/onyx/db/models.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add relationship to `LLMProvider` and `ModelConfigurations` classes
* Use sqlalchemy ORM relationships instead of manually populating fields
* Upgrade migration
* Update interface
* Remove all instances of model_names and display_model_names from backend
* Add more tests and fix bugs
* Run prettier
* Add types
* Update migration to perform data transformation
* Ensure native llm providers don't have custom max input tokens
* Start updating frontend logic to support custom max input tokens
* Pass max input tokens to LLM class (to be passed into `litellm.completion` call later)
* Add ModelConfigurationField component for custom llm providers
* Edit spacing and styling of model configuration matrix
* Fix error message displaying bug
* Edit opacity of `FiX` field for first index
* Change opacity back
* Change roundness
* Address comments on PR
* Perform fetching of `max_input_tokens` at the beginning of the callgraph and rope it throughout the entire callstack
* Change `add` to `execute`
* Move `max_input_tokens` into `LLMConfig`
* Fix bug with error messages not being cleared
* Change field used to fetch LLMProvider
* Fix model-configuration UI
* Address comments
* Remove circular import
* Fix failing tests in GH
* Fix failing tests
* Use `isSubset` instead of equality to determine native vs custom LLM Provider
* Remove unused import
* Make responses always display max_input_tokens
* Fix api endpoint to hit
* Update types in web application
* Update object field
* Fix more type errors
* Fix failing llm provider tests
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* tool to generate vespa schema variations for our cloud
* extraneous assign
* use a real templating system instead of search/replace
* fix float
* maybe this should be double
* remove redundant var
* template the other files
* try a spawned process
* move the wrapper
* fix args
* increase timeout
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
* tool to generate vespa schema variations for our cloud
* extraneous assign
* float, not double
* back to double
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* refactor a mega function for readability and make sure to increment retry_count on exception so that we don't infinitely loop
* improve session and page level context handling
* don't use pydantic for the session context
* we don't need retry success
* move playwright handling into the session context
* need to break on ok
* return doc from scrape
* fix comment
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* Fix duplicate kwarg issue
* Change how vertex_credentials are passed
* Modify temporary dict instead
* Change string to a global constant
* Add extra condition to if-check during population of map
