e399eeb014
fix: Query History Export ( #4841 )
...
* Move task registration to earlier in the API
* Remove unnecessary check
2025-06-09 10:10:53 -07:00
b133e8fcf0
enforce sub_question ordering for chat_message ( #4848 )
2025-06-09 02:37:35 +00:00
ba9b24a477
Enhance credential management with multi-auth support and improved validation ( #4846 )
...
Co-authored-by: Subash <subash@onyx.app >
2025-06-09 00:53:18 +00:00
cc7fb625a6
KG autofill metadata ( #4834 )
...
* feat: autofill metadata
* fix: typo
* fix: enum
* fix: nit
2025-06-08 21:56:50 +00:00
2b812b7d7d
Kg batch clustering ( #4847 )
...
* super genius kg_entity parent migration
* feat: batched clustering
* fix: nit
2025-06-08 21:16:10 +00:00
c5adbe4180
Knowledge Graph v1 ( #4626 )
...
* db setup
* transfer 1 - incomplete
* more adjustments
* relationship table + query update
* temp view creation
* restructuring
* nits
* updates
* separate read_only engine
* extraction revamp
* focus on metadata relatonships 1
* dev
* migration downgrade fix
* rebase migration change
* a3+
* progress
* base
* new extraction
* progress
* fixed KG extraction
* nits
* updates
* simplifications & cleanup
* fixes
* updates
* more feature flag checks
* fixes
* extraction process fix
* read-only user creation as part of setup
* fix for missing entity attributes
* kg read-only user creation as part of migration
* typo
* EL initial comments
* initial Account/SF Connector chnges
* SF Connector update
- include account information
* base w/ salesforce
* evan updates + quite a bit more
* kg-filtered search
* EL changes pt 2
* migrations and env vars
* quick migration fix
* migration update
* post_rebase fixes
* mypy fixes
* test fixes
* test fix
* test fix
* read_only pool + misc
* nf
* env vars
* test improvements
* salesforce fix
* test update
* small changes
* small adjustments
* SF Connector fix & kg_stage removal for one table
* mypy fix
* small fixes
* EL + RK (pt 1) comments
* nit
* setting updated
* Salesforce test update
* EL comments
* read-only user replacement & cleanup
* SQL View fix
* converting entity type-name separators
* sql view group ownership
* view fix
* SQL tweak
* dealing with docs that were skipped by indexing
* increased error handling
* more error handling
* Output formatting fix
* kg-incremental-reindexing
* 0-doc found improvement
* celery
* migration correction
* timeout adjustments
* nit
* Updated migration
* Entity Normalization for KG Dev 1 (#4746 )
* feat: trigrams column
* fix: reranking and db
* feat: v1
* fix: convert to orm
* feat: parallel
* fix: default to id_name
* fix: renamed semantic_id and semantic_id_trigrams
* fix: scalar subquery
* fix: tuning + redundancy
* fix: threshold
* fix: typo
* fix: shorten names
* wip
* fix: reverted
* feat: config
* feat: works but it was dumb
* feat: clustering works
* fix: mypy
* normalization <-> language awareness for SQL generation
* small type fixes
---------
Co-authored-by: joachim-danswer <joachim@danswer.ai >
* mypy
* typo and dead code
* kg_time_fencing
* feat: remove temp views on migration downgrade
* remove functions and triggers for now
* rebase adjustments
* EL code review results
* quick fix + trigger/funcs for single tenant
* fix: typo, mypy, dead code
* fix: autoflake
* small updatesd
* nit
* fix: typo
* early + faster view creation
* Extension creation in MT migration
* nit changes to default ETs
* Incremental Clustering and KG Refactor V1 (#4784 )
Optimized/restructured incremental clustering. New pipeline actually that moves vespa updates to clustering.
Also, celery configuration has been updated.
---------
Co-authored-by: joachim-danswer <joachim@danswer.ai >
* prompt tweak & ET extraction reset
* more general hierarchical structure
* feat: better vespa reset logic
* prompt optimization and entity replacemants
* small prompt changes
* KG Refactor V2 (#4814 )
Clustering & Extraction improvements & various nits
Co-authored-by: joachim-danswer <joachim@danswer.ai >
* add connector-level coverage days
* fix: nit
* initial EL responses
* refactor: helper functions for formatting
* fix: more helper fns & comments
* fix: comment code that's been implemented elsewhere
* fix: tenant_id missing arg
* fix: removed debugging stuff
* fix: moved kg_interactions db query to helper fn
* fix: tenant_id
* fix: tenant_id & removed outdated helper fn
* fix always set entity class
* fix: typo
* fix alembic heads
* fix: celery logging
* fix: migrations fix
* fix: multi tenant permissions
* fix: temp connector fix
* fix: downgrade
* Fix upgrade migration
* fix: tenant for normalization
* added additional acl
* stray EL comments
* fix: connector test
* fix mypy
* fix: temporary connector test fix
* fix: jira connector test
* nit
* small nits
* fix: black
* fix: mypy
* fix: mypy
---------
Co-authored-by: Rei Meguro <36625832+Orbital-Web@users.noreply.github.com >
2025-06-07 23:14:20 +00:00
21dc3a2456
Restart script clarity ( #4839 )
...
* Add error clarity to restart containers script
* erroneous cleanup on exit
* fix when starting containers for the first time
---------
Co-authored-by: Wenxi Onyx <wenxi-onyx@Wenxis-MacBook-Pro.local >
2025-06-06 09:11:58 -07:00
9631f373f0
Restart script clarity ( #4837 )
...
* Add error clarity to restart containers script
* erroneous cleanup on exit
* space
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: Wenxi Onyx <wenxi-onyx@Wenxis-MacBook-Pro.local >
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-05 19:24:06 -07:00
dc4b9bc003
Fixed indexing when no sites are specified ( #4822 )
...
* Fixed indexing when no sites are specificed
* Added test for Sharepoint all sites index
* Accounted for paginated results.
* Typing
* Typing
---------
Co-authored-by: Wenxi Onyx <wenxi-onyx@Wenxis-MacBook-Pro.local >
2025-06-05 23:25:20 +00:00
affb9e6941
Extend the onyx_vespa_schemas.py script ( #4835 )
2025-06-05 22:47:32 +00:00
dc542fd7fa
Enable default quantization ( #4815 )
...
* Adjust migration
* update default in form
* Add cloud indices for bfloat16
* Update backend/shared_configs/configs.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update vespa schema gen script
* Move embedding configs
* Remove unused imports
* remove import from shared configs
* Remove unused model
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-05 14:02:08 -07:00
85eeb21b77
add slack percentage progress ( #4809 )
...
* add percentage progress
* range checking
* formatting
* for new channels, skip them if the most recent messages are all from bots
* comments
* bypass bot channels
* code review
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-06-05 17:52:46 +00:00
4bb3ee03a0
Update GitHub Connector metadata ( #4769 )
...
* feat: updated github metadata
* feat: nullity check
* feat: more metadata
* feat: userinfo
* test: connector test + more metadata
* feat: num files changed
* feat str
* feat: list of str
2025-06-04 18:33:14 +00:00
1bb23d6837
Upgrade asyncpg for Python 3.12 ( #4699 )
2025-06-04 11:44:52 -07:00
f447359815
bump up agent timeouts across the board ( #4821 )
2025-06-04 14:36:46 +00:00
094cc940a4
Small embedding model cleanups ( #4820 )
...
* Small embedding model cleanups
* fix
* address greptile
* fix build
2025-06-04 00:10:44 +00:00
80ecdb711d
New metadata for Jira for KG ( #4785 )
...
* new metadata components
* nits & tests
2025-06-03 20:12:56 +00:00
a599176bbf
Improve reasoning detection ( #4817 )
...
* Improve reasoning detection
* Address greptile comments
* Fix mypy
2025-06-03 20:01:12 +00:00
4c93fd448f
fix: updating gemini models ( #4806 )
...
Co-authored-by: Jeffrey Drakos <jeffreydrakos@Jeffreys-MacBook-Pro-2.local >
2025-06-03 11:16:42 -07:00
84d916e210
Fix hard delete of agentic chats ( #4803 )
...
* Fix hard delete of agentic chats
* Update backend/tests/integration/tests/chat/test_chat_deletion.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Address Greptile comments
* fix tests
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-03 11:14:11 -07:00
f57ed2a8dd
Adjust script
2025-06-02 18:39:00 -07:00
713889babf
[Connectors][Script] Resume Paused Connectors ( #4798 )
...
* [Connectors][Script] Resume Paused Connectors
* Addressing comment
2025-06-02 18:34:00 -07:00
58c641d8ec
Remove ordering-only flow
2025-06-02 18:29:42 -07:00
94985e24c6
Adjust user file access
2025-06-02 17:28:49 -07:00
4c71a5f5ff
drive perm sync logs + misc deployment improvements ( #4788 )
...
* some logs
* give postgress more memory
* give postgress more memory
* give postgress more memory
* revert
* give postgress more memory
* bump external access limit
* vespa timeout
* deployment consistency
* bump vespa version
* skip upgrade check
* retry permission by ids
* logs
* fix temp docx file issue
* fix drive file deduping
* RK comments
* mypy
* aggregate logs
2025-06-01 23:36:57 +00:00
b19e3a500b
try fixing slack bot ( #4792 )
...
* try fixing slack bot
* add logging
* just use if
* safe msg get
* .close isn't async
* enforce block list size limit
* various fixes and notes
* don't use self
* switch to punkt_tab
* fix return condition
* synchronize waiting, use non thread local redis locks
* fix log format, make collection copy more explicit for readability
* fix some logging
* unnecessary function
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-31 00:39:14 +00:00
267fe027f5
Fix failed docs table ( #4800 )
...
* Fix initial LLM provider set up
* Fix IndexAttemptErrorsModal pagination
2025-05-30 22:19:52 +00:00
0d4d8c0d64
jira daylight savings handling ( #4797 )
2025-05-30 19:13:38 +00:00
6f9d8c0cff
Simplify passing in of file IDs for filtering ( #4791 )
...
* Simplify passing in of file IDs for filtering
* Address RK comments
2025-05-30 05:08:21 +00:00
5031096a2b
Fix frozen add token rate limit migration
2025-05-29 22:22:36 -07:00
797e113000
add a comment ( #4789 )
...
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-29 14:11:19 -07:00
ef4d5dcec3
new slack rate limiting approach ( #4779 )
...
* fix slack rate limit retry handler for groups
* trying to mitigate memory usage during csv download
* Revert "trying to mitigate memory usage during csv download"
This reverts commit 48262eacf6rkuo@onyx.app >
2025-05-29 19:49:32 +00:00
0b5e3e5ee4
skip excel files that openpyxl fails on ( #4787 )
2025-05-29 18:09:46 +00:00
9f72826143
Bugfix/slack bot debugging ( #4782 )
...
* adding some logging
* better var name
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-28 18:43:11 +00:00
16a14bac89
Feature/tenant reporting 2 ( #4750 )
...
* add more info
* fix headers
* add filename as param (merge)
* db manager entry in launch template
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-27 23:24:47 +00:00
0b01d7f848
refactor: stream_llm_answer ( #4772 )
...
* refactor: stream_llm_answer
* fix: lambda
* fix: mypy, docstring
2025-05-26 22:29:33 +00:00
23ff3476bc
print sanitized api key to help troubleshoot ( #4764 )
...
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-24 22:27:37 +00:00
0c7ba8e2ac
Fix/add back search with files ( #4767 )
...
* Allow search w/ user files
* more
* More
* Fix
* Improve prompt
* Combine user files + regular uploaded files
2025-05-24 15:44:39 -07:00
dad99cbec7
v1 refresh drive creds during perm sync ( #4768 )
2025-05-23 23:01:26 +00:00
3e78c2f087
Fix POSTGRES_IDLE_SESSIONS_TIMEOUT ( #4765 )
2025-05-23 14:55:23 -07:00
b824951c89
add probe signals for beat ( #4760 )
...
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-23 01:41:11 +00:00
ca20e527fc
fix tool calling for bedrock claude models ( #4761 )
...
* fix tool calling for bedrock claude models
* unit test
* fix unit test
2025-05-23 01:13:18 +00:00
c8e65cce1e
add k8s probes ( #4752 )
...
* add file signals to celery workers
* improve probe script
* cancel tref
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-22 20:21:59 +00:00
6c349687da
improve impersonation logging slightly ( #4758 )
...
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-22 11:17:27 -07:00
9dbe12cea8
Feat: Search Eval Testing Overhaul (provide ground truth, categorize query, etc.) ( #4739 )
...
* fix: autoflake & import order
* docs: readme
* fix: mypy
* feat: eval
* docs: readme
* fix: oops forgot to remove comment
* fix: typo
* fix: rename var
* updated default config
* fix: config issue
* oops
* fix: black
* fix: eval and config
* feat: non tool calling query mod
2025-05-21 19:25:10 +00:00
e78637d632
mitigate memory usage during csv download ( #4745 )
...
* mitigate memory usage during csv download
* more gc tweaks
* missed some small changes
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-21 00:44:27 +00:00
cac03c07f7
v1 answer refactor ( #4721 )
...
* v1 answer refactor
* fix tests
* good catch, tests
* more cleanup
2025-05-20 23:34:27 +00:00
95dabfaa18
fix: Add back Teams' replies processing ( #4744 )
...
* Add replies to document construction and edit tests
* Update tests
* Add replies processing to teams
* Fix test
* Add try-except block around potential failure
* Update entity-id during ConnectorFailure raise
2025-05-20 22:55:28 +00:00
e92c418e0f
Feature/openapi ( #4710 )
...
* starting openapi support
* fix app / app_fn
* send gitignore
* dedupe function names
* add readme
* modify gitignore
* update launch template
* fix unused path param
* fix mypy
* local tests pass
* first pass at making integration tests work
* fixes
* fix script path
* set python path
* try full path
* fix output dir
* fix integration test
* more build fixes
* add generated directory
* use the config
* add a comment
* add
* modify tsconfig.json
* fix index linting bugs
* tons of lint fixes
* new gitignore
* remove generated dir
* add tasks template
* check for undefined explicitly
* fix hooks.ts
* refactor destructureValue
* improve readme
---------
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app >
2025-05-20 21:33:18 +00:00
0593d045bf
Fix ext_perm_user sign-up for non multi-tenant ( #4743 )
...
* OAuth w/ external user fix
* Apply to basic auth as well
2025-05-20 20:17:01 +00:00
