271 Commits

Author SHA1 Message Date
rkuo-danswer
261150e81a
Validate permission locks (#3799)
* WIP for external group sync lock fixes

* prototyping permissions validation

* validate permission sync tasks in celery

* mypy

* cleanup and wire off external group sync checks for now

* add active key to reset

* improve logging

* reset on payload format change

* return False on exception

* missed a return

* add count of tasks scanned

* add comment

* better logging

* add return

* more return

* catch payload exceptions

* code review fixes

* push to restart test

---------

Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2025-01-31 17:33:07 +00:00
pablonyx
5e21dc6cb3
Optimize /persona query (#3859)
* k

* delete

* k
2025-01-30 23:20:19 +00:00
Weves
39b3a503b4 Add more group sync logging 2025-01-30 14:42:14 -08:00
pablonyx
9bdb581220
Update slack configs (#3776)
* update

* fix build
2025-01-28 21:10:09 +00:00
rkuo-danswer
fa78f50fe3
Bugfix/celery ignore result (#3770)
* try using a redis replica in some areas

* harden up replica usage

* ignore results

---------

Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2025-01-27 08:53:01 +00:00
pablonyx
7f10494bbe
Better vespa interface (#3781)
* k

* much cleaner vespa util class

* log

* typing

* improvement

* improve
2025-01-26 21:22:44 +00:00
Chris Weaver
6551d6bc87
Add support for overridding scopes for OIDC (#3759) 2025-01-23 21:20:34 -08:00
Richard Kuo (Danswer)
ba771483d8 missed ee_tasks_to_schedule declaration 2025-01-23 12:32:43 -08:00
rkuo-danswer
a2d8e815f6
Feature/more celery fanout (#3740)
* WIP

* migrate most beat tasks to fan out strategy

* fix kwargs

* migrate EE tasks

* lock on the task_name level

* typo fix

---------

Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2025-01-23 19:08:42 +00:00
hagen-danswer
b9c29f2a36
Fix pagination for index attempts table DAN-1284 (#3722)
* Fix pagination for index attempts table

* fixed index attempts pagination

* fixed query history table

* query clearnup

* fixed test

* fixed weird tests???
2025-01-22 01:51:16 +00:00
hagen-danswer
b25668c83a
fixed group sync to account for changes in drive permissions (#3666)
* fixed group sync to account for changes in drive permissions

* mypy

* addressed

* reeeeeeeee
2025-01-19 00:08:50 +00:00
hagen-danswer
896e716d02
query history pagination tests (#3700)
* dummy pr

* Update prompts.yaml

* fixed tests and added query history pagination test

* done

* fixed

* utils!
2025-01-18 21:28:03 +00:00
skylares
af953ff8a3
Paginate Query History table (#3592)
* Add pagination for query history table

* Fix method name

* Fix mypy
2025-01-17 15:31:42 -08:00
rkuo-danswer
6fc52c81ab
Bugfix/beat redux (#3639)
* WIP

* WIP

* try spinning out check for indexing into a system task

* check for the correct delimiter

* use constants

---------

Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
2025-01-17 20:59:43 +00:00
hagen-danswer
1ad2128b2a
Combined Persona and Prompt API (#3690)
* Combined Persona and Prompt API

* quality

* added tests

* consolidated models and got rid of redundant fields

* tenant appreciation day

* reverted default
2025-01-17 20:21:20 +00:00
Chris Weaver
8a4d762798
Fix follow ups in thread + fix user name (#3686)
* Fix follow ups in thread + fix user name

* Add back single history str

* Remove newline
2025-01-16 02:40:25 +00:00
hagen-danswer
b1957737f2
refactored _add_user_filter usage (#3674)
* refactored db.connector_credential_pair

* Rerfactored the db.credentials user filtering

* the restr
2025-01-14 23:35:52 +00:00
Richard Kuo (Danswer)
d8aa21ca3a fix missed var names 2025-01-13 14:32:26 -08:00
Chris Weaver
9ea2ae267e
Performance monitoring (#3658)
* Initial scaffolding for metrics

* iterate

* more

* More metrics + SyncRecord concept

* Add indices, standardize timing

* Small cleanup

* Address comments
2025-01-13 12:36:45 -08:00
hagen-danswer
6afd27f9c9
fix group sync name capitalization (#3653)
* fix group sync name capitalization

* everything is lowercased now

* comments

* Added test for be2ab2aa50ee migration

* polish
2025-01-10 16:51:33 -08:00
pablonyx
cab7e60542
Proper anonymous user restricting (#3645) 2025-01-10 11:31:11 -08:00
hagen-danswer
50131ba22c Better logging for confluence space permissions 2025-01-09 15:13:02 -08:00
hagen-danswer
c55de28423
added distinct when outer joining for user filters (#3641)
* added distinct when outer joining for user filters

* Added distinct when outer joining for user filters for all
2025-01-09 14:15:38 -08:00
hagen-danswer
d40fd82803
Conf doc sync improvements (#3643)
* Reduce number of requests to Confluence

* undo

* added a way to dynamically adjust the pagination limit

* undo
2025-01-09 12:56:56 -08:00
hagen-danswer
a95f4298ad
Improved logging for confluence calls (#3622)
* Improved logging for confluence calls

* cleanup

* idk

* combined logging
2025-01-07 21:53:08 +00:00
pablonyx
d9e9c6973d
Multitenant anonymous (#3595)
* anonymous users for multi tenant setting

* nit

* k
2025-01-07 02:57:20 +00:00
hagen-danswer
e329b63b89
Added Permission Syncing for Salesforce (#3551)
* Added Permission Syncing for Salesforce

* cleanup

* updated connector doc conversion

* finished salesforce permission syncing

* fixed connector to batch Salesforce queries

* tests!

* k

* Added error handling and check for ee and sync type for postprocessing

* comments

* minor touchups

* tested to work!

* done

* my pie

* lil cleanup

* minor comment
2025-01-07 00:37:03 +00:00
pablonyx
f16ca1b735
minor auth fix (#3613) 2025-01-06 17:51:02 +00:00
pablonyx
c8090ab75b
Auth fix + Registration Clarity (#3590)
* clarify auth flow

* k

* nit

* k

* fix typing
2025-01-06 02:17:45 +00:00
hagen-danswer
e100a5e965 Properly account for anonymous access in Confluence 2025-01-05 18:01:39 -08:00
pablonyx
ffec19645b
JWT -> Redis (#3574)
* functional v1

* functional logout

* minor clean up

* quick clean up

* update configuration

* ni

* nit

* finalize

* update login page

* delete unused import

* quick nit

* updates

* clean up

* ni

* k

* k
2025-01-04 19:35:43 +00:00
pablonyx
6c018cb53f
add personal assistant usage stats (#3543) 2025-01-04 18:38:41 +00:00
rkuo-danswer
b92c183022
re-prep user group deletion on the actual deletion (#3588)
* re-prep user group deletion on the actual deletion

* user group needs to be synced to be prepped

---------

Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2025-01-03 22:35:56 +00:00
skylares
c191e23256
Pagination Hook (#3494)
* Backend changes for pagination hook + Paginated users table

* Frontend changes for pagination hook

* Fix invited users endpoint

* Fix layout shift & add enter to submit user invites

* mypy

* Cleanup

* Resolve PR concerns & remove UserStatus

* Fix errors

---------

Co-authored-by: hagen-danswer <hagen@danswer.ai>
2025-01-03 14:32:55 -08:00
rkuo-danswer
a83c9b40d5
Bugfix/oauth fix (#3507)
* old oauth file left behind

* fix function change that was lost in merge

* fix some testing vars

---------

Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2024-12-30 01:49:12 +00:00
pablonyx
27acd3387a
Auth specific rate limiting (#3463)
* k

* v1

* fully functional

* finalize

* nit

* nit

* nit

* clean up with wrapper + comments

* k

* update

* minor clean
2024-12-29 23:34:23 +00:00
hagen-danswer
8837b8ea79
Curators can now update the curator relationship (#3536)
* Curators can now update the curator relationship

* mypy

* mypy

* whoops haha
2024-12-24 18:49:58 +00:00
pablonyx
4e2c90f4af
Proper user deletion / organization leaving (#3460)
* Proper user deletion / organization leaving

* minor nit

* update

* udpate provisioning

* minor cleanup

* typing

* post rebase
2024-12-20 21:01:03 +00:00
pablonyx
4d2e26ce4b
MT Cloud Tracking Fix (#3514) 2024-12-19 08:47:02 -08:00
pablonyx
b3ecf0d59f
Migrate user milestone logic (#3493) 2024-12-17 15:59:56 -08:00
pablonyx
48be6338ec
Update Hubpost tracking form submission (#3261)
* Update Hubpost tracking form submission

* minor cleanup

* validated

* validate

* nit

* k
2024-12-17 02:31:09 +00:00
Yuhong Sun
814f97c2c7
MT Cloud Monitoring (#3465) 2024-12-15 16:05:03 -08:00
pablodanswer
7064c3d06f update legal references 2024-12-13 10:39:01 -08:00
pablodanswer
21ec5ed795 welcome to onyx 2024-12-13 09:56:10 -08:00
rkuo-danswer
dee1a0ecd7
Feature/google drive oauth (#3365)
* first cut at slack oauth flow

* fix usage of hooks

* fix button spacing

* add additional error logging

* no dev redirect

* early cut at google drive oauth

* second pass

* switch to production uri's

* try handling oauth_interactive differently

* pass through client id and secret if uploaded

* fix call

* fix test

* temporarily disable check for testing

* Revert "temporarily disable check for testing"

This reverts commit 4b5a022a5fe38b05355a561616068af8e969def2.

* support visibility in test

* missed file

---------

Co-authored-by: Richard Kuo <rkuo@rkuo.com>
Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
2024-12-12 18:01:59 +00:00
hagen-danswer
6d6e12119b
made external group emails lowercase (#3410) 2024-12-10 22:08:00 +00:00
Chris Weaver
970320bd49
Persona / prompt hardening (#3375)
* Persona / prompt hardening

* fix it
2024-12-09 03:39:59 +00:00
hagen-danswer
53428f6e9c
More logging/fixes (#3364)
* More logging for external group syncing

* Fixed edge case where some spaces were not being fetched

* made refresh frequency for confluence syncs configurable

* clarity
2024-12-06 21:56:29 +00:00
rkuo-danswer
7a3c06c2d2
first cut at slack oauth flow (#3323)
* first cut at slack oauth flow

* fix usage of hooks

* fix button spacing

* add additional error logging

* no dev redirect

* cleanup

* comment work in progress

* move some stuff to ee, add some playwright tests for the oauth callback edge cases

* fix ee, fix test name

* fix tests

* code review fixes
2024-12-06 19:55:21 +00:00
hagen-danswer
32ec4efc7a tygod for tests 2024-12-06 08:03:34 -08:00