* feat: add option to treat all non-attachment fields as metadata in Airtable connector
- Added new UI option 'treat_all_non_attachment_fields_as_metadata'
- Updated backend logic to support treating all fields except attachments as metadata
- Added tests for both default and all-metadata behaviors
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: handle missing environment variables gracefully in airtable tests
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: clean up test file and handle environment variables properly
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: add missing test fixture and fix formatting
Co-Authored-By: Chris Weaver <chris@onyx.app>
* chore: fix black formatting
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: add type annotation for metadata dict in airtable tests
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: add type annotation for mock_get_api_key fixture
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: update Generator import to use collections.abc
Co-Authored-By: Chris Weaver <chris@onyx.app>
* refactor: make treat_all_non_attachment_fields_as_metadata a direct required parameter
- Move parameter from connector_config to direct class parameter
- Place parameter right under table_name_or_id argument
- Make parameter required in UI with no default value
- Update tests to use new parameter structure
Co-Authored-By: Chris Weaver <chris@onyx.app>
* chore: fix black formatting
Co-Authored-By: Chris Weaver <chris@onyx.app>
* chore: rename _METADATA_FIELD_TYPES to DEFAULT_METADATA_FIELD_TYPES and clarify usage
Co-Authored-By: Chris Weaver <chris@onyx.app>
* chore: fix black formatting in docstring
Co-Authored-By: Chris Weaver <chris@onyx.app>
* test: make airtable tests fail loudly on missing env vars
Co-Authored-By: Chris Weaver <chris@onyx.app>
* style: fix black formatting in test file
Co-Authored-By: Chris Weaver <chris@onyx.app>
* style: add required newline between test functions
Co-Authored-By: Chris Weaver <chris@onyx.app>
* test: update error message pattern in parameter validation test
Co-Authored-By: Chris Weaver <chris@onyx.app>
* style: fix black formatting in test file
Co-Authored-By: Chris Weaver <chris@onyx.app>
* test: fix error message pattern in parameter validation test
Co-Authored-By: Chris Weaver <chris@onyx.app>
* style: fix line length in test file
Co-Authored-By: Chris Weaver <chris@onyx.app>
* test: simplify error message pattern in parameter validation test
Co-Authored-By: Chris Weaver <chris@onyx.app>
* test: add type validation test for treat_all_non_attachment_fields_as_metadata
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: add missing required parameter in test
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: remove parameter from test to properly validate it is required
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: add type validation for treat_all_non_attachment_fields_as_metadata parameter
Co-Authored-By: Chris Weaver <chris@onyx.app>
* style: fix black formatting in airtable_connector.py
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: update type validation test to handle mypy errors
Co-Authored-By: Chris Weaver <chris@onyx.app>
* fix: specify mypy ignore type for call-arg
Co-Authored-By: Chris Weaver <chris@onyx.app>
* Also handle rows w/o sections
* style: fix black formatting in test assertion
Co-Authored-By: Chris Weaver <chris@onyx.app>
* add TODO
* Remove unnecessary check
* Fix test
* Do not break existing airtable connectors
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Chris Weaver <chris@onyx.app>
Co-authored-by: Weves <chrisweaver101@gmail.com>
* Added ability to use a tag to insert the current datetime in prompts
* made tagging logic more robust
* rename
* k
---------
Co-authored-by: Yuhong Sun <yuhongsun96@gmail.com>
* Combined Persona and Prompt API
* quality
* added tests
* consolidated models and got rid of redundant fields
* tenant appreciation day
* reverted default
* Added Permission Syncing for Salesforce
* cleanup
* updated connector doc conversion
* finished salesforce permission syncing
* fixed connector to batch Salesforce queries
* tests!
* k
* Added error handling and check for ee and sync type for postprocessing
* comments
* minor touchups
* tested to work!
* done
* my pie
* lil cleanup
* minor comment
* discord: frontend and backend poll connector
* added requirements for discord installation
* fixed the mypy errors
* process messages not part of any thread
* minor change
* updated the connector; this logic works & am able to docs when i print
* minor change
* ability to enter a start date to pull docs from and refactor
* added the load connector and fixed mypy errors
* local commit test
done!
* minor refactor and properly commented everything
* updated the logic to handle permissions and index active/archived threads
* basic discord test template
* cleanup
* going away with the danswer discord client class ; using an async context manager
* moved to proper folder
* minor fixes
* needs improvement
* fixed discord icon
---------
Co-authored-by: hagen-danswer <hagen@danswer.ai>
- renamed post-reranking/validation citation information consistently to final_... (example: doc_id_to_rank_map -> final_doc_id_to_rank_map)
- changed and renamed objects containing initial ranking information (now: display_...) consistent with final rankings (final_...). Specifically, {} to [] for displayed_search_results
- for CitationInfo, changed citation_num from 'x-th citation in response stream' to the initial position of the doc [NOTE: test implications]
- changed tests:
onyx/backend/tests/unit/onyx/chat/stream_processing/test_citation_processing.py
onyx/backend/tests/unit/onyx/chat/stream_processing/test_citation_substitution.py
* first cut at slack oauth flow
* fix usage of hooks
* fix button spacing
* add additional error logging
* no dev redirect
* early cut at google drive oauth
* second pass
* switch to production uri's
* try handling oauth_interactive differently
* pass through client id and secret if uploaded
* fix call
* fix test
* temporarily disable check for testing
* Revert "temporarily disable check for testing"
This reverts commit 4b5a022a5fe38b05355a561616068af8e969def2.
* support visibility in test
* missed file
---------
Co-authored-by: Richard Kuo <rkuo@rkuo.com>
Co-authored-by: Richard Kuo (Danswer) <rkuo@onyx.app>
* Mismatch issue of Documents shown and Citation number in text fix
When document order presented to LLM differs from order shown to user, wrong doc numbers are cited.
Fix:
- SearchTool.get_search_result returns now final and initial ranking
- initial ranking is passed through a few objects and used for replacement in citation processing
Notes:
- the citation_num in the CitationInfo() object has not been changed.
* PR fixes
- linting
- removed erroneous tab
- added a substitution test case
- adjusted original citation extraction use case
* Included a key test and
* Fixed extra spaces
* Updated test documentation
Updated:
- test_citation_substitution (changed description)
- test_citation_processing (removed data only relevant for the substitution)
