set field size limit (#4683)

* set field size limit

* don't use sys.maxsize

---------

Co-authored-by: Richard Kuo <rkuo@rkuo.com>
Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
This commit is contained in:
rkuo-danswer
2025-05-09 15:46:13 -07:00
committed by GitHub
parent 1a8b7abd00
commit 84566debab

View File

@@ -351,6 +351,9 @@ class OnyxSalesforceSQLite:
if self._conn is None:
raise RuntimeError("Database connection is closed")
# some customers need this to be larger than the default 128KB, go with 16MB
csv.field_size_limit(16 * 1024 * 1024)
updated_ids = []
with self._conn: