c5adbe4180
* db setup * transfer 1 - incomplete * more adjustments * relationship table + query update * temp view creation * restructuring * nits * updates * separate read_only engine * extraction revamp * focus on metadata relatonships 1 * dev * migration downgrade fix * rebase migration change * a3+ * progress * base * new extraction * progress * fixed KG extraction * nits * updates * simplifications & cleanup * fixes * updates * more feature flag checks * fixes * extraction process fix * read-only user creation as part of setup * fix for missing entity attributes * kg read-only user creation as part of migration * typo * EL initial comments * initial Account/SF Connector chnges * SF Connector update - include account information * base w/ salesforce * evan updates + quite a bit more * kg-filtered search * EL changes pt 2 * migrations and env vars * quick migration fix * migration update * post_rebase fixes * mypy fixes * test fixes * test fix * test fix * read_only pool + misc * nf * env vars * test improvements * salesforce fix * test update * small changes * small adjustments * SF Connector fix & kg_stage removal for one table * mypy fix * small fixes * EL + RK (pt 1) comments * nit * setting updated * Salesforce test update * EL comments * read-only user replacement & cleanup * SQL View fix * converting entity type-name separators * sql view group ownership * view fix * SQL tweak * dealing with docs that were skipped by indexing * increased error handling * more error handling * Output formatting fix * kg-incremental-reindexing * 0-doc found improvement * celery * migration correction * timeout adjustments * nit * Updated migration * Entity Normalization for KG Dev 1 (#4746) * feat: trigrams column * fix: reranking and db * feat: v1 * fix: convert to orm * feat: parallel * fix: default to id_name * fix: renamed semantic_id and semantic_id_trigrams * fix: scalar subquery * fix: tuning + redundancy * fix: threshold * fix: typo * fix: shorten names * wip * fix: reverted * feat: config * feat: works but it was dumb * feat: clustering works * fix: mypy * normalization <-> language awareness for SQL generation * small type fixes --------- Co-authored-by: joachim-danswer <joachim@danswer.ai> * mypy * typo and dead code * kg_time_fencing * feat: remove temp views on migration downgrade * remove functions and triggers for now * rebase adjustments * EL code review results * quick fix + trigger/funcs for single tenant * fix: typo, mypy, dead code * fix: autoflake * small updatesd * nit * fix: typo * early + faster view creation * Extension creation in MT migration * nit changes to default ETs * Incremental Clustering and KG Refactor V1 (#4784) Optimized/restructured incremental clustering. New pipeline actually that moves vespa updates to clustering. Also, celery configuration has been updated. --------- Co-authored-by: joachim-danswer <joachim@danswer.ai> * prompt tweak & ET extraction reset * more general hierarchical structure * feat: better vespa reset logic * prompt optimization and entity replacemants * small prompt changes * KG Refactor V2 (#4814) Clustering & Extraction improvements & various nits Co-authored-by: joachim-danswer <joachim@danswer.ai> * add connector-level coverage days * fix: nit * initial EL responses * refactor: helper functions for formatting * fix: more helper fns & comments * fix: comment code that's been implemented elsewhere * fix: tenant_id missing arg * fix: removed debugging stuff * fix: moved kg_interactions db query to helper fn * fix: tenant_id * fix: tenant_id & removed outdated helper fn * fix always set entity class * fix: typo * fix alembic heads * fix: celery logging * fix: migrations fix * fix: multi tenant permissions * fix: temp connector fix * fix: downgrade * Fix upgrade migration * fix: tenant for normalization * added additional acl * stray EL comments * fix: connector test * fix mypy * fix: temporary connector test fix * fix: jira connector test * nit * small nits * fix: black * fix: mypy * fix: mypy --------- Co-authored-by: Rei Meguro <36625832+Orbital-Web@users.noreply.github.com>
Deploying Onyx using Docker Compose
For general information, please read the instructions in this README.
Deploy in a system without GPU support
This part is elaborated precisely in this README in section Docker Compose. If you have any questions, please feel free to open an issue or get in touch in slack for support.
Deploy in a system with GPU support
Running Model servers with GPU support while indexing and querying can result in significant improvements in performance. This is highly recommended if you have access to resources. Currently, Onyx offloads embedding model and tokenizers to the GPU VRAM and the size needed depends on chosen embedding model. For example, the embedding model nomic-ai/nomic-embed-text-v1 takes up about 1GB of VRAM. That means running this model for inference and embedding pipeline would require roughly 2GB of VRAM.
Setup
To be able to use NVIDIA runtime, following is mandatory:
- proper setup of NVIDIA driver in host system.
- installation of
nvidia-container-toolkitfor passing GPU runtime to containers
You will find elaborate steps here:
Installation of NVIDIA Drivers
Visit the official NVIDIA drivers page to download and install the proper drivers. Reboot your system once you have done so.
Alternatively, you can choose to install the driver versions via package managers of your choice in UNIX based systems.
Installation of nvidia-container-toolkit
For GPUs to be accessible to containers, you will need the container toolkit. Please follow these instructions to install the necessary runtime based on your requirement.
Launching with GPU
-
To run Onyx with GPU, navigate to
docker_composedirectory and run the following:docker compose -f docker-compose.gpu-dev.yml -p onyx-stack up -d --pull always --force-recreate- or run:docker compose -f docker-compose.gpu-dev.yml -p onyx-stack up -d --build --force-recreateto build from source- Downloading images or packages/requirements may take 15+ minutes depending on your internet connection.
-
To shut down the deployment, run:
- To stop the containers:
docker compose -f docker-compose.gpu-dev.yml -p onyx-stack stop - To delete the containers:
docker compose -f docker-compose.gpu-dev.yml -p onyx-stack down
- To stop the containers: