highperfocused/danswer
1
0
Fork 0
mirror of https://github.com/danswer-ai/danswer.git synced 2025-03-18 13:51:46 +01:00
Code Issues Packages Projects Releases Wiki Activity
danswer/deployment/docker_compose
History
hagen-danswer 47a550221f
slackbot doesnt respond without citations/quotes (#1798) 
* slackbot doesnt respond without citations/quotes

fixed logical issues

fixed dict logic

* added slackbot shim for the llm source/time feature

* mypy fixes

* slackbot doesnt respond without citations/quotes

fixed logical issues

fixed dict logic

* Update handle_regular_answer.py

* added bypass_filter check

* final fixes
2024-07-11 00:18:26 +00:00
..
docker-compose.dev.yml
slackbot doesnt respond without citations/quotes (#1798)
2024-07-11 00:18:26 +00:00
docker-compose.gpu-dev.yml
slackbot doesnt respond without citations/quotes (#1798)
2024-07-11 00:18:26 +00:00
docker-compose.prod-no-letsencrypt.yml
Docker Compose Eval Pipeline Cleanup (#1780)
2024-07-06 12:04:57 -07:00
docker-compose.prod.yml
Docker Compose Eval Pipeline Cleanup (#1780)
2024-07-06 12:04:57 -07:00
docker-compose.search-testing.yml
Eval Script Incremental Write (#1784)
2024-07-06 15:43:40 -07:00
env.multilingual.template
Touchup for Multilingual Users (#1725)
2024-06-26 22:44:06 -07:00
env.nginx.template
Add support for non-letsencrypt-based https in docker compose setup (#628)
2023-10-25 20:35:47 -07:00
env.prod.template
Tiny Mail From Fix (#953)
2024-01-15 22:03:38 -08:00
init-letsencrypt.sh
Update init-letsencrypt.sh (#1380)
2024-06-04 10:38:01 -07:00
README.md
Feature/support all gpus (#1515)
2024-06-08 17:45:26 -07:00

README.md

Deploying Danswer using Docker Compose

For general information, please read the instructions in this README.

Deploy in a system without GPU support

This part is elaborated precisely in in this README in section Docker Compose. If you have any questions, please feel free to open an issue or get in touch in slack for support.

Deploy in a system with GPU support

Running Model servers with GPU support while indexing and querying can result in significant improvements in performance. This is highly recommended if you have access to resources. Currently, Danswer offloads embedding model and tokenizers to the GPU VRAM and the size needed depends on chosen embedding model. Default embedding models intfloat/e5-base-v2 takes up about 1GB of VRAM and since we need this for inference and embedding pipeline, you would need roughly 2GB of VRAM.

Setup

To be able to use NVIDIA runtime, following is mandatory:

  • proper setup of NVIDIA driver in host system.
  • installation of nvidia-container-toolkit for passing GPU runtime to containers

You will find elaborate steps here:

Installation of NVIDIA Drivers

Visit the official NVIDIA drivers page to download and install the proper drivers. Reboot your system once you have done so.

Alternatively, you can choose to install the driver versions via package managers of your choice in UNIX based systems.

Installation of nvidia-container-toolkit

For GPUs to be accessible to containers, you will need the container toolkit. Please follow these instructions to install the necessary runtime based on your requirement.

Launching with GPU

  1. To run Danswer with GPU, navigate to docker_compose directory and run the following:

    • docker compose -f docker-compose.gpu-dev.yml -p danswer-stack up -d --pull always --force-recreate
      • or run: docker compose -f docker-compose.gpu-dev.yml -p danswer-stack up -d --build --force-recreate to build from source
    • Downloading images or packages/requirements may take 15+ minutes depending on your internet connection.

  2. To shut down the deployment, run:

    • To stop the containers: docker compose -f docker-compose.gpu-dev.yml -p danswer-stack stop
    • To delete the containers: docker compose -f docker-compose.gpu-dev.yml -p danswer-stack down