1.8 KiB
Docker Compose is a way of installing and launching the web UI in an isolated Ubuntu image using only a few commands.
Prerequisites
You need Docker Compose v2.17 or higher:
~$ docker compose version
Docker Compose version v2.21.0
Installation instructions: https://docs.docker.com/engine/install/
For NVIDIA GPUs, you also need the NVIDIA Container Toolkit.
Quick start
There are four Docker variants available under docker/:
| Directory | GPU | Notes |
|---|---|---|
docker/nvidia |
NVIDIA | Requires NVIDIA Container Toolkit |
docker/amd |
AMD | Requires ROCm-compatible GPU |
docker/intel |
Intel Arc | Beta support |
docker/cpu |
None | CPU-only inference |
To launch (using NVIDIA as an example):
cd text-generation-webui/docker/nvidia
cp ../.env.example .env
# Optionally edit .env to customize ports, TORCH_CUDA_ARCH_LIST, etc.
docker compose up --build
The web UI will be available at http://localhost:7860.
User data
Create a user_data/ directory next to the docker-compose.yml to persist your models, characters, presets, and settings between container rebuilds:
mkdir -p user_data
This directory is mounted into the container at runtime. You can place a CMD_FLAGS.txt inside it to pass persistent flags to the web UI (e.g., --api).
Models can be downloaded through the web UI's “Model” tab once it's running, and they will be saved to user_data/models/.
Dedicated docker repository
An external repository maintains a docker wrapper for this project as well as several pre-configured 'one-click' docker compose variants. It can be found at: Atinoda/text-generation-webui-docker.