Text-to-image model

Self-hosting Kolors: the VRAM sits in the text encoder, not the UNet

Kuaishou's Kolors team swapped T5/CLIP for ChatGLM3-6B, which is why its Chinese semantics and in-image Chinese text rendering lead the open-source field — at the cost of a 12.5GB text encoder. Understand that one fact and a single 24GB card is enough.

Kolors · self-hosted

Kolors (Chinese name 可图) is the latent-diffusion text-to-image model released by Kuaishou's Kolors team on 6 July 2024. Code lives at github.com/Kwai-Kolors/Kolors, weights under the Kwai-Kolors org on Hugging Face. The architectural bet is explicit in the technical report: where Imagen and Stable Diffusion 3 use T5, Kolors builds on the General Language Model family and uses ChatGLM3-6B as its text encoder, with a multimodal LLM recaptioning the training corpus. The payoff is genuinely bilingual prompt understanding, a prompt budget of 256 tokens instead of SDXL's 77, and Chinese text rendering that has stayed ahead of the open-source pack. The diffusion side is conventional: an SDXL-isomorphic U-Net backbone of roughly 2.6B parameters, native 1024×1024, and the plain SDXL VAE.

That structure inverts every deployment instinct you carry over from SDXL. In fp16 the UNet is only 5.16GB while the text encoder is 12.5GB — shipped as seven pytorch_model-0000x-of-00007.bin shards in pickle format, not safetensors — plus a VAE of a couple hundred megabytes, for about 17.9GB of weights; the full repo including the fp32 UNet is 29GB. So the lever for cutting VRAM is not the UNet, it is quantising ChatGLM3. The community standard, Kijai/ChatGLM3-safetensors, offers 8-bit at 6.78GB and 4-bit at 3.92GB; drop one in and the whole pipeline lands just above 10GB, at which point 1024 sampling costs about what SDXL costs. The other rough edges are equally specific: the official conda env still pins Python 3.8; the SDXL VAE brings its familiar fp16 overflow-to-black-image bug, so run the VAE in fp32 or swap in the fp16-fix build; prompts over 256 tokens are truncated silently; and in diffusers the IP-Adapter image encoder must be loaded with revision="refs/pr/4" or you are stuck reading pickle checkpoints from the original repo.

One thing to be straight about: the open-weight line stops at Kolors 1.0. The repo's News section ends on 13 November 2024 with the Kolors-Portrait-with-Flux and Kolors-Character-With-Flux demos; no newer base weights have been published, and later Kolors generations are served through Kuaishou's Kling AI platform as an API rather than as downloadable weights. What the Kwai-Kolors org has open-sourced since is different work: CoTyle (20B, Apache-2.0, released 18 November 2025 — it conditions style on a numeric code instead of a long prompt, with Qwen-Image as the open-release base and Kolors as the commercial one) and MetaView (single-image novel view synthesis, an MM-DiT backbone with Depth Anything 3 geometry priors, accepted to ECCV 2026). Which means: if you want to self-host 可图 today, Kolors 1.0 is what you actually get — and it holds up. It is cheap, it fits on one 24GB card, and the ecosystem is complete: Canny/Depth/Pose ControlNets, an inpainting checkpoint, IP-Adapter-Plus and FaceID-Plus, and Dreambooth-LoRA training code. On NexGPU the entry ticket is a single RTX 3090 at $0.193/GPU-hour.

01 —

Every Kolors weight you can actually download

Base model, quantised text encoders, three ControlNets, inpainting and image-prompt adapters — and what each costs you in VRAM

Version	Parameters	VRAM	Context	Notes
Kolors (Kwai-Kolors/Kolors)	~2.6B UNet + ChatGLM3-6B text encoder	~17.9GB of fp16 weights (UNet 5.16GB + text encoder 12.5GB + SDXL VAE); 29GB for the full repo with fp32	256-token prompt / native 1024×1024	The 2024.07.06 base checkpoint. Bilingual Chinese/English, best known for legible in-image Chinese text. Repo default inference is EulerDiscreteScheduler, CFG 5.0, 50 steps.
Kolors-diffusers	Same weights, diffusers layout	~16.5GB if you pull only the fp16 variant	KolorsPipeline / KolorsImg2ImgPipeline	Natively supported from diffusers 0.30; from_pretrained(variant="fp16") just works. The official example uses DPMSolverMultistepScheduler with Karras sigmas, CFG 6.5, 25 steps — half the wall-clock of the repo's 50-step default.
Quantised ChatGLM3 text encoder (Kijai/ChatGLM3-safetensors)	The 6B text encoder, quantised standalone	4-bit 3.92GB / 8-bit 6.78GB / fp16 12.5GB	256-token limit unchanged	The only lever that meaningfully moves Kolors' VRAM. At 4-bit, single-image 1024 sampling costs about what SDXL costs and 16GB cards become viable.
Kolors-ControlNet-Canny / Depth / Pose	~1B each	About +2GB in fp16 per adapter loaded	1024×1024	Released 2024.07.26 and 2024.08.06. Depth additionally needs the dpt_hybrid-midas weights and Pose needs the DWPose ONNX models — neither ships in the main repo, and that is where most first runs stall.
Kolors-Inpainting	~3B	Replaces the base UNet, about +6GB in fp16	1024×1024	A dedicated inpainting checkpoint. Use it for background replacement, product-shot text edits and watermark removal rather than masking with the base model.
Kolors-IP-Adapter-Plus / IP-Adapter-FaceID-Plus	Adapter + OpenAI CLIP-ViT-L-336 image encoder	The diffusers docs state this needs more than 24GB, otherwise enable_model_cpu_offload() is required	Image prompting / face-identity preservation	Not interchangeable with SDXL IP-Adapters — you must use the Kolors ones. Load image_encoder with revision="refs/pr/4" or the safetensors files will not resolve.

02 —

Pick the card by what you are doing

Real NexGPU per-second rates, matched honestly against Kolors' actual footprint

First run, checking Chinese text rendering, full fp16 generation
RTX 3090 24GB$0.193/GPU-hour
17.9GB of fp16 weights plus 1024 sampling activations sits comfortably inside 24GB, and this is the cheapest card on our platform that holds the whole of Kolors resident.
Batch generation, online serving, lowest cost per image
RTX 4090 24GB$0.540/GPU-hour
Same 24GB, but Ada roughly doubles sampling throughput — you wait half as long for a batch and the absolute per-image cost is still fractions of a cent.
Stacking IP-Adapter-Plus, FaceID or several ControlNets at once
RTX 5090 32GB$0.723/GPU-hour
The diffusers docs put IP-Adapter above 24GB; 32GB means you never turn on cpu offload, and offload is what turns a 3-second image into a 15-second one.
Dreambooth-LoRA fine-tuning, training and evaluating adapters side by side
RTX A6000 48GB$0.817/GPU-hour
Training-state UNet, optimiser state, 1024-resolution activations and a ChatGLM3 encoder you have not finished precomputing embeddings for — 48GB means you never trade batch size against gradient checkpointing.

03 —

Kolors running on NexGPU in four steps

From a prebuilt PyTorch image to your first image with legible Chinese characters in it

01
Spin up a 24GB instance and install the deps
Pick the PyTorch image from the 2,000+ prebuilt library and connect over SSH or Jupyter. The repo's conda env pins Python 3.8, but the diffusers route does not need that — use the image's Python. What matters is diffusers 0.30 or newer and transformers 4.38 or newer.
```
pip install "diffusers>=0.30.0" "transformers>=4.38.0" accelerate sentencepiece safetensors
```
02
Pull only the fp16 weights
Half of that 29GB repo is an fp32 UNet you will never load. Grab the fp16 variant of Kolors-diffusers instead, roughly 16.5GB. Note the text encoder arrives as seven .bin pickle shards — if your loader hardcodes use_safetensors=True this step fails, so either drop that flag or substitute the community ChatGLM3 safetensors build.
```
huggingface-cli download Kwai-Kolors/Kolors-diffusers --local-dir weights/Kolors
```

Start the pipeline and generate

Use DPMSolverMultistepScheduler with Karras sigmas, CFG 6.5, 25 steps — twice as fast as the repo's 50-step Euler default at essentially the same quality. If your first image comes out pure black, that is the SDXL VAE overflowing in fp16: run the VAE in fp32 or swap in the fp16-fix build.

python -c "import torch; from diffusers import KolorsPipeline, DPMSolverMultistepScheduler; p=KolorsPipeline.from_pretrained('weights/Kolors', torch_dtype=torch.float16, variant='fp16').to('cuda'); p.scheduler=DPMSolverMultistepScheduler.from_config(p.scheduler.config, use_karras_sigmas=True); p(prompt='a photo of a ladybug, macro, zoom, high quality, cinematic, holding a sign that says Kolors', guidance_scale=6.5, num_inference_steps=25).images[0].save('kolors.png')"

04
Add IP-Adapter and ControlNet, or move to ComfyUI
Image prompting goes through Kolors-IP-Adapter-Plus — mind the revision argument on the image encoder, it is the single most common failure. Turn on enable_model_cpu_offload() only if you are short on VRAM; on a 32GB card, don't. For a visual workflow, use the ComfyUI image plus kijai's ComfyUI-KwaiKolorsWrapper, which exposes fp16 / 8-bit / 4-bit text encoders as a node option.
```
pipe.load_ip_adapter("Kwai-Kolors/Kolors-IP-Adapter-Plus", subfolder="", weight_name="ip_adapter_plus_general.safetensors", revision="refs/pr/4", image_encoder_folder=None)
```

What an image actually costs

Assume 1024×1024, DPM++ Karras, 25 steps, roughly 6 seconds an image. An RTX 3090 24GB at $0.193/GPU-hour gets you about 600 images an hour: $0.193 ÷ 600 ≈ $0.00032 per image. Move to an RTX 4090 24GB at $0.540/GPU-hour and Ada roughly doubles throughput — call it 2.7 seconds an image, about 1,330 an hour, so $0.540 ÷ 1,330 ≈ $0.00041 per image. A third more per image, and half the waiting; when a deadline is the constraint that arithmetic is easy. Keeping the weights around is a separate line: the fp16 set is 16.5GB, and at the $0.414/GB-month median that is 16.5 × 0.414 ≈ $6.83 a month. Compute billing stops the moment the instance stops; storage keeps accruing until you destroy the volume, so a short project that tears down afterwards pays nothing for it. Pulling 1,000 finished 1024 PNGs home (about 1.5GB) costs roughly 1.5 × $0.0081 ≈ $0.012 in egress at the median rate. Everything is metered per second and priced per hour — no minimum, no setup fee, no quota request.

04 —

FAQ

What is the minimum VRAM to run Kolors locally? Will 16GB do?

Full fp16 is about 17.9GB of weights, and once you add 1024 sampling activations you really want a 24GB card. To get under 16GB, swap ChatGLM3-6B for the 4-bit build (3.92GB) — that alone frees 8.5GB and lands the pipeline just above 10GB. On NexGPU you rarely need to bother: an RTX 3090 24GB is $0.193/GPU-hour, which costs less than the time you would spend fighting quantisation, and it holds the full fp16 model.

How does Kolors relate to SDXL? Can I reuse SDXL LoRAs and ControlNets?

The UNet backbone is SDXL-isomorphic at roughly 2.6B and the VAE is literally the SDXL VAE, so your speed and memory intuitions transfer. The text encoder does not: ChatGLM3-6B changes the conditioning dimensions and everything feeding cross-attention, so SDXL LoRAs, textual inversions and ControlNets are all incompatible. Use Kwai-Kolors' own ~1B Canny/Depth/Pose ControlNets and the Kolors IP-Adapter. If you want both ecosystems side by side, run two per-second-billed NexGPU instances in parallel rather than rebuilding environments on one box.

Why does adding IP-Adapter immediately OOM?

You did not misconfigure anything — the diffusers documentation states plainly that Kolors with an IP-Adapter needs more than 24GB of VRAM and recommends enable_model_cpu_offload() on consumer cards. The other frequent error is the image encoder failing to load, which needs revision="refs/pr/4" passed to from_pretrained to resolve the safetensors files. Either eat the offload penalty, or take an RTX 5090 32GB on NexGPU at $0.723/GPU-hour, which holds the adapter and image encoder resident with no offloading at all.

Can I use Kolors commercially? What does the licence actually say?

The code is Apache-2.0; the weights carry a separate Kolors Model License. The core terms: after registering with the licensor, if the combined monthly active users of all your products and services did not exceed 300 million in the preceding calendar month, that registration is deemed to grant the corresponding commercial licence — above that threshold you need explicit approval. When you redistribute, downstream recipients must be told the model's source and given a copy of the agreement, and modified files must carry prominent change notices. The licence also forbids using Kolors outputs to improve any large model other than Kolors and its derivatives. Handle the paperwork your side; on the compute side NexGPU spans 51 countries and regions with 1,175 verified rentable nodes, so you can pick a region that matches your data-residency requirements.

Is Kolors still being updated? Which version do the open weights stop at?

The open base line stops at Kolors 1.0 from July 2024; the repo's News ends on 13 November 2024 with two Flux collaboration demos, and no newer base weights have shipped — later Kolors generations run as an API on Kuaishou's Kling AI platform rather than as downloadable weights. What the Kwai-Kolors org has released since is CoTyle (20B, Apache-2.0, 18 November 2025, style controlled by a numeric code, Qwen-Image as the open base) and MetaView (single-image novel view synthesis, MM-DiT with Depth Anything 3 geometry priors, accepted to ECCV 2026). None of that diminishes Kolors 1.0, still one of the cheapest open models for legible Chinese text — and on NexGPU you can evaluate all three in an afternoon for less than the price of lunch.

How long can a Kolors prompt be? Why does the back half seem to do nothing?

KolorsPipeline's encode_prompt defaults to max_sequence_length=256 — far roomier than SDXL's 77 tokens, but anything past it is truncated silently with no warning. You wrote it; the model never saw it. That is the usual explanation for a prompt tail that appears inert. To map where your own prompts hit the ceiling, spin up an RTX 3090 24GB at $0.193/GPU-hour and batch a controlled comparison — a few cents buys the whole curve.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.