Can Two L40S Replace One A100? A Four-Category Decision by Task Bottleneck

Conclusion First

Two L40S cards together provide 96GB of VRAM (48GB each), with FP8 and BF16 compute power exceeding that of a single A100 80GB; however, there is no NVLink between the two cards—they can only communicate over PCIe 4.0. Each card has 864 GB/s memory bandwidth, roughly half that of the A100's HBM2e. Therefore, whether they can replace an A100 depends on where your task is bottlenecked: if bottlenecked by compute power and total VRAM, two L40S are more cost-effective; if bottlenecked by inter-card communication and memory bandwidth, they cannot replace it.

Ask Yourself Four Questions First

1. Are the two cards working together on the same job (one model split in half), or are they running separate tasks?
2. If they need to communicate across cards, how frequent is the communication—does every batch require synchronization, or is it just a one-time weight split at the beginning?
3. Is it a single-request, long-context scenario sensitive to first-token and output-token latency?
4. Do you need FP64, or do you need to partition one card into several isolated instances?

Question 1 determines whether you need to care about interconnect; question 2 determines whether PCIe is sufficient; question 3 determines whether memory bandwidth matters; question 4 is essentially a veto.

These Scenarios Can Be Replaced, and Are Usually More Cost-Effective

High-concurrency, batched inference. When deploying 70B-level models with vLLM or TGI and using FP8/INT8 quantization, the weights plus KV cache fit comfortably in 96GB—more spacious than a single 80GB card. Once batching increases (roughly Batch Size 8 or above), the L40S's FP8 compute and Transformer Engine can boost throughput, making the cost per million tokens more favorable. This is an architectural difference: L40S uses Ada Lovelace with 4th-gen Tensor Cores, natively supporting FP8; A100 uses Ampere with 3rd-gen Tensor Cores, which does not support FP8. For inference trade-offs between the two, see L40S vs A100 for inference.

Multimodal and generative media. The L40S retains RT Cores and includes modern NVENC/NVDEC encoding/decoding engines (including AV1, with three encoder/decoder sets per card); the A100 lacks both, so video work mostly falls back to CPU software encoding. When running ComfyUI, Stable Diffusion, video generation rendering, or Whisper transcription, two L40S can serve independently, and a single card itself is no slouch.

Lightweight fine-tuning and "one split into two". Parameter-efficient fine-tuning like LoRA and QLoRA does not require dense inter-card communication; splitting two L40S into two independent tasks, each with 48GB, yields a larger total VRAM pool than a single A100. For how large a model can fit in VRAM, see What size model can 48GB VRAM run?.

These Scenarios Cannot Be Replaced

Distributed training that relies on high-frequency inter-card communication. The L40S lacks NVLink interfaces, so dual cards can only use PCIe 4.0 x16, with a theoretical bidirectional bandwidth of about 64 GB/s; A100 SXM's NVLink 3.0 offers 600 GB/s, and A100 PCIe can also achieve 600 GB/s between two cards via a bridge. For tensor parallelism (splitting a layer across two cards) or gradient synchronization in full-parameter training, PCIe becomes the bottleneck. Note the baseline version: A100 PCIe and SXM4 versions have different interconnect capabilities; comparing with the PCIe version narrows the gap, but it is still a PCIe vs NVLink difference in magnitude. In contrast, data parallelism is fine as long as the model fits on a single card; what truly depends on interconnect is tensor parallelism and pipeline parallelism.

Low-concurrency, long-context single-stream decoding. Autoregressive decoding is typically memory-bandwidth-bound: each generated token requires reading all weights. The A100's HBM2/HBM2e bandwidth is 1,555–2,039 GB/s, while the L40S's GDDR6 is 864 GB/s, so a single A100 is still faster for single-request response. If your service serves few users and demands low latency, the dual-card compute advantage does not translate into a latency advantage.

FP64 scientific computing and multi-tenant hard partitioning. The A100 has full double-precision units (FP64 compute 9.7–19.5 TFLOPS) and supports MIG, allowing a single card to be physically partitioned into up to 7 isolated instances; the L40S lacks dedicated FP64 acceleration and does not support MIG. Simulations like fluid dynamics, molecular modeling, and quantum chemistry, as well as multi-tenant hosting requiring hard isolation, are not suitable for two L40S.

On Renting: Several Points That Actually Affect Operations

• Two cards mean two units of compute time. Billing includes only compute, storage, and traffic; the unit price is locked at order time and lasts until destruction. When running dual-card tasks, do not estimate budget based on a single card's duration.
• Stopping does not mean no cost. After stopping, compute fees stop, but storage is still billed by capacity times duration; only destruction stops everything. If you pause mid-task to modify code or switch images, remember storage fees are still accruing.
• Split into two independent tasks if possible. This avoids any cross-card configuration and is not affected by PCIe bandwidth; image templates allow one-click deployment (vLLM, TGI, Ollama, PyTorch, ComfyUI, Whisper, etc.).
• If you must cross cards, choose the right parallelism. Cross-card is suitable for data parallelism, or simply using the two cards' VRAM as two separate pools; expecting to run tensor parallelism over PCIe is usually worse than choosing a card with stronger interconnect.
• If unsure which card to pick, check by model and scenario first. Choose GPU by model lists suitable card types by VRAM threshold; real-time unit price lets you calculate the cost of the entire task by the hour.

To summarize briefly: for throughput, multimodality, and two independent environments, two L40S are the more flexible and economical choice; for tensor parallelism, single-stream low latency, FP64, or hard isolation, you still need cards like the A100 with NVLink, HBM, and MIG.