Cost, Billing, and OpsOctober 13, 2026Flatkey Team

NVLink vs NVSwitch: GPU Interconnects Explained for AI Clusters

NVLink is the GPU-to-GPU link, NVSwitch is the fabric, and InfiniBand or Ethernet is the scale-out network. Here is how to choose for AI clusters.

NVLink vs NVSwitch: GPU Interconnects Explained for AI Clusters

If you are comparing NVLink vs NVSwitch, the short answer is this:

NVLink is the high-bandwidth GPU-to-GPU link.
NVSwitch is the switching fabric that lets many NVLink-connected GPUs communicate with each other without forcing every GPU pair into a small point-to-point lane.
InfiniBand or Ethernet is still the scale-out network that connects systems, racks, storage, and larger clusters.

That distinction matters when you are renting GPUs, buying an HGX or DGX system, or deciding whether a workload really needs direct cluster access instead of a managed model API. A fast GPU is not enough if the model spends too much time waiting on tensor-parallel synchronizations, all-reduce, all-gather, reduce-scatter, or all-to-all traffic.

This guide explains the nvidia nvlink vs nvswitch differences in practical cluster terms: what each component does, where InfiniBand fits, when the topology changes LLM performance, and what questions to ask before committing to an expensive GPU cluster architecture.

NVLink vs NVSwitch: the quick difference

Layer	What it is	Typical scope	What it improves	What it does not replace
NVLink	Direct high-bandwidth GPU interconnect	GPU-to-GPU links inside supported NVIDIA platforms	Tensor parallelism, peer GPU memory movement, collective communication inside a scale-up domain	It is not a cluster network by itself
NVSwitch	Switch silicon and fabric for NVLink endpoints	Multi-GPU server, baseboard, or rack-scale NVLink domain depending on platform	All-to-all GPU communication, non-blocking or higher-bisection GPU fabric, larger GPU domains	It does not remove the need for InfiniBand/Ethernet for cluster scale-out
InfiniBand / Ethernet	Network fabric between systems and racks	Node-to-node, storage, management, multi-rack clusters	Distributed training across many nodes, storage access, service traffic, scheduling domains	It does not create local NVLink bandwidth between GPUs inside a node

The nvlink vs nvswitch difference is easiest to remember as link vs fabric. NVLink is the road between GPUs. NVSwitch is the interchange that allows many of those roads to meet without turning every trip into a narrow point-to-point path.

What NVLink does

NVLink is NVIDIA's high-speed interconnect for GPU-to-GPU communication. It is designed for the traffic patterns that become expensive over ordinary PCIe paths: model parallel layers, synchronized tensor shards, peer memory access, and high-frequency collectives.

On current public NVIDIA platforms, the generation matters:

Platform generation	Public NVIDIA reference point	NVLink bandwidth per GPU
Hopper	H100/H200 generation	900 GB/s
Blackwell	B200/GB200 generation	1,800 GB/s
Vera Rubin	Announced next-generation platform	3,000 GB/s preliminary

For an infrastructure buyer, those numbers are not a guarantee that every workload gets a matching speedup. They tell you the communication ceiling available to the topology. The actual gain depends on model size, parallelism strategy, batch size, sequence length, framework behavior, placement, and whether the workload is communication-bound.

Use NVLink as a buying signal when your workload uses multiple GPUs for one request, one training step, or one model shard group. It is much less important when each GPU serves independent jobs with little peer communication.

What NVSwitch does

NVSwitch is the switching layer that connects many NVLink endpoints into a higher-bandwidth GPU fabric. Without a switch fabric, a multi-GPU system can end up splitting each GPU's available link budget across many point-to-point connections. With NVSwitch, supported systems can give every GPU a more uniform path to every other GPU in the NVLink domain.

That is why nvswitch vs nvlink is not a choice between two substitutes. NVSwitch uses NVLink connections to build a fabric. The switch is useful because eight, sixteen, or seventy-two GPUs do not behave like two GPUs cabled together.

In public NVIDIA examples:

DGX H100/H200 systems use eight H100 or H200 GPUs and list 900 GB/s GPU-to-GPU bandwidth with cluster networking via ConnectX-7 InfiniBand/Ethernet ports.
DGX B200 systems use eight B200 GPUs and list two fifth-generation NVLink switches with 14.4 TB/s aggregate bandwidth; compare the buying tradeoffs in our B200 vs H200 guide.
GB200 NVL72 connects 36 Grace CPUs and 72 Blackwell GPUs in a rack-scale design; NVIDIA describes it as a 72-GPU NVLink domain with 130 TB/s of low-latency GPU communication. For platform selection, see GB200 vs B200.

The practical takeaway: NVSwitch becomes more important as the number of GPUs in one scale-up domain grows and as more of the workload's critical path becomes GPU-to-GPU communication.

NVLink vs NVSwitch vs InfiniBand

The phrase nvlink vs nvswitch vs infiniband often hides three different design questions:

Inside one GPU system: Do the GPUs have enough peer bandwidth for tensor parallelism, pipeline stages, or multi-GPU inference?
Inside one rack-scale domain: Can many GPUs act as one larger accelerator fabric for large model parallelism?
Across many nodes or racks: Can the cluster network carry distributed training, data loading, storage, checkpointing, and service traffic predictably?

NVLink and NVSwitch answer the first two questions. InfiniBand or Ethernet answers the third.

For example, an eight-GPU HGX or DGX node with NVSwitch can be excellent for a model replica that needs tight tensor parallelism across GPUs in the node. If you need hundreds or thousands of GPUs, you still need a scale-out network. On Blackwell rack systems, NVIDIA combines large NVLink domains with InfiniBand or Spectrum-X Ethernet to scale beyond the rack.

Do not ask whether NVLink replaces InfiniBand. Ask which traffic should stay inside the NVLink domain and which traffic must cross the cluster fabric.

Why AI workloads care about the topology

Modern LLM workloads are not just matrix multiplication. Once a model is split across devices, every layer can introduce communication.

Common communication patterns include:

All-reduce: each rank contributes values and receives the reduced result.
Reduce-scatter and all-gather: common building blocks for tensor and data parallel strategies.
All-to-all: each rank sends different chunks to other ranks, which matters for some expert-parallel and routing-heavy designs.
KV cache and activation movement: implementation-dependent, but often sensitive to locality and placement.

NCCL documents these collectives as rank-based operations across CUDA devices. That is important because the software sees ranks, not marketing terms. A good fabric only helps if your rank placement, framework, container, NCCL settings, and scheduler layout let the job use it.

In NVIDIA's LLM inference discussion, tensor-parallel inference requires GPUs to exchange results after work on a model layer before the next layer can proceed. When that exchange is slow, Tensor Cores can sit idle. That is the core reason NVSwitch can matter for multi-GPU serving.

When NVSwitch is worth paying attention to

Prioritize NVSwitch-backed systems when one of these is true:

You serve models that need tensor parallelism across multiple GPUs to hit latency targets.
Training or fine-tuning jobs show high collective time in profilers.
You plan to run large batch inference where GPU-to-GPU synchronization increases with concurrency.
You need predictable all-to-all communication across every GPU in an eight-GPU node.
You are evaluating rack-scale systems such as GB200 NVL72 where the NVLink domain is a primary product feature.

NVSwitch is less decisive when:

Jobs are embarrassingly parallel and each GPU runs independent work.
The workload is CPU, storage, or data-loader bound.
The model comfortably fits and serves on a single GPU.
The application calls hosted model APIs and does not manage hardware topology directly.
The bottleneck is the inter-node fabric rather than intra-node GPU communication.

The right question is not "Does this GPU have NVLink?" It is "Will my critical workload keep multiple GPUs in the same NVLink/NVSwitch domain busy enough to justify the premium?"

Procurement checklist for NVLink vs NVSwitch systems

Use this checklist before renting or buying a GPU cluster, and pair it with a GPU cluster TCO review before signing a long reservation.

Question	Why it matters
What exact GPU SKU and form factor is offered?	H100 PCIe, H100 SXM, H200, B200, and GB200 systems can expose very different topology and bandwidth.
Is the system NVLink-only point-to-point, NVSwitch-backed, or a rack-scale NVLink domain?	"Has NVLink" is not enough. You need the fabric shape.
How many GPUs are in one NVLink domain?	Domain size affects tensor parallel and model-parallel placement options.
What is the inter-node network: InfiniBand, RoCE, Spectrum-X Ethernet, or ordinary Ethernet?	NVSwitch does not fix a weak scale-out network.
Are NCCL tests available for the exact rented shape?	Synthetic collectives reveal whether topology, drivers, rails, and containers are wired correctly.
Can you pin a full node or domain?	Shared tenancy can break assumptions about locality and repeatability.
Are network rails and NIC placement documented?	NCCL may need correct interface selection and rail-aware placement at scale.
What happens when one GPU, switch tray, or NIC is degraded?	Resiliency and replacement policy affect production uptime.
What benchmark must pass before a long reservation?	Accepted-output cost is more useful than a raw hourly price.

Benchmark plan: prove the fabric before you reserve

For high-cost GPU reservations, run a topology acceptance test before committing to a long term.

Inventory the topology. Capture nvidia-smi topo -m, GPU SKU, driver, CUDA, NCCL version, NICs, and container image.
Run NCCL collectives. Test all-reduce, reduce-scatter, all-gather, and all-to-all at message sizes that resemble your workload.
Measure real workload latency. Use your actual model, context length, batch policy, and parallelism settings.
Compare within-domain and cross-node placement. Check whether the scheduler accidentally moves ranks across slower paths.
Track cost per accepted output. For inference, measure accepted tokens or successful responses per dollar, not just tokens per second. For training, measure loss-progress or completed samples per dollar.
Record failure behavior. Kill a worker, restart a container, or simulate a degraded NIC if the provider allows it.

If a provider cannot tell you the topology or run a basic NCCL benchmark, treat that as a risk signal. Hourly rates also need context from GPU rental prices, capacity commitments, network add-ons, and failed-run costs.

Common mistakes

Mistake 1: Treating NVLink as a synonym for NVSwitch. A platform can expose NVLink links without giving you the same all-to-all behavior as an NVSwitch-backed system.

Mistake 2: Assuming NVSwitch eliminates network design. NVSwitch helps the scale-up GPU domain. Multi-node training still needs a serious InfiniBand or Ethernet design.

Mistake 3: Buying bandwidth for independent jobs. If every GPU runs a separate inference worker, NVSwitch may not improve utilization enough to justify the cost.

Mistake 4: Comparing hourly GPU prices without topology. Two "8x H100" offers can behave differently if one is SXM with NVSwitch and another is PCIe or has weaker networking.

Mistake 5: Ignoring software placement. NCCL rank mapping, container versions, network interface selection, and scheduler policy can erase hardware advantages.

Decision matrix

Workload	NVLink importance	NVSwitch importance	Scale-out network importance	Practical recommendation
Single-GPU inference	Low	Low	Low to medium	Optimize GPU memory, batching, and model choice before paying for NVSwitch.
2-8 GPU tensor-parallel inference	High	High on 8-GPU systems	Medium	Prefer NVSwitch-backed H100/H200/B200 systems and benchmark real latency.
Multi-node distributed training	High inside each node	High inside each node	Very high	Evaluate both node fabric and InfiniBand/Ethernet; do not judge by GPU SKU alone.
Expert-parallel or all-to-all-heavy models	High	High	High when crossing nodes	Test all-to-all and placement directly; topology can dominate.
Batch offline generation	Medium	Medium	Medium	Measure accepted output per dollar; NVSwitch may help at higher batch sizes.
API-first product workload	Usually abstracted away	Usually abstracted away	Provider responsibility	Compare direct GPU control against a model/API baseline before renting hardware.

Where Flatkey fits

Flatkey is a unified LLM API gateway: one key, one bill, and an OpenAI-compatible endpoint across many text, image, and video models. That does not replace direct GPU clusters when you need custom kernels, private weights, fine-tuning control, or topology-specific performance work.

It is useful as a baseline when the business goal is output, not infrastructure ownership. Before reserving a topology-heavy cluster for inference, compare the cluster plan against a managed model/API route:

What quality and latency do you get from hosted models?
What is the cost per accepted response after retries and failed generations?
How much engineering time will cluster operations consume?
Does the workload require direct GPU memory control, or only reliable model outputs?

If the model/API baseline already meets the product requirement, NVLink vs NVSwitch may be an optimization you do not need to own yet. If the workload needs private deployment, custom serving, or hard latency targets at scale, compare GPU cloud vs model API tradeoffs, then use the checklist above to buy the right topology instead of the most recognizable GPU name.

FAQ

Is NVSwitch faster than NVLink?

NVSwitch is not simply "faster NVLink." NVLink is the GPU link. NVSwitch is the fabric that connects multiple NVLink endpoints so many GPUs can communicate efficiently. In supported systems, NVSwitch can preserve high GPU-to-GPU bandwidth across more GPU pairs than a simple point-to-point layout.

Do I need NVSwitch for LLM inference?

You need to care about NVSwitch when inference uses multiple GPUs for one model replica, especially tensor parallelism under latency or throughput targets. If each GPU serves independent single-GPU jobs, NVSwitch may matter much less.

Does NVLink replace InfiniBand?

No. NVLink and NVSwitch are scale-up GPU fabrics. InfiniBand or Ethernet connects nodes, racks, storage, and larger clusters. Large AI systems often need both.

What is the biggest nvlink vs nvswitch difference?

The biggest nvlink vs nvswitch difference is scope. NVLink is a direct GPU communication link. NVSwitch connects many NVLink-equipped GPUs into a broader all-to-all fabric.

Should I choose H100, H200, B200, or GB200 based on NVSwitch?

Do not choose only on the interconnect label. Match the GPU memory, precision support, availability, cooling, software maturity, NVLink domain, scale-out network, and accepted-output cost to your workload. Then run a topology benchmark before committing.

Conclusion

For AI clusters, nvlink vs nvswitch is not a battle between two interchangeable features. NVLink is the high-speed GPU link. NVSwitch is the fabric that makes many NVLink-connected GPUs usable as a larger communication domain. InfiniBand or Ethernet is the scale-out network around that domain.

The buying lesson is simple: ask for the topology, not just the GPU name. If your workload is communication-bound, NVSwitch can be central to performance. If your workload is output-bound and does not require hardware control, benchmark a model/API path with Flatkey before you rent a topology you will have to operate.