Load Balancing & Clustering

When a Discord bot manages thousands of concurrent voice connections, hosting all players on a single machine leads to thread contention, GC pauses, and UDP packet drops. The Lavalink Gateway distributes player workloads across a dynamic cluster of nodes.

Load Balancing Topology

                      Discord Music Bot (Client)
                                  │
                       WebSocket Handshake (WSS)
                                  ▼
                        Lavalink Gateway
                                  │
         ┌────────────────────────┼────────────────────────┐
         │ (Weight: 35%)          │ (Weight: 40%)          │ (Weight: 25%)
         ▼                        ▼                        ▼
  ┌──────────────┐         ┌──────────────┐         ┌──────────────┐
  │  Lavalink A  │         │  Lavalink B  │         │  Lavalink C  │
  │  (SG Region) │         │  (SG Region) │         │  (EU Region) │
  ├──────────────┤         ├──────────────┤         ├──────────────┤
  │ 350 Players  │         │ 400 Players  │         │ 250 Players  │
  │ CPU: 32%     │         │ CPU: 38%     │         │ CPU: 24%     │
  └──────────────┘         └──────────────┘         └──────────────┘

Selection Heuristics

The Gateway evaluates multiple real-time node telemetry metrics before routing a new player creation request (PLAYER_CREATE / voiceStateUpdate):

1. Active Player Count: Avoids overloading individual nodes past their configured soft limits.
2. CPU & System Load: Prioritizes nodes with lower instantaneous CPU utilization to ensure consistent audio frame delivery.
3. Deficit Frame Rates: Nodes experiencing frame deficits or high jitter are temporarily deprioritized until metrics normalize.
4. Geographic Proximity: Where possible, players in Southeast Asian voice servers are pinned to low-latency Singapore nodes (e.g., <2ms RTT to Discord voice servers).

Node Scaling

Because all load distribution logic is handled centrally by the Gateway:

• New nodes can be provisioned and registered in seconds without reloading client bots.
• Low-traffic periods can trigger automated node scale-down to optimize cloud hosting costs.