The challenge of running lavalink

By Darren Nathanael on Sat, Feb 12, 2022 / Updated at Wed, Feb 14, 2024

Hello 👋

Hi, I’m Darren! I am a sysadmin. Meaning I have a lot of servers that I rent for testing and development. I also have a discord music bot Model Buses that uses lavalink.

Having one lavalink server isn’t enough…. jk it’s prob enough, but I like to have a few more for redundancy.

As of writing this, my bot model buses are on 97 guilds, which isn’t that much. So having multiple lavalink servers is kind of overkill. But hey! That’s me! I like overkill stuff. And overcomplicating things.

Update: Discord Support as per usual, hated me. anyhow the bot is now offline.

Why

Well, the reason I created this is that prominent lavalink hosting such as lava.link which is owned by something.host is great, But it’s dropping WS connection after a while due to it using Cloudflare. And it also disallowed playback from HTTP, Bandcamp and SoundCloud AFAIK, which is sad because I use those services. So I created this. Lavalink hosting that doesn’t use Cloudflare and doesn’t drop WS connections every few hours has all the features such as filters and all sources enabled.

This cost a lot of money to run, so if you are using it and would like to support me, you can donate via https://ko-fi.com/darrenofficial or by joining my Discord server.

Currently, I pay around $380/year with all the nodes combined, around 30 dollars per month.

Currently, I have three nodes running; those are equipped with 16 GB ram and a six-core processor. And I have one HAProxy instance running on my webserver.

How do I load balance it?

HAProxy. Those HAProxy uses lessconn to load-balance the traffic. I have an HAProxy instance running on my OVH instance.

YouTube Rate limiting

YouTube has a rate limit. I have 2 X /48 IPv6 on each node to encounter it, which is around two million IPs per node.

IPv6 Routes Table

Discord voice channels/gateway are fragile

Discord has global networks all around the world to serve as voice servers. All audio is delivered over a UDP socket to the voice server. Discord was not designed for playing music, so it’s prone to drop audio frames. Due to the nature of UDP, a packet can be lost/dropped or for a packet to be duplicated. These packets are not resent if it’s dropped. This problem also appears on the receiver’s side, as they also use a UDP connection to the voice server.

As of me writing this, my lavalink is connected to 859 voice channels, and there are over 107 actively playing music. Each of the nodes handles around 300 bots/connections.

Sometimes my lavalink has small dropped frames; these dropped frames are called “stuttering”, or sometimes, it just outright disconnects the bot, making it unusable.

Another cause for stuttering is when the voice server itself is misbehaving, where users are forced to manually change their voice region (and thus voice server). Most users & devs don’t realise this and blame the lavalink server/music bot for not working correctly.

Updating and maintaining

Lavalink is a wrapper for lavaplayer, a java library that handles audio playback. This means that occasionally I have to restart the server for updates. I solved this issue by using “soft maintenance”, a feature on HAProxy that allows me to softly shut down the server, meaning no disconnect happens on the client-side.

Support me

Running this is expensive, so I’m open to any donations.

Discord: DarrenOfficial#3451

Discord Server: https://discord.darrennathanael.com/

Ko-fi: https://ko-fi.com/darrenofficial