And not without good reason, NGINX is, by far, the best HTTP(S) server available on Linux, below I will share why I believe this, introduce you to some features you might not be familiar with.

Table of Contents

• Rate Limiting
  • Rate limiting AI-Crawlers
• Reverse Proxy
  • Load Balancing
• Cool snippets
  • Aggressively rate-limit clients without User-Agents
  • Access-Control based on IP-Address

Rate Limiting

If you are anything like me, you did not initially know that this is even a thing.

I used to think that implementing rate-limiting was up to the micro/macro/whatever service that NGINX is “managing”.

In reality, NGINX ships with a very powerful and well-thought-out rate-limiting module by default, and it’s pretty expressive and useful when you have some tricks up your sleeve.

I will not cover the fundamentals here, you can find the technical details in the official NGINX documentation. Instead I want to introduce some more advanced useful use-cases I found for this module.

Rate limiting AI-Crawlers

Lets face reality for a second. OpenAIs ChatGPT, Anthropics Claude, and Googles Gemini are here to stay. Unfortunately for us poor system administrators, this means an unusual amount of traffic we need to deal with, for as long as the arms-race for the best possible model between these behemoths runs.

Fortunately for us, we can rate limit only the AI-Crawlers without restraining actual organic traffic.

Meet the map-directive and how we can leverage this to select only particular user agents. a[ai_crawlers]

map $http_user_agent $ai_crawlers {
    default "";
    ~(GPTBot|ClaudeBot|anthropic-ai|Google-Extended|GoogleOther|Bytespider|CCBot|Amazonbot|FacebookBot|AppleBot-Extended|Cohere-ai) $1;
}

limit_req_zone $ai_crawlers zone=ai_zone:16m rate=1r/s;
limit_req_status 429;

server {
    location / {
        limit_req zone=ai_zone nodelay burst=5;
    }
}

This limits all User-Agents contained in the above RegExp to 1 request per second.

I first encountered this on StackOverflow where the solutions are similar, but in my opinion not as elegant as this one.

The map matches against a regular expression that lists multiple strings contained within the AI-Crawler User-Agents. By matching those strings in a capture list, we can access the string we found via the $1 capture variable, which is what we use as the value at the end of the line.

You could also drop the $1 from that line in favour of something else, say bad bad bot.

If you do that, all those User-Agents evaluate to the bad bad bot string, which means that every client whose User-Agent evaluates to that string shares the rate-limit. I.e. Googles Gemini would share the rate-limit with Anthropic, etc.

The value bad bad bot itself doesn’t matter, this is arbitrary. I will explain shortly.

Let us understand what the limit_req_zone does.

limit_req_zone defines a shared memory zone (what the workers use to communicate, the zone=ai_zone:16m in our example means the zone will be 16MiB in size.) into which a hashmap is placed. The first parameter ($ai_crawlers, the map) serves as the key into the hashmap.

Lets manually unravel the logic. Say a client with the User-Agent Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.3; +https://openai.com/gptbot hits our server, if we follow the $ai_crawlers map, we see that this User-Agent gets mapped to GPTBot

NOTE: Actually, it maps to $1, but expanding that according to the capture list, it becomes GPTBot.

The request gets inserted in the the shared hashmap at key(GPTBot), any further requests that match the User-Agent also get sent into that key. This limits all clients matching that string (GPTBot).

As an attentive reader, you might be concerned about the default "", would this not mean that all User-Agents not matching that User-Agent also get limit to 1r/s? Since key("") will always yield the same value?

No. NGINX got our back, as per the documentation:

Syntax: limit_req_zone key zone=name:size rate=rate; Default: — Context: http Sets parameters for a shared memory zone that will keep states for various keys. In particular, the state stores the current number of excessive requests. The key can contain text, variables, and their combination. Requests with an empty key value are not accounted. NGINX Documentation

Reverse Proxy

The most widely utilised feature of NGINX is its reverse proxy capability. How to set this up is trivial and documented in thousands of places.

But did you know about load balancing?

Load balancing is done by the ngx_http_upstream_module module, which has a handful of nifty features improving reliability of your service under pressure.

First of all, lets see how a basic upstream can look:

upstream backend {
    server backend1.example.com weight=5;
    server 127.0.0.1:8080       max_fails=3 fail_timeout=30s;
    server unix:/tmp/backend3;

    server backup1.example.com  backup;
}

As you can see, servers can be a multitude of things. 1. UNIX sockets . External Domains/Servers . IP-Addresses

But each server can also have parameters, and these are where we start having fun, you can read more about all possible parameters here.

Load Balancing

Within a upstream group, we can instruct NGINX to perform load balancing for incoming requests via directives. There a number of them we should be aware of, by default NGINX dials our upstreams servers round-robin.

1. least_conn Routes incoming requests to whatever server has the least amount of active connections. . least_time Routes incoming requests to whatever server has the shortest average response time and least amount of connections. . ip_hash Fixes incoming requests to a given upstream by the clients IP address, all requests originating from that client will end up on the same server. _. hash *key* Same as ip_hash, but the key is freely definable, if you have read limiting AI-Crawlers, same principle is applicable here. Remember that we can map lots of things!

Other cool snippets

Aggressively rate-limit clients without User-Agents

On a production server, I noticed that an unusual amount of bad bots scanning for vulnerabilities do not send a User-Agent.

map $http_user_agent $ua_missing {
    ""      1;
    default 0;
}

location / {
    if ($ua_missing) {
        return 403;
    }
}

Access-Control based on IP-Address

A colleague once asked me whether it is possible to only allow specific IPs to certain routes.

It is

location /api {
    satisfy any;
    allow     10.0.0.0/8;
    allow    127.0.0.0/8;
    allow  172.16.0.0/20;
    allow 192.168.0.0/16;
    deny all;

    # ...
}

What is Gentoo

To those unfamiliar, Gentoo is a Linux distribution with a unusually unique approach to package management.

While virtually every major distribution under the sun distributes binary packages in their package repositories, Gentoo’s package manager portage primarily distributes not binary packages, but recipes to compile the package locally on your machine.

With that out of the way, let’s dive in why this is better than Ubuntu, or even Arch.

The good

… Maturity

Gentoo is mature. Having been around since 2002, it stood against the test of time.

… Source-first

Gentoo most prominent perk is that the distribution is source-first.

This means is that every package, and dependency your machine needs will be compiled on your machine, with your compiler, your compiler flags, and your specific requirements (called USE-flags).

You might think, “Why would I want to compile all my packages, that takes ages!”, You aren’t wrong. A system update usually takes a few hours. But, you get rewarded with a remarkably stable system. When I first started with Gentoo, I expected the distribution to be either rolling-release, or even bleeding-edge, but surprisingly the Gentoo team is very determined to keep systems as stable as can be.

They succeed at this, too. With tools like eselect news read which directly distributes concise, easy-to-follow, news about your installed packages directly in the terminal.

… Customization

Gentoo heavily lies into customization via USE flags, these enable the user to only enable certain parts of software that they actually need.

If you read this blog post, you’re probably pretty familiar with configure scripts, and the general process of compiling software. If not, this following passage will probably not sound like such a great feature to you, but alas. USE-flags in Gentoo specify what features of a package should be enabled when it is compiled. This is directly useful to make you system slim, and prevent bloating the installation with stuff you don’t actually need.

Examples

PostgreSQL: For development, I need the PostgreSQL headers, and client installed on my system. Depending on the distribution, you might be forced to pull in both client and server, when you only need the client. On Gentoo, I set the USE-flag dev-db/postgresql -server, with this, Portage only builds the client. The server binary is never produced.

Waybar: Waybar has tons of modules for different purposes, GPS, or WiFi, or Backlight, etc. Most of which I don’t need, I am on a desktop, I don’t have WiFi, nor Backlight, nor a battery nor GPS. Gentoo enables me to only select the parts of Waybar that I need, I don’t need to pull in mpd or something that I don’t want or need on my system. This directly reduces the amount of dependencies on my system, as well as the compilation times. Neat.

Emacs: Certain features are hidden between flags. When Treesitter support first landed in Emacs 29, the (IIRC, Arch-Linux Package) didn’t have it enabled by default, and there was no way of enabling Treesitter until the packagers came around to publishing a new version with new dependencies. Back then, I removed Emacs via pacman and manually compiled it to get the functionality, but then I was pinned to that point in time, unless I manually update it. On Gentoo, I now just cherry-pick the USE-flags my Emacs installation needs, and I only get those features compiled, no need to add in X11 support, I am on Wayland.

… Security

Security is what Gentoo provides in a manner that Ubuntu, and consorts cannot due to their packaging philosophy.

If you’re familiar with tech and open-source, you have not missed the increase in supply-chain attacks in recent time. First there was the almost catastrophic libxz takeover, then there have been numerous attacks on the NPM ecosystem. These are all very real threats that we need to be aware of, whether you’re a private user, or you maintain a number of servers.

What Gentoo offers me is peace of mind. My machine compiles the packages it needs locally plus they are pinned for long enough, that transient threats (such as the libxz attack) will probably have blown over by the time they land on my system. Now granted, you cannot be 100% secure against supply-chain attacks, but being source-first mitigates the threat stemming from “black box” binaries. I shiver to this day when I think about what could have been if Andres Freund hadn’t caught the libxz attack…

With binary blobs, you cannot even check what your system is running. Sure, you might go to GitHub/FreeDesktop/Whatever and check the source out, but can you guarantee that the source wasn’t patched before landing on your system as a binary blob? … Let’s paint an even darker picture, what if the packagers themselves are not even malicious, but instead were infected with a backdoored compiler by some state actor?

What running binary blobs means is that you are running code compiled by someone you probably do not know, with patches you have not seen, on a machine you do not trust.

Now, in the real world the threat from all this is rather slim. Ubuntu, as far as I am aware, builds the core PPA in their build-farm, i.e. dedicated machines that do nothing but compile packages, already quite good. Generally, I believe it’s save to say that commercially supported distributions such as Ubuntu or Fedora should carry less risk here. Archlinux, with the AUR and its *-bin packages, carries the biggest risk. Introducing malware to uninformed users has the lowest entry burden here.

Now, in a perfect world, packages are compiled by in-person verified and trusted parties on strictly air-gapped machines that themselves have been produced from-source by verifiably non-backdoored compilers, where package sources are carried over via USB-Stick…

… Yeah, that doesn’t sound feasible. This threat of being fed malicious software stays pervasive on any distribution, not even Gentoo is immune against this, but being source-first, Gentoo does a damn good job of minimising the threat from binaries.

… Documentation

The Gentoo Wiki is a phenomenal, well written, knowledge base about all aspects of your system.

But the real kicker are the tools distributed with the portage.

equery

equery has a long list of things it can do, but what I more often than not had to use it for are three things:

1. equery u app-editors/emacs: shows you what USE-flags are enabled, and disabled for this package. With a short description about what the flag does if enabled. . equery f app-editors/emacs: lists all files installed by this package. . equery b /usr/share/emacs/site-lisp: does the reverse of f, and searches for the package that installed the given file.

eselect

eselect has two main use-cases.

One that is similar toupdate-alternatives, updating symlinks to use a particular version of Java, Rust, Lua, whatever.

The other is reading news, this is where Gentoo maintainers publish upcoming changes, bumping the default python version, etc. All with documentation on what this means for you, and what you might need to do to ensure clean updates.

The bad

… Compile times

Being a source-first distribution, your machine will spend a lot of time compiling your packages, even on beefy hardware. Running an emerge --update @world might take a few hours (looking at you, www-browser/chromium)

Granted, this can be mitigated by using --getbinpkg, but that directly negates the good this distribution brings.

… Installation

It has only been a few months since moving to this distro, and yet I remember the installation being a pain in the ass.

Installing Archlinux back in the day with pacstrap and whatnot after coming from a GUI-based distribution was already a challenge. But the complexity of installing Gentoo does absolutely surpass that of a text-based Archlinux installation.

… Conflicts

While updating usually works, when you do run into some incompatibility, it is a major pain in the ass.

The most painful case are updates to deep-rooted dependencies. dev-libs/boost dropping a new version can delay updates until dependant packages get pushed to the new version.

To outline this, here is how a dependency conflict looks, when the above case occurs:

Calculating dependencies... done!
Dependency resolution took 12.00 s (backtrack: 0/20).


Total: 0 packages, Size of downloads: 0 KiB

WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict:

x11-libs/wxGTK:3.2-gtk3

  (x11-libs/wxGTK-3.2.8.1-r2:3.2-gtk3/3.2::gentoo, ebuild scheduled for merge) USE="X libnotify lzma opengl sdl spell tiff wayland -curl -debug -doc -gstreamer -keyring (-pch) -test -webkit" ABI_X86="(64) -32 (-x32)" conflicts with
    >=x11-libs/wxGTK-3.2.7:3.2-gtk3=[gstreamer,libnotify,opengl,sdl,tiff,X] required by (dev-python/wxpython-4.2.4:4.0/4.0::gentoo, installed) USE="-debug -test -webkit" ABI_X86="(64)" PYTHON_TARGETS="python3_13 -python3_11 -python3_12 -python3_14"
                                     ^^^^^^^^^                             
    >=x11-libs/wxGTK-3.2.7:3.2-gtk3/3.2=[gstreamer,libnotify,opengl,sdl,tiff,X] required by (dev-python/wxpython-4.2.4:4.0/4.0::gentoo, installed) USE="-debug -test -webkit" ABI_X86="(64)" PYTHON_TARGETS="python3_13 -python3_11 -python3_12 -python3_14"
                                         ^^^^^^^^^                             

media-video/ffmpeg:0

  (media-video/ffmpeg-8.1:0/60.62.62::gentoo, ebuild scheduled for merge) USE="X alsa bzip2 cairo dav1d drm dvd fontconfig gnutls gpl jpegxl lame lcms libass opengl opus pulseaudio sdl svg svt-av1 theora truetype vaapi vorbis vpx vulkan webp x264 xml zlib -amf -amr -amrenc (-appkit) -bluray -bs2b -cdio -chromaprint -chromium -codec2 (-cuda) -doc -fdk -flite -frei0r -fribidi -gcrypt -gme -gmp -gsm -iec61883 -ieee1394 -jack -jpeg2k -kvazaar -ladspa -libaom -libaribb24 -libcaca -libilbc -liblc3 -libplacebo -librtmp -libsoxr -lv2 -lzma -modplug -nvenc -ocr -openal -opencl -opencolorio -openh264 -openmpt -openssl -qrcode -qsv -quirc -rabbitmq -rav1e -rist -rubberband -samba -snappy -sndio -speex -srt -ssh -twolame -v4l -vdpau -verify-sig -vidstab -vmaf -x265 -xvid -zeromq -zimg -zvbi" ABI_X86="(64) -32 (-x32)" conflicts with
    media-video/ffmpeg:0/59.61.61= required by (dev-qt/qtmultimedia-6.10.3:6/6.10.3::gentoo, installed) USE="X alsa dbus ffmpeg opengl pipewire pulseaudio qml v4l vulkan wayland -custom-cflags -eglfs -gstreamer -test -vaapi" ABI_X86="(64)"
                      ^^^^^^^^^^^^
    >=media-video/ffmpeg-6.1:0/59.61.61=[opus,x264] required by (media-video/obs-studio-32.1.0-r1:0/0::gentoo, installed) USE="alsa pipewire pulseaudio truetype wayland -browser -decklink -fdk -jack -lua (-mpegts) -nvenc -python -qsv -sndio -speex -test-input -v4l -vlc -websocket" ABI_X86="(64)" LUA_SINGLE_TARGET="luajit" PYTHON_SINGLE_TARGET="python3_13 -python3_11 -python3_12 -python3_14"
                            ^^^^^^^^^^^^           
    media-video/ffmpeg:0/59.61.61= required by (media-libs/tg_owt-0_pre20250515-r2:0/20250515::gentoo, installed) USE="X screencast" ABI_X86="(64)"
                      ^^^^^^^^^^^^
    media-video/ffmpeg:0/59.61.61= required by (net-misc/freerdp-3.24.2:3/3::gentoo, installed) USE="X aad alsa client cups ffmpeg fuse icu jpeg pulseaudio sdl usb xv -camera -debug -gstreamer -kerberos -openh264 -server -smartcard -systemd -test -valgrind -verify-sig (-wayland) -xinerama" ABI_X86="(64)"
                      ^^^^^^^^^^^^
    media-video/ffmpeg:0/59.61.61=[encode(+),lame(-),opus,theora,vorbis,vpx,x264,xvid] required by (media-gfx/blender-4.4.3:4.4/4.4::gentoo, installed) USE="X bullet color-management cycles cycles-bin-kernels embree ffmpeg fftw fluid gmp nanovdb nls oidn openexr opengl openmp openpgl opensubdiv openvdb pdf pipewire potrace pugixml pulseaudio sdl sndfile tbb tiff truetype vulkan wayland webp -alembic -collada (-cuda) -debug -doc -gnome (-hip) (-hiprt) -jack -jemalloc -jpeg2k -man -ndof -openal -optix -osl -renderdoc -test -valgrind" ABI_X86="(64)" LLVM_SLOT="19 -18" PYTHON_SINGLE_TARGET="python3_13 -python3_11 -python3_12"
                      ^^^^^^^^^^^^                                               ^^^^ 
    >=media-video/ffmpeg-6.1:0/59.61.61=[encode(+),threads(+)] required by (media-video/mpv-0.41.0-r1:0/2::gentoo, installed) USE="X alsa cdda cli drm dvd egl iconv jpeg lcms libmpv lua pipewire pulseaudio uchardet vulkan wayland xv zlib (-aqua) -archive -bluray (-coreaudio) -debug -dvb -gamepad -jack -javascript -libcaca -nvenc -openal -rubberband -sdl (-selinux) -sixel -sndio -soc -test -tools -vaapi -vdpau -zimg" ABI_X86="(64)" LUA_SINGLE_TARGET="(luajit) (-lua5-1)" PYTHON_SINGLE_TARGET="python3_13 -python3_11 -python3_12 -python3_14"
                            ^^^^^^^^^^^^                      
    <media-video/ffmpeg-8:=[encode(+),lame(-),opus,theora,vorbis,vpx,x264,xvid] required by (media-gfx/blender-4.4.3:4.4/4.4::gentoo, installed) USE="X bullet color-management cycles cycles-bin-kernels embree ffmpeg fftw fluid gmp nanovdb nls oidn openexr opengl openmp openpgl opensubdiv openvdb pdf pipewire potrace pugixml pulseaudio sdl sndfile tbb tiff truetype vulkan wayland webp -alembic -collada (-cuda) -debug -doc -gnome (-hip) (-hiprt) -jack -jemalloc -jpeg2k -man -ndof -openal -optix -osl -renderdoc -test -valgrind" ABI_X86="(64)" LLVM_SLOT="19 -18" PYTHON_SINGLE_TARGET="python3_13 -python3_11 -python3_12"
    ^                   ^                                                 ^^^^ 
    >=media-video/ffmpeg-6:0/59.61.61=[opus,vpx] required by (net-im/telegram-desktop-6.5.1:0/0::gentoo, installed) USE="X dbus fonts libdispatch screencast wayland -enchant -webkit" ABI_X86="(64)"
                          ^^^^^^^^^^^^          


Nothing to merge; quitting.

1. dev-qt/qtmultimedia
2. media-video/obs-studio
3. media-libs/tg_owt
4. net-misc/freerdp
5. media-gfx/blender
6. …

… all still depend on media-video/ffmpeg-59.61.61

Afterthoughts

I am still very new to Gentoo, but in the short time I have yet had with the distribution, it is, by a long shot, my favourite approach to Linux.

Before moving to Gentoo, I used to run Artix (Archlinux without systemd), but I have never felt more in satisfied with my computer and the software it is running than I have since moving to Gentoo.