Description
Welcome to Failure Failure - a high-available system. This challenge simulates a real-world failover scenario where one server is prioritized over the other. A load balancer stands between you and the truth - and it won't hand over the flag until you force its hand.
Setup
Launch the challenge instance. The instance details page lists the SSH host, port, username, and either a password or an SSH key. There are no out-of-band credentials - they only appear on that page.
The service sits behind a load balancer that routes between a primary and failover server.
Solution
Walk me through it- Step 1SSH in and identify the servicesSSH in using the user/host/port shown on the instance details page (and either the listed password or the provided key). Once on the box, list active systemd units so you know which is primary and which is the failover before stopping anything.bash
ssh <user>@<HOST> -p <PORT_FROM_INSTANCE>bash# Once connected, find the primary vs secondary services:bashsystemctl list-units --type=service --state=runningbashsystemctl list-unit-files --type=service | grep -E 'primary|secondary'Learn more
SSH (Secure Shell) provides an encrypted terminal session over a network. In this challenge it gives you an interactive shell on the remote server where you can inspect and control running services directly. SSH is the standard administration channel for Linux servers in both enterprise and cloud environments.
The challenge models a high-availability (HA) architecture where two services run in an active-passive pair. A load balancer or init system health-checks the primary; if it fails, traffic or responsibility shifts to the secondary. Understanding how services start, stop, and fail over is fundamental to Linux systems administration and also to understanding how attackers abuse misconfigurations in these setups.
- Step 2Stop the primary serviceThe primary service is running and takes priority. Stop it with systemctl so the secondary service (which contains the flag) becomes active.bash
sudo systemctl stop primary.servicebash# Check that the secondary is now active:bashsudo systemctl statusLearn more
systemd is the init system used by most modern Linux distributions. It manages services as units described in
.servicefiles. Thesystemctl stopcommand sendsSIGTERM(and thenSIGKILLif the process does not exit) to the service's main process and marks the unit as inactive. Stopping a service is non-destructive: it does not delete the service file or prevent the service from being restarted later.Services can declare ordering and dependency relationships with
After=,Requires=,Wants=, andBindsTo=. A failover service might be configured withAfter=primary.serviceand triggered automatically when the primary exits. Alternatively, a systemd socket unit or a separate health-check timer can activate the secondary. Understanding these unit relationships helps diagnose why a secondary service does or does not start automatically.In production HA systems, intentional service failures are tested with chaos engineeringtools like Netflix's Chaos Monkey. The principle is the same as this challenge: verify that the failover path actually works by deliberately stopping the primary.
- Step 3Read the flag from the secondary serviceOnce the primary is down, tail just the secondary service's logs. Scoping to one unit avoids drowning in unrelated journal output.bash
journalctl -u secondary.service -n 50bash# Or follow live:bashjournalctl -u secondary.service -fbashsudo systemctl status secondary.serviceLearn more
journald is systemd's logging daemon. It captures stdout, stderr, and syslog output from every service and stores them in a structured binary journal.
journalctl -ftails the journal in real time (liketail -fon a log file). You can filter by unit withjournalctl -u secondary.serviceto see only that service's output.Secrets should never be printed to the system journal in production. Journal files are typically readable by any user in the
systemd-journalgroup and are sometimes forwarded to centralised logging systems where many people have access. This challenge deliberately puts the flag in the journal to make the failover path observable. In real systems, secrets are read from environment variables, mounted secrets volumes (in Kubernetes), or secret management services like HashiCorp Vault or AWS Secrets Manager.The
systemctl statuscommand shows a service's current state, the last few log lines, and key metadata like its PID and memory usage. It is the first command to run when diagnosing a misbehaving service in production.For more on the surrounding Linux command-line workflow, see the Linux CLI for CTF post.
Flag
picoCTF{f41lur3_f41lur3_...}
The flag lives in the secondary (failover) service. SSH in and stop the primary service with systemctl - the secondary takes over and reveals the flag via journalctl.