The website is down: find out why, one layer at a time
A website is down and nobody knows why. Read what curl tells you, check the server with ssh, systemctl and journalctl, fix it and prove it from outside.
"The website is down." That's all you'll ever be told. The trick is to stop guessing and ask one layer at a time: can I find the machine, can I reach it, is the program running, and why did it stop?
The machines here are pretend ones inside this page (your laptop, web, db and a router), so nothing you do reaches a real network. Everything else works like the real thing.
1. What exactly fails?
Start from where the user is, your laptop, and read the error. curl's error is the most useful clue you'll get:
curl -I http://web/| What you see | What it means |
|---|---|
Could not resolve host | DNS: the name doesn't turn into an address |
Failed to connect … Couldn't connect to server | the machine answered "nothing here": no program listens on that port |
Connection timed out | no answer at all: a firewall drops it, or the machine is off |
HTTP/1.1 502 Bad Gateway | the web server is up, the app behind it isn't |
HTTP/1.1 500 | the app is up and failing |
2. Is the machine there?
ping web checks the machine answers at all. nc -zv web 22 checks one port: if ssh's port answers and 80 doesn't, the machine is fine and the web server isn't.
3. On the server: is it running, and why not?
ssh web systemctl status nginx sudo journalctl -u nginx -n 20 ss -tlnp
systemctl status says whether the service is running and shows its last log lines; journalctl -u shows more of them, with times, which is where "why" usually is. ss -tlnp lists what listens on which port.
4. Fix it, then check from outside
Start or restart the service with sudo systemctl start nginx. Then go back to your laptop (exit) and try again from there. A fix only counts when it works from where the user is.
Your turn
The shop at http://web/ is down. Find out why, get it back up, and check it from your laptop.
Your task
The shop at http://web/ is down. Find out why, get it back up, and check it from your laptop.
- See how it fails from your laptop (not done yet)
- Check nginx on web (not done yet)
- The shop answers again, and you are back on your laptop (not done yet)
The checklist ticks itself off as you work in the terminal: any way that gets the result counts.
Hints come one at a time, then one way to do it. Try each before the next.
What this lesson uses, in one place.
curl -I- Headers only (a HEAD request) More on curl
nc -z- Just check, send nothing
systemctl status- Is it running? recent log lines More on systemctl
journalctl -u- Only this service More on journalctl
ss -tlnp- TCP; listening sockets; numbers, not names; the program using each More on ss
Every everyday command on one page: the Linux commands cheat sheet.