Troubleshooting Web Servers, Databases, and Other Services

This guide presents troubleshooting strategies for when you can’t connect to your web server, database, or other services running on your Linode. This guide assumes that you have access to SSH. If you can’t log in with SSH, review Troubleshooting SSH and then return to this guide.

Where to go for help outside this guide

This guide explains how to use different troubleshooting commands on your Linode. These commands can produce diagnostic information and logs that may expose the root of your connection issues. For some specific examples of diagnostic information, this guide also explains the corresponding cause of the issue and presents solutions for it.

If the information and logs you gather do not match a solution outlined here, consider searching the Linode Community Site for posts that match your system’s symptoms. Or, post a new question in the Community Site and include your commands' output.

Linode is not responsible for the configuration or installation of software on your Linode. Refer to Linode’s Scope of Support for a description of which issues Linode Support can help with.

This guide explains how to use different troubleshooting commands on your Linode. These commands can …

General Troubleshooting Strategies

This section highlights troubleshooting strategies that apply to every service.

Check if the Service is Running

The service may not be running. Check the status of the service:

Restart the Service

If the service isn’t running, try restarting it:

Enable the Service

If your system was recently rebooted, and the service didn’t start automatically at boot, then it may not be enabled. Enable the service to prevent this from happening in the future:

Check your Service’s Bound IP Address and Ports

Your service may be listening on an unexpected port, or it may not be bound to your public IP address (or whatever address is desirable). To view which address and ports a service is bound on, run the ss command with these options:

sudo ss -atpu

Review the application’s documentation for help determining the address and port your service should bind to.

Note

One notable example is if a service is only bound to a public IPv4 address and not to an IPv6 address. If a user connects to your Linode over IPv6, they will not be able to access the service.

Analyze Service Logs

If your service doesn’t start normally, review your system logs for the service. Your system logs may be in the following locations:

Your service’s log location will vary by the application, but they are often stored in /var/log. The less command is a useful tool for browsing through your logs.

Try pasting your log messages into a search engine or searching for your messages in the Linode Community Site to see if anyone else has run into similar issues. If you don’t find any results, you can try asking about your issues in a new post on the Linode Community Site. If it becomes difficult to find a solution, you may need to rebuild your Linode.

Review Firewall Rules

If your service is running but your connections still fail, your firewall (which is likely implemented by the iptables software) may be blocking the connections. To review your current firewall ruleset, run:

sudo iptables -L # displays IPv4 rules
sudo ip6tables -L # displays IPv6 rules

Note

Your deployment may be running FirewallD or UFW, which are frontends used to more easily manage your iptables rules. Run these commands to find out if you are running either package:

sudo ufw status
sudo firewall-cmd --state

Review How to Configure a Firewall with UFW and Introduction to FirewallD on CentOS to learn how to manage and inspect your firewall rules with those packages.

Firewall rulesets can vary widely. Review the Control Network Traffic with iptables guide to analyze your rules and determine if they are blocking connections. For example, a rule which allows incoming HTTP traffic could look like this:

-A INPUT -p tcp -m tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT

Disable Firewall Rules

In addition to analyzing your firewall ruleset, you can also temporarily disable your firewall to test if it is interfering with your connections. Leaving your firewall disabled increases your security risk, so we recommend re-enabling it afterward with a modified ruleset that will accept your connections. Review Control Network Traffic with iptables for help with this subject.

1. Create a temporary backup of your current iptables:

   sudo iptables-save > ~/iptables.txt
   

2. Set the INPUT, FORWARD and OUTPUT packet policies as ACCEPT:

   sudo iptables -P INPUT ACCEPT
   sudo iptables -P FORWARD ACCEPT
   sudo iptables -P OUTPUT ACCEPT
   

3. Flush the nat table that is consulted when a packet that creates a new connection is encountered:

   sudo iptables -t nat -F
   

4. Flush the mangle table that is used for specialized packet alteration:

   sudo iptables -t mangle -F
   

5. Flush all the chains in the table:

   sudo iptables -F
   

6. Delete every non-built-in chain in the table:

   sudo iptables -X
   

7. Repeat these steps with the ip6tables command to flush your IPv6 rules. Be sure to assign a different name to the IPv6 rules file (e.g. ~/ip6tables.txt).

Troubleshoot Web Servers

If your web server is not running or if connections are timing out, review the general troubleshooting strategies.

Note

Troubleshooting specific to Apache is outlined in Troubleshooting Common Apache Issues.

If your web server is responding with an error code, your troubleshooting will vary by what code is returned. For more detailed information about each request that’s failing, read your web server’s logs. Here are some commands that can help you find your web server’s logs:

• Apache:

  grep ErrorLog -r /etc/apache2  # On Ubuntu, Debian
  grep ErrorLog -r /etc/httpd    # On CentOS, Fedora, RHEL
  

• NGINX:

  grep error_log -r /etc/nginx
  

Frequent Error Codes

• HTTP 401 Unauthorized, HTTP 403 Forbidden

  The requesting user did not have sufficient permission or access to the requested URL. Review your web server authorization and access control configuration:

  • Apache - Access Control

  • Apache - Authentication and Authorization

  • NGINX - Restricting Access with HTTP Basic Authentication

• HTTP 404 Not Found

  The URL that a user requested could not be found by the web server. Review your web server configuration and make sure your website files are stored in the right location on your filesystem:

  • Apache - Mapping URLs to Filesystem Locations

  • NGINX - How nginx processes a request

• HTTP 500, 502, 503, 504

  The web server requested a resource from a process it depends on, but the process did not respond as expected. For example, if a database query needs to be performed for a web request, but the database isn’t running, then a 50X code will be returned. To troubleshoot these issues, investigate the service that the web server depends on.

Troubleshoot Databases

Is your Disk Full?

One common reason that a database may not start is if your disk is full. To check how much disk space you are using, run:

df -h

Note

This reported disk usage is not the same as the reported storage usage in the Linode Manager. The storage usage in the Linode Manager refers to how much of the disk space you pay for is allocated to your Linode’s disks. The output of df -h shows how full those disks are.

You have several options for resolving disk space issues:

• Free up space on your disk by locating and removing files you don’t need, using a tool like ncdu.

• If you have any unallocated space on your Linode (storage that you pay for already but which isn’t assigned to your disk), resize your disk to take advantage of the space.

• Upgrade your Linode to a higher-tier resource plan and then resize your disk to use the newly available space. If your Linode has a pending free upgrade for your storage space, you can choose to take this free upgrade to solve the issue.

Database Performance Troubleshooting

If your database is running but returning slowly, research how to optimize the database software for the resources your Linode has. If you run MySQL or MariaDB, read How to Optimize MySQL Performance Using MySQLTuner.