Why is my site consistently showing not found?

I recently connected migrated my website from Bluehost to a private server, and I am having trouble with the configuration of my apache server. No matter what page I attempt to visit on my site, it displays a 404 this url is not found on this server error.

My site is written in both perl and php. Mostly perl. Would a perl version mismatch between my old server and my new server make the not found error appear? I would think that would cause a 500 server error.

What am I doing wrong? What do I change in order to make it work?

I have played around with the part of my httpd.conf file that controls my website.

My site is behind a reverse proxy that points to my backend server running the site at 333.333.333.333/test

Here it is:

SuexecUserGroup "#1001" "#1001"
ServerName test.example.org
ServerAlias www.test.example.org
ServerAlias mail.test.example.org
ServerAlias webmail.test.example.org
ServerAlias admin.test.example.org
DocumentRoot /home/theuser/public_html/testrbs
ErrorLog /var/log/virtualmin/test.example.org_error_log
CustomLog /var/log/virtualmin/test.example.org_access_log combined
ScriptAlias /cgi-bin/ /home/theuser/public_html/test/cgi-bin
DirectoryIndex index.cgi index.html index.htm index.php index.php4 index.php5
<Directory "/home/theuser/public_html/test">
Options -Indexes +IncludesNOEXEC +SymLinksIfOwnerMatch 
allow from all
AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
Require all granted
</Directory>
<Directory "/home/theuser/public_html/test/cgi-bin">
allow from all
AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch
Require all granted
</Directory>

The 404 error you’re encountering suggests that your Apache server cannot find the requested files or directories, rather than an issue with the scripts themselves (like a Perl or PHP version mismatch). Let’s systematically address the problem based on your setup:

1. Verify the DocumentRoot Path

The DocumentRoot you specified is:

DocumentRoot /home/theuser/public_html/testrbs

Ensure the directory /home/theuser/public_html/testrbs exists and contains the appropriate files, particularly index.cgi, index.html, or other index files listed in your DirectoryIndex.

Run the following command to check:

ls -la /home/theuser/public_html/testrbs

If the directory doesn’t exist or is incorrect, update the DocumentRoot to point to the actual directory where your website files are stored.

2. Verify the Reverse Proxy Setup

You mentioned the site is behind a reverse proxy pointing to 333.333.333.333/test. Confirm that:

  1. The proxy server is correctly configured and forwarding requests to your backend server.
  2. Apache on the backend server is serving files from /home/theuser/public_html/testrbs.

If your reverse proxy is not correctly routing requests, you may need to update its configuration.

3. Check File and Directory Permissions

Ensure Apache has read and execute permissions for the files and directories. Run the following commands to set appropriate permissions:

sudo chown -R theuser:theuser /home/theuser/public_html/testrbs
sudo chmod -R 755 /home/theuser/public_html/testrbs

Additionally, verify that Apache can access the parent directories (/home and /home/theuser). If necessary, add execute permissions:

sudo chmod o+x /home /home/theuser

4. Confirm the ScriptAlias Directive

You have the following configuration for CGI scripts:

ScriptAlias /cgi-bin/ /home/theuser/public_html/test/cgi-bin

Ensure the directory /home/theuser/public_html/test/cgi-bin exists and contains the necessary scripts. Also, check permissions to ensure Apache can execute the scripts:

ls -la /home/theuser/public_html/test/cgi-bin

If the directory or scripts are missing, update the ScriptAlias path or ensure the files are correctly placed.

5. Validate the AllowOverride Directive

You have the following in your configuration:

AllowOverride All Options=ExecCGI,Includes,IncludesNOEXEC,Indexes,MultiViews,SymLinksIfOwnerMatch

This directive allows .htaccess files to override specific settings. Check for a .htaccess file in /home/theuser/public_html/testrbs that might conflict with your configuration. For example, it might contain rules causing the 404 error.

6. Enable CGI Execution

Make sure Apache is configured to execute CGI scripts. Ensure the mod_cgi module is enabled:

sudo a2enmod cgi
sudo systemctl restart apache2

If you’re using mod_fcgid or mod_proxy_fcgi, confirm they are properly configured for Perl scripts.

7. Debug the Error Log

Check the Apache error log for detailed information about why the 404 error is occurring:

sudo tail -f /var/log/virtualmin/test.example.org_error_log

The log may indicate specific missing files or misconfigurations causing the issue.

8. Validate Proxy Configuration

If requests are proxied to 333.333.333.333/test, confirm the backend server has a matching configuration to serve files correctly. Update the reverse proxy configuration to include:

ProxyPass / http://333.333.333.333/test/
ProxyPassReverse / http://333.333.333.333/test/

9. Restart Apache

After making changes, restart Apache to apply them:

sudo systemctl restart apache2

10. Test with Minimal Configuration

If the issue persists, simplify your configuration temporarily to isolate the problem. For example:

<VirtualHost *:80>
    ServerName test.example.org
    DocumentRoot /home/theuser/public_html/testrbs

    <Directory "/home/theuser/public_html/testrbs">
        Options +ExecCGI
        AllowOverride All
        Require all granted
    </Directory>
</VirtualHost>

11. Verify the Perl and PHP Setup

While a mismatch in Perl or PHP versions usually leads to a 500 error, ensure that:

  • Perl scripts include the correct shebang line (e.g., #!/usr/bin/perl).
  • Necessary Perl and PHP modules are installed.

You can test individual scripts using:

perl /home/theuser/public_html/test/cgi-bin/example.cgi

Conclusion

Start by verifying the directory structure, permissions, and proxy setup. Simplify the Apache configuration and progressively add complexity once the basic setup works. Checking the error logs will be critical in pinpointing the exact issue. Let me know if you encounter further challenges!