What are access logs?

A web server’s access log file contains detailed information about each request made to the server. While it’s possible to format the log entries to include certain information about each request, access logs typically list details such as:

• What files were requested – HTML, images, other website content, etc.
• Request type, such as GET, POST, etc.
• What kind of device and browser made the request
• Which pages users, crawlers, and bots are viewing
• The IP address of the computer or device making the request
• HTTP status codes showing the results of each request
• Server response time

What is the purpose of access logs?

Even though the information contained in access logs might seem basic, it can reveal tons of valuable data. Using the insights provided by access logs can identify opportunities for improvements to server and website performance and security, user experience, marketing campaigns, and more.

Marketing

Marketing and web professionals all benefit from knowing what pages referred visitors and what actions those visitors take when they land on web pages. Information such as geo-specific IP addresses show where visitors live and whether or not they are returning visitors.

User experience

Monitoring access logs can also provide insight into a site’s technical SEO by exposing excessive HTTP error codes or redirects and watching how search crawlers access pages. Too many redirects or errors may hinder a website’s overall SEO performance. Watching crawl patterns can alert website owners as to how search engines see website content and identify areas for improvement for specific pages.

Access logs also tell a story about how users naturally navigate through a website, revealing how that experience can be improved. Visitors often follow a similar pathway of links from page to page within a site. By identifying these types of pages, they can be preloaded using different techniques, resulting in faster page loading times. 

Troubleshooting

Troubleshooting issues often begins with looking through the access log and error log. Since the access log contains an entry showing the status of each incoming request, it helps shed light on where issues need attention.

Website security

Monitoring access logs can expose malicious behaviors, which may help identify ways to improve security and prevent attacks. If a website has suffered an attack or security breach, the access log shows exactly what data was requested.

Web server access logs

Apache and Nginx servers both create access log files and store them in slightly different file locations. Depending on the operating system (OS) and configuration, these paths may be different. 

FTP or SFTP is often used to access the access.log file on the server. Users can then view log files with a text editor like Notepad++ or Atom.  

Since access log files can be pretty large, loading them in a text editor can be sluggish. Command line tools like less and tail can help load these types of large files quickly since they don’t read the entire file at first.

 Less includes many commands to help navigate through file content and search for specific words or strings, a valuable tool when sifting through a long access log. Tail shows just the last portion of the access log, so users can quickly see the most recently logged events. 

The tail command’s -f option is often used to monitor access logs while running programs log events. The resulting ability to see what is happening in real-time is invaluable for developers, system administrators, and other IT professionals.

Linux access logs

Like web server access logs, Linux operating systems produce various log files that detail events across the whole computer system’s hardware and software. Many users find it worthwhile to monitor at least a few important logs:

These plain text files are found in their individual directories under the /var/log directory. Some applications use an app-specific location for storing log files that may be outside of the default /var/log/ directory. All of the file paths referenced above may differ depending on the OS and configuration.

Analyzing access logs

Log file analysis provides key information needed to understand what is taking place on a server or website. Logging analysis tools can be essential for interpreting large amounts of data that can build up quickly in various log files.

The Webalizer is a basic example of a log analysis tool that examines web server logs like access and usage logs to generate more user-friendly HTML reports. These reports present usable data in a graphical format for better understanding and easier interpretation.

Some hosting configurations for web applications consist of hybrid environments where log files may appear across multiple locations. Monitoring and analyzing individual log files becomes impractical in complex scenarios like these. Customized solutions for collecting, storing, and parsing logs from a centralized location have become increasingly necessary for efficiently using log data.

WordPress logs

By default, WordPress isn’t configured to produce a log file. The wp-config.php file must be edited to enable the WP_DEBUG constant used to put WordPress in debug mode. It’s not generally recommended to leave WP_DEBUG enabled on a live site, as it’s meant for testing and development.

To enable it, locate the WP_DEBUG line in the wp-config.php file and set the value to true.

define( 'WP_DEBUG', true );

Instead of storing debug information in a log file, WordPress can also display error messages on web pages by setting the WP_DEBUG_DISPLAY option to true.

define( 'WP_DEBUG_DISPLAY', true);

To view log files at a later time or store them for future reference, it’s necessary to enable the debug.log file by setting the WP_DEBUG_LOG line to true. 

define( 'WP_DEBUG_LOG', true );

With this option enabled, WordPress stores error messages in a debug.log file  located in the wp-content directory. Users access the log file via SSH, file manager, or FTP, then view the file with a text editor.

Various plugins can also enable the WordPress debug mode and log file. While plugins remove some of the technical barriers to enabling debug options, the result is still somewhat limited. Compared to the more comprehensive data in web server access logs, the debug.log file is limited to error messages, notices, and warnings, specifically from WordPress. The access logs from your website’s host server offer more opportunities for mining data to uncover insights into your website’s traffic and overall statistics.