How to monitor NGINX logs

In part one of our introduction to NGINX “What is NGINX” , we went over the basic history of NGINX, the difference between Apache and NGINX, and why you would use NGINX over Apache in certain environments and web applications. Today we’ll be diving deeper into NGINX and going over topics such as web server performance, monitoring said performance, how to obtain and archive logs for deeper analysis, and how to even tell which web server you’re running on your environment.

Identifying which server you’re environment is using

There are several different ways of going about how to discover which web server a certain environment is running. If you have SSH access and it’s a Linux server then this is relatively easy as you are free to dig around your server. Now, what if you want to find out information on a certain website, say for instance Adobe.com? Well, there are some web tools to go about this such as Google Chrome Developer Tools, but what if you don’t have a browser handy and just have a command line? We can use a tool called “telnet” to make a TCP connection to the server, specifically port 80, and make an HTTP request. The command would look something like this:

“telnet adobe.com 80”.

You would get output similar to:

“Connected to adobe.com. Escape character is '^]'”

Once connected, make an HTTP request with the following command:

“HEAD / HTTP/1.0”,

Hit Enter twice, and you will find the info you are looking for:

Depending on which distribution your server is using, you can either type “httpd -v” or “nginx -v”. Depending on the output, you should get a version number for either one of those commands, or you would get something similar to “command not found”.

It’s also very possible that your server has both NGINX and Apache installed. Another useful command would be to try “sudo systemctl status nginx” or “sudo systemctl status httpd” which would give you output that shows if NGINX or Apache is running or is stopped.

Even with that info, you still can be running into a situation where your web application or web site is hosted and using Apache for web front facing traffic so you will get a “running” status for Apache, but you will also get a running status for NGINX as well if your server has it installed and using it as a proxy. This is where logs come in.

NGINX logs, what info do they hold, and where can you find them?

Logging is the basis for understanding your application. Knowing how to read logs and parse critical information is fundamental to any system administrator or web engineers success at maintaining their environment. Luckily with NGINX, administrators and engineers have great control over the logging information that is important to your environment and provides key metrics into your web server or web applications performance. Like Apache, NGINX provides valuable insight to both access and error logs to gain a deeper understanding of what’s going on.

Not only does NGINX provide vast information for logging, it allows you to structure the format of logs to either JSON or XML for easier parsing. Located in /var/log directory, these logs will help you understand client usage, traffic patterns, who your clients are, their location of origin, and how many times they’ve access certain parts of your website or application. Access logs are also key to discovering latency issues such as a lag responses, and issues with upstream servers.

Error logs are the other critical piece of information that administrators and engineers will need to capture. Error logs are the first place to find errors produced by application servers like FastCGI, or when configuration files aren’t functioning properly. Parsing through error logs will highlight useful information to debug connections for certain types of clients and be able to pinpoint what the root cause is, such as memory allocation, client IP issues, and server faults. The one thing about NGINX compared to Apache when it comes to error logs is that NGINX’s error logs can’t be formatted which is why Sumo Logic is such an important tool in any administrators environment.

What are some key metrics to monitor for NGINX and how can I fine tune my environment?

Tuning NGINX is something any administrator will need to adopt as applications and environments are becoming more dynamic with their needs and the list of variable items becomes longer for applications to function properly. The amount of time needed for performance tuning and testing can be reduced greatly by automation and monitoring tools. One of the most important types of tuning and testing involves connection tuning for keeping multiple connections open at once (which is one of the main selling points of NGINX).

Automating your tests with tools such as Apache JMeter, Locust, or Gatlin is great to provide standardization in your testing and repeatability clean data metrics. Anytime you change a variable in your testing, it taints the results, so keeping the tests automated and standardized is critical. Start by creating a baseline configuration or template, run it against your environment or service, and slowly change different inputs such as emulated user concurrency to identify weak points in your server configuration. Tune NGINX and repeat your tests until you reach your desired results.

Automated testing tools help you build metrics that you can rely on. Running a test before making any tweaks to NGINX helps provide a baseline and gives you a basis to go off of.

Another popular test involves keeping connections open to clients. This test highlights the number of possible requests allowed to be made over a single connection from clients and the amount of idle time allowed to persist before being timed out. Different NGINX modules and directives can be used to monitor such performance, but keepalive_requests and keepalive_timeout are two of the most popular and useful. These directives allow administrators to adjust the number of concurrent requests that are made over a connection and also the length of idle time that these said connections can stay open.

Conclusion

Just like Apache, NGINX provides administrators with a wealth of information about their server environment including access logs and error logs. Part three of this compilation, we’ll be going over how Sumo Logic can help engineers and administrators parse through the abundance of information provided by these logs, apply that to focus on web server performance, and apply that to key metrics of NGINX web servers.