Customize the Default Log Format in Kong API Gateway

This is a continuation of my article series on Kong API gateway and this one will go though the steps to change the default log format of the Kong API gateway with the desired style.

Role of Nginx

Kong is built on NGINX, and uses Lua to implement its API functionality, whereas the API management module relies completely on native, high‑performance capabilities that are implemented as NGINX Plus modules. So the format of the log lines in Kong is determined by corresponding configuration in the NGINX service.

As you can see in the above default log line, it only contain few basic information. But you might need additional parameters for your use-case. But one important point to remember is including unnecessary information in the logs or defining a bulky log structure will hamper the performance of the API gateway.

There are few options for us to change the default log format with our intended style.

Injecting Nginx directives

Tweaking the Nginx configuration of your Kong instances allows you to update the default configurations. When Kong starts, it builds an Nginx configuration file. You can inject custom Nginx directives to this file directly via your Kong configuration.

a) Define the desired log format using the nginx_http_log_format parameter in the kong.conf;

b) Then point the updated log pattern to the default access log configuration of the kong.conf;

You can read more information on this method from here.

Environment variables

This is my favorite method and one of the reasons which make Kong super easy to use. When loading properties out of a configuration file, Kong will also look for environment variables of the same name. This allows you to fully configure Kong via environment variables, which is very convenient for container-based infrastructures, for example:

To override a setting using an environment variable, declare an environment variable with the name of the setting, prefixed with KONG_ and capitalized.

a) In Docker, you would use environment variables as per the usual Docker configuration;

b) Then inject this configuration to Kong configuration via the below environment variables;

When you define your own log pattern, this documentation will be useful to understand the variables available in NGINX log module.

My Log Format

For my case, I wanted a very simple log format with minimal information required to do any troubleshooting. So what I wanted to see for the successful API requests were,

  • Request source IP
  • Request time
  • Response time
  • API Request information (HTTP method, Context path etc )
  • API Response code
  • User Agent Header (to identify the requestor)

Additionally, for a failed API request by default Kong will print extra information about the error details .

The $time_local parameter in the default log line is the response time for the API request. So for me to obtain the request time, I had to configure the below additional environment variable

If you also need to print the request duration time, you ca

So below are all the environment variables I had to configure for my docker image.

The resultant log pattern will be like,

Additionally if you need to log the request duration, then you can use the variable $request_time when defining the log format. So the resulting log format will be like

Time stamps in the above log lines are in the unix epoch time. So we need to use the epoch convertor to covert the ms time to UTC or required time. https://www.epochconverter.com

Below is a sample converted timestamp from the above log line,

1620205189.300:

  • Wednesday, May 5, 2021 8:59:49.300 AM
  • Wednesday, May 5, 2021 4:59:49.300 PM GMT+08:00

1620205189.259:

  • Wednesday, May 5, 2021 8:59:49.259 AM
  • Wednesday, May 5, 2021 4:59:49.259 PM GMT+08:00

If you subtract these two values, it will notice that it’s giving a value of 0.041 which is the same value of request duration time as in the above log line.

Thanks!

Senior Software Engineer | BSc (Hons) Engineering | CIMA | Autodidact | Knowledge-Seeker