Full NGINX Logging Example

In a previous post I talked about how to setup NGINX to log both the request and response headers and bodies to help with debugging. It’s a lot easier to set up a HTTP reverse proxy than sort out all the private keys when trying to capture HTTPS with wireshark.

I needed to use that again recently and it took me a little while to remember exactly how to put it all together again, so this is just a really short follow up post with a full minimal example of the nginx.conf file needed.

worker_processes 1;
load_module modules/ndk_http_module.so;
load_module modules/ngx_http_lua_module.so;
pcre_jit on;

events {
  worker_connections 1024;
}

error_log /dev/stderr;

http {
  log_format log_req_resp '$remote_addr - $remote_user [$time_local] '
  '"$request" $status $body_bytes_sent '
  '"$http_referer" "$http_user_agent" $request_time req_body:"$request_body" resp_body:"$resp_body" '
  'req_headers:"$req_header" resp_headers:"$resp_header"';

  server {
    listen 80;
    access_log /dev/stdout log_req_resp;

    root /var/www/html;

    lua_need_request_body on;

    set $resp_body "";
    body_filter_by_lua '
      local resp_body = string.sub(ngx.arg[1], 1, 1000)
      ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
      if ngx.arg[2] then
        ngx.var.resp_body = ngx.ctx.buffered
      end
    ';

    set $req_header "";
    set $resp_header "";
    header_filter_by_lua '
      local h = ngx.req.get_headers()
      for k, v in pairs(h) do
        if (type(v) == "table") then
          ngx.var.req_header = ngx.var.req_header .. k.."="..table.concat(v,",").." "
        else
          ngx.var.req_header = ngx.var.req_header .. k.."="..v.." "
        end
      end
      local rh = ngx.resp.get_headers()
      for k, v in pairs(rh) do
        ngx.var.resp_header = ngx.var.resp_header .. k.."="..v.." "
      end
      ';

  }
}

It also has a small improvement to allow for duplicate HTTP headers in the request (which is in spec). It will now concatenate the values in to a comma separated list.

This is intended to be used with the following Docker container (the default nginx container doe not have the lua module installed)

FROM debian:latest

RUN apt-get update && apt-get install -y libnginx-mod-http-lua libnginx-mod-http-ndk

CMD ["nginx", "-g", "daemon off;"]

Run as follows

docker run --rm -v `pwd`/nginx.conf:/etc/nginx/nginx.conf:ro -p 80:80 custom-nginx

2 thoughts on “Full NGINX Logging Example”

  1. With this configuration I’ve this error:

    [error] 6#6: *19 failed to run header_filter_by_lua*: header_filter_by_lua:12: attempt to concatenate local ‘v’ (a table value)

    1. duplicate the if block from the first for loop in the second, (but this implies your back end is sending duplicates HTTP Response headers)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.