Logging request & response body and headers with nginx

I’ve been working a problem to do with oAuth token refresh with the Amazon Alexa team recently and one of the things they have asked for is a log of the entire token exchange stage.

Normally I’d do this with something like Wireshark but as the server is running on a Amazon EC2 instance I didn’t have easy access to somewhere to tap the network so I decided to look for another way.

The actual oAuth code is all in NodeJS + Express but the whole thing is fronted by nginx. You can get nginx to log the incoming request body relatively simply, there is a $request_body variable that can be included in the logs, but there is no equivalent $resp_body.

To solve this I turned to Google and it turned up this answer on Server Fault which introduced me to the embedded lua engine in nginx. I’ve been playing with lua for some things at work recently so I’ve managed to get my head round the basics.

The important bit of the answer is:

lua_need_request_body on;

set $resp_body "";
body_filter_by_lua '
  local resp_body = string.sub(ngx.arg[1], 1, 1000)
  ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
  if ngx.arg[2] then
     ngx.var.resp_body = ngx.ctx.buffered
  end
';

I also wanted the request and response headers logging so a little bit more lua got me those as well:

set $req_header "";
  set $resp_header "";
  header_filter_by_lua ' 
  local h = ngx.req.get_headers()
  for k, v in pairs(h) do
      ngx.var.req_header = ngx.var.req_header .. k.."="..v.." "
  end
  local rh = ngx.resp.get_headers()
  for k, v in pairs(rh) do
      ngx.var.resp_header = ngx.var.resp_header .. k.."="..v.." "
  end
';

This combined with a custom log format string gets me everything I need.

log_format log_req_resp '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" $request_time req_header:"$req_header" req_body:"$request_body" resp_header:"$resp_header" resp_body:"$resp_body"';

2 thoughts on “Logging request & response body and headers with nginx”

  1. Hello,
    I’ve tried use your solution in nginx 1.14 (docker image) and I got the error like “unknown directive lua_need_request_body”

    What version of nginx are you using?

    Thanks.

    1. You need to make sure the lua module is installed and enabled in the nginx config file.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>