Not interested in explanations? Fast travel to the tl;dr 🚀

So, you’ve got a Kubernetes cluster and ingress-nginx as ingress controller.

You may have used configuration snippets to include ingress-specific NGINX configuration, such as static headers:

nginx.ingress.kubernetes.io/configuration-snippet: |
  proxy_set_header foo "bar";

What if we wanted to conditionally add headers, e.g. only if not set?

We might try a Lua block:

nginx.ingress.kubernetes.io/configuration-snippet: |
  rewrite_by_lua_block {
    headers = ngx.req.get_headers()
    if not headers["foo"] then
      ngx.req.set_header["foo"] = "bar"
    end
  }

Only to find out in ingress-nginx logs that configuration reload fails:

[emerg] 31379#31379: is duplicate in /tmp/nginx-cfg011294657:1037
nginx: [emerg] is duplicate in /tmp/nginx-cfg011294657:1037
nginx: configuration file /tmp/nginx-cfg011294657 test failed

Indeed, the default NGINX configuration template already has a rewrite_by_lua_block directive in the same location:

rewrite_by_lua_block {
  lua_ingress.rewrite({{ locationConfigForLua $location $all }})
  balancer.rewrite()
  plugins.run()
}

At this point, we might elect one of the following workarounds:

  • Use a custom template and get rid of the existing rewrite_by_lua_block directive. However, we would lose default configuration injected via lua_ingress.rewrite(). We’d also miss configuration injected via Lua plugins.
  • Write a Lua plugin, using a ngx.var.host filter to determine which headers to add in rewrite() (akin to the hello_world plugin example). However, this separates an ingress definition from its headers, which is inelegant and poorly maintainable.

If there was a way to reference a custom annotation, we could use it however we please:

rewrite_by_lua_block {
  lua_ingress.rewrite({{ locationConfigForLua $location $all }})
  balancer.rewrite()
  plugins.run()

  -- Custom headers
  {{ insert "my-annotations/custom-headers" }}
}

Unfortunately, regular ingress-nginx annotations are retrieved in Go code and referenced from the NGINX template via dedicated objects and attributes, e.g. for a configuration snippet:

{{/* Add any additional configuration defined */}}
{{ $location.ConfigurationSnippet }}

Customizing and recompiling ingress-nginx just for an annotation would be a tad overkill.

Luckily, there is a function to access raw ingress information, including its annotations. Actually, it is already used to initialize $ing at the top of the location directive:

{{ $ing := (getIngressInformation $location.Ingress $server.Hostname $location.Path) }}

tl;dr

From an NGINX template, we can access custom annotations via $ing:

rewrite_by_lua_block {
  lua_ingress.rewrite({{ locationConfigForLua $location $all }})
  balancer.rewrite()
  plugins.run()

  -- Custom headers
  {{ if index $ing.Annotations "my-annotations/custom-headers" }}
  {{ index $ing.Annotations "my-annotations/custom-headers" }}
  {{ end }}
}

It is then possible to insert conditional headers directly in an ingress definition, just like static headers positioned via a configuration snippet:

my-annotations/custom-headers: |
  headers = ngx.req.get_headers()
  if not headers["foo"] then
    ngx.req.set_header["foo"] = "bar"
  end

Next Post