Human-friendly Domains with Knative

4 minutes read
k8s • knative • devops

Introduction

Recently, I started playing with Kubernetes and Knative. The trigger was the credits I received to play with Oracle Cloud. I did not know how to use them and on which services (Java is the only Oracle’s product I use :D), so I googled “oracle cloud kubernetes” and 15 minutes later I had a running cluster to play with. Nice!

After installing Knative (k apply -f FTW!) and Gloo as a gateway, I decided to move one of my pet projects to this cluster, just to get the feeling of what it takes to run a few services on it.

BTW, I expected a (better) integration of Oracle Cloud’s Image Registry and their k8s service, but, after all, that was the only thing I had to do manually.

The problem

Once I deployed the services, I was able to send some HTTP requests to $serviceName.$namespace.example.com-like domains.
My project’s domain is obviously not example.com, so I went to https://knative.dev/docs/serving/using-a-custom-domain/ and changed it accordingly.

But I stil didn’t like $serviceName.$namespace.proj.org-style domains for exposed services. I could drop the namespace (so that it becomes $serviceName.proj.org), but my service names did not match the public domains I wanted to expose.

My knowledge if Kubernetes made me think that most of the time there is some annotation to adjust the defaults. Well, not this time.

There were some discussions on how to make custom domain names easier to set:

But none of them are resolved yet.

Another way is to use Virtual Services but this seems to be Istio specific, and I decided to use Gloo. If you’re running in Istio, check this: https://github.com/knative/docs/blob/v0.13.x/docs/serving/samples/knative-routing-go/routing.yaml

It turns out, custom domains with Knative isn’t something you can get out of the box!

We just need another annotation

After looking at Knative configs (and asking help from fellow VMwareans), I started looking at domainTemplate property of config-network ConfigMap:

# domainTemplate specifies the golang text template string to use
# when constructing the Knative service's DNS name. The default
# value is "{{.Name}}.{{.Namespace}}.{{.Domain}}". And those three
# values (Name, Namespace, Domain) are the only variables defined.s
#
# Changing this value might be necessary when the extra levels in
# the domain name generated is problematic for wildcard certificates
# that only support a single level of domain name added to the
# certificate's domain. In those cases you might consider using a value
# of "{{.Name}}-{{.Namespace}}.{{.Domain}}", or removing the Namespace
# entirely from the template. When choosing a new value be thoughtful
# of the potential for conflicts - for example, when users choose to use
# characters such as `-` in their service, or namespace, names.
# {{.Annotations}} can be used for any customization in the go template if needed.
# We strongly recommend keeping namespace part of the template to avoid domain name clashes
# Example '{{.Name}}-{{.Namespace}}.{{ index .Annotations "sub"}}.{{.Domain}}'
# and you have an annotation {"sub":"foo"}, then the generated template would be {Name}-{Namespace}.foo.{Domain}
domainTemplate: "{{.Name}}.{{.Namespace}}.{{.Domain}}"

This one caught my attention:

{{.Annotations}} can be used for any customization in the go template if needed.

“Can be”? Will be! 🤓

After some trial and error, I ended up with:

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-network
  namespace: knative-serving
data:
  domainTemplate: |-
    {{if index .Annotations "org.proj.subdomain" -}}
      {{- index .Annotations "org.proj.subdomain" -}}
    {{else -}}
      {{- .Name}}.{{.Namespace -}}
    {{end -}}
    .{{.Domain}}

(be careful here, every new line/space will end up in the domain name and the config validation will fail)

Here we check if org.proj.subdomain annotation is present and use its value as a sub-domain. If not, we fallback to the standard $serviceName.$namespace.$domain scheme.

Now we can deploy our Knative services as usual but enjoy human-friendly domains:

---
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: app
  annotations:
    # Will be exposed as `api.proj.org` instead of `app.myproj.proj.org`
    org.proj.subdomain: api
spec:
  ...

And it is gateway agnostic and just uses Knative’s configuration, which I find very nice (less custom things to have).

Conclusion

I hope that Knative will add support for such customization soon. Meanwhile, it requires a minor configuration change.

Who knows, maybe I should even contribute it to their default config?

comments powered by Disqus