Kubetap - kubectl plugin to interactively proxy Kubernetes Services with ease
On Thursday I released a new open source project, kubetap, a kubectl plugin to interactively proxy Kubernetes Services. Currently only HTTP (via mitmproxy) is implemented, but other tap types are planned for the future, such as TCP/UDP and gRPC.
This tool was created because I needed to inspect and manipulate traffic destined for microservices deep with a technology stack. Alternatives, such as solo's Squash or telepresence were far too heavy handed for the task at hand. I hope that kubetap will be useful for developers and security auditors alike.
Code repo is here: https://github.com/soluble-ai/kubetap
Project site here: https://soluble-ai.github.io/kubetap/
If you have any issues, please feel free to open a GitHub ticket.
19 comments · 87% upvoted · other discussions
Sorry, if this is a stupid question, but I'm not understanding the use case for something like this over just doing a
port-forwardEriner_ [OP] 5 points · May 2020
Hi, not a dumb question!
kubectl port-forwardbinds a listening port on a remote Pod to your machine's localhost. After aport-forward, you can reach Pods athttp://127.0.0.1:4000, as an example.Kubetap allows you do deploy an intercepting proxy like mitmproxy in front of a Service. Any traffic that is destined for the target Service will instead first pass through the proxy.
normal Request -> Ingress -> Service -> Pod -> (app_container)
kubetap Request -> Ingress -> Service -> Pod -> (kubetap_container) -> (app_container)
otterley 2 points · May 2020
I'm still not sure I understand. Can you describe the ideal use case for this? Who's this for, why would they want it, and why is it better than existing solutions?
Better still if you can persuade people in an introduction in the documentation.
Eriner_ [OP] 1 points · May 2020
Please check the project site - I have a video and cover use cases in the introduction page.
otterley 0 points · May 2020
I did - and I think it could use more explanation. Videos are great, but someone with less time should be able to understand by reading a couple of well written paragraphs.
Eriner_ [OP] 1 points · May 2020
I created a written documentation of the use cases on the front page of the project site. There are pictures, but it is in fact only a few paragraphs :)
If you think something is missing, please let me know precisely what is not in the docs and I'll be happy to add it.
otterley 2 points · May 2020
I was referring to the repo’s README file as opposed to the website. Sorry about the confusion. The project site is pretty good.
Eriner_ [OP] 1 points · May 2020
Should I add something to the README to funnel people to the project site more? I feel like there are already enough links to it, but I want to make sure others don't have the same poor initial experience you did. That said, I also don't want to duplicate the entire doc site into the README.
BattlePope 3 points · May 2020
That's cool! However, I thought nginx ingress (maybe others) actually bypass the service and talk to pod endpoints directly. Does this catch that traffic as well?
Eriner_ [OP] 2 points · May 2020
Ingresses don't bypass Services, in fact Ingresses expose Services: https://kubernetes.io/docs/concepts/services-networking/ingress/#what-is-ingress
BattlePope 2 points · May 2020
Actually, maybe specific to the nginx ingress, it's an implementation detail:
Hence my question! The service is used to discover the endpoints, but not for traffic.
Eriner_ [OP] 1 points · May 2020
I haven't added test cases for using an nginx ingress, and depending on how exactly that is implemented (your ref above), I suspect that a tap would currently not break an environment but also wouldn't capture incoming traffic that traverses the ingress.
The way kubetap works, it resolves a target Pod by evaluating what Deployments match the Selectors field in a Service. The Deployment is then patched to inject the sidecar, and the Service targetPorts are re-routed to the sidecar.
If the above reference means that the nginx ingress does the same as I described above, then kubetap should work. I'll be publishing a blog post on Tuesday covering why I decided to implement kubetap the way I did, if you're interested in that.
If you happen to have an environment with nginx ingress, I'd appreciate hearing on if it works for you or not.
BattlePope 2 points · May 2020
I do use nginx ingress, and I'm curious, so I'll give this a test later.
BattlePope 2 points · May 2020
Nice! I can confirm this works with nginx-ingress. The ingress controller sees the service's endpoint list change to the mitm proxy container's port, and so basically works as normal. You can see two consecutive requests logged here, the first to the normal service port (8080), and the second to the mitm sidecar (kubetap-listen) after kubetap was turned on:
Eriner_ [OP] 1 points · May 2020
Awesome! Thanks for the feedback!