Topic
3 posts.
I wanted to see how Kong handles traffic that has nothing to do with AI, so I followed Microsoft's HDInsight article on connecting a local Jupyter to Spark through Livy, and put Kong where the HDInsight gateway sits.
In my last Kong post I broke requests one at a time and read the evidence by hand. This time I asked what the gateway can tell me over hours and across several agents at once, and put its metrics next to the model server and the GPUs.
I put Kong Gateway and its AI Proxy plugin in front of my local LLM, measured what the extra hop costs, then broke requests on purpose to see where the evidence of each failure ends up.