Gregory J. Suarez · Engineering notebook
I learn by building things and taking them apart, then I write down what I find: Spark internals, Kafka and Flink pipelines, API gateways and, lately, language models. Some of it becomes videos and hands-on labs, too.
Latest post
Interactively, Qwen Code never gave me trouble. The first time I ran it as a one-shot -p job, the answer was slow and a read-only df was refused. I followed it through the source to two bugs, filed both, and wrote a fix for the one that comes down to a single regular expression.
Read the post →I wanted to see how Kong handles traffic that has nothing to do with AI, so I followed Microsoft's HDInsight article on connecting a local Jupyter to Spark through Livy, and put Kong where the HDInsight gateway sits.
In my last Kong post I broke requests one at a time and read the evidence by hand. This time I asked what the gateway can tell me over hours and across several agents at once, and put its metrics next to the model server and the GPUs.
I put Kong Gateway and its AI Proxy plugin in front of my local LLM, measured what the extra hop costs, then broke requests on purpose to see where the evidence of each failure ends up.
A talk from IBM, Confluent and Google Cloud argued that AI agents fail for lack of fresh, refined context, not for lack of a better model. I rebuilt the idea with Kafka, Flink, MCP and a local LLM, then asked one question four ways.
In my last post I left one problem unexplained: ordinary Python UDFs that never finished on cached or Parquet data. This weekend I took a closer look. Here is how I narrowed it down, what was going on, and where the fix stands.
1:19:12 The Personal Learning Series
A narrated, fully animated video that follows one sentence through a language model: tokens, vectors, attention, transformer layers, sampling, training, and what it all means.
The books I study from and recommend. Every one of them has earned its place.
Also in the library: DeepLearning.AI, Andrew Ng's AI courses, and Wolfram Mathematica, with tips on the Personal plan.
At Microsoft, I spent many years supporting open-source distributed systems like Spark, Hive, HBase and Kafka in the cloud and on-premises. I connect customers with the engineers who build the product, and I see problems through until they're solved. I also learn by building: lately that means taking language models apart and turning what I find into videos, labs and notes.
More about me →