Spark 4.2 turns on Arrow for ordinary Python UDFs
Spark 4.2 changes how ordinary Python functions trade data with the JVM. The option has been there since Spark 3.4. Here is what it does, how it got here, and what I measured.
Topic
Apache Arrow
1 post.