Andy Muir created KAFKA-8629:
Summary: Kafka Streams Apps to support small native images through
GraalVM
Key: KAFKA-8629
URL: https://issues.apache.org/jira/browse/KAFKA-8629
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 2.3.0
Environment: OSX
Linux on Docker
Reporter: Andy Muir
I'm investigating using [GraalVM|http://example.com/] to help with reducing
docker image size and required resources for a simple Kafka Streams
microservice. To this end, I'm looking at running a microservice which:
1) consumes from a Kafka topic (XML)
2) Transforms into JSON
3) Produces to a new Kafka topic.
The Kafka Streams app running in the JVM works fine.
When I attempt to build it to a GraalVM native image (binary executable which
does not require the JVM, hence smaller image size and less resources), I
encountered a few
[incompatibilities|https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md]
with the source code in Kafka.
I've implemented a workaround for each of these in a fork (link to follow) to
help establish if it is feasible. I don't intend (at this stage) for the
changes to be applied to the broker - I'm only after Kafka Streams for now. I'm
not sure whether it'd be a good idea for the broker itself to run as a native
image!
There were 2 issues getting the native image with kafka streams:
1) Some Reflection use cases using MethodHandle
2) Anything JMX
To work around these issues, I have:
1) Replaced use of MethodHandle with alternatives
2) Commented out the JMX code in a few places
While the first may be sustainable, I'd expect that the 2nd option should be
put behind a configuration switch to allow the existing code to be used by
default and turning off JMX if configured.
*I haven't created a PR for now, as I'd like feedback to decide if it is going
to be feasible to carry this forwards.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
