cckellogg commented on a change in pull request #6752:
URL: https://github.com/apache/pulsar/pull/6752#discussion_r411063261
##########
File path:
pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntime.java
##########
@@ -155,6 +158,9 @@
String pythonDependencyRepository,
String pythonExtraDependencyRepository,
String pulsarDockerImageName,
+ String javaFunctionDockerImageName,
+ String pythonFunctionDockerImageName,
+ String goFunctionDockerImageName,
Review comment:
> We allow users to configure image for a specific language function(Eg:
Python function)
I agree it would be nice to have images for specific runtimes.
Are users in charge of building their own function images? Or will the
project be publishing images for each function runtime?
If users are expected to create their own images thing could get tricky
because currently the pulsar-admin tool is required for functions to run in
kubernetes. It's used to download the function binary
(https://github.com/apache/pulsar/blob/master/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntime.java#L802).
This means every image needs java. Ideally, if you are running python or go
functions you should not need a java dependency.
> For the user-defined k8s spec, we should consider how to load the
user-implemented tar package into the pod.
If you are writing custom plugins it usually makes sense to build your own
image and you can dump the plugin in the `pulsar/lib` directory and it will
automatically end up on the class path.
Here are a few possible ways to solve multiple images.
1 - the current way with a flat config
```
javaFunctionDockerImageName
pythonFunctionDockerImageName
goFunctionDockerImageName
```
2 - pass a map of runtime to image
I like a map since it allows for easier modifications in the future.
However, it can be a little more configuration.
```
runtime -> {}
java -> {
image: java-image
}
python -> {
image: python-image
},
go -> {
image: go-image
},
```
3 - implement a KubernetesManifestCustomizer for multiple images and make
that plugin part of the project. The Customizer class is a configuration
`runtimeCustomizerClassName` in the function_worker.yaml. This Customizer can
also take configuration.
Options 1 and 2 require this code to be updated (
This code needs to be updated
(https://github.com/apache/pulsar/blob/master/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntime.java#L973).
Thoughts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
