lostluck opened a new issue, #25582: URL: https://github.com/apache/beam/issues/25582
### What would you like to happen? Occasionally a Java SDK worker harness is unable to boot up due to extremely long command line arguments in the form of the list files to include on the classpath. https://github.com/apache/beam/blob/master/sdks/java/container/boot.go#L236 This can be difficult for end users to work around since they are generally unaware of dependant jars and similar, depending on pipeline construction. A solution exists by using another jar, known as a pathing jar, which simply refers to the desired contents of the class path. This solution is unfortunately required vs the more updated solution of an @argfile parameter for starting the JVM. However the beam container needs to support back to Java 8, and @argfile wasn't introduced until Java 9. Since this would need to be authored in Go in order to build the pathing jar with the contents of the provisioned manifest. Unfortunately, there doesn't seem to be existing support for this as a Go package. https://pkg.go.dev/search?q=jar+java&m= the tools are largely about reading Jars rather than creating them. Fortunately, the [Jar specification](https://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html) is relatively simple, as it's generally a Zip file with a Meta-INF directory. Authoring zip files in Go is robustly supported in the Go standard library: https://pkg.go.dev/archive/zip. The proposal is to build such a pathing jar in memory from the existing local artifacts, write it out, and then use that as the single class path parameter when invoking java. ------ For reference: Gradle itself has support for building pathing jars. https://github.com/gradle/gradle/pull/10544/files#diff-bda9c25c55281a1f596c7e7892ce79631e74ac6eef32fe06fef664da23759c62R349 https://stackoverflow.com/questions/5434482/how-can-i-create-a-pathing-jar-in-gradle Java natively can create jars... https://docs.oracle.com/javase/7/docs/api/java/util/jar/JarOutputStream.html#JarOutputStream(java.io.OutputStream) Apparently a "pathing jar" requires the files listed to be relative to the location of the jar. That's not too bad. It's a matter of creating the appropriate manifest in the Go boot script however (if not on the sservice side) ### Issue Priority Priority: 2 (default / most feature requests should be filed as P2) ### Issue Components - [ ] Component: Python SDK - [X] Component: Java SDK - [ ] Component: Go SDK - [ ] Component: Typescript SDK - [ ] Component: IO connector - [ ] Component: Beam examples - [ ] Component: Beam playground - [ ] Component: Beam katas - [ ] Component: Website - [ ] Component: Spark Runner - [ ] Component: Flink Runner - [ ] Component: Samza Runner - [ ] Component: Twister2 Runner - [ ] Component: Hazelcast Jet Runner - [ ] Component: Google Cloud Dataflow Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
