bamaer commented on PR #8997: URL: https://github.com/apache/hop/pull/8997#issuecomment-6270478743
Thanks @hansva, nice cleanup of lib/core. I checked that the jars plugins now take as `provided` (and the ones excluded from the Spark assembly) are all present in lib/core, and that pty4j 0.14 only uses Kotlin stdlib APIs available in the 1.9.25 we ship. Code reviewed, not built or run locally. **Should fix** - `BeamDataFlowPipelineRunConfiguration.getPipelineOptions()`: an empty field now means 128 MB instead of Beam's Dataflow default of 1 MB (`DataflowRunner.GCS_UPLOAD_BUFFER_SIZE_BYTES_DEFAULT`). The option is passed to the workers too and is the chunk size for every GCS write (file output, BigQuery file loads), so existing Dataflow configs would hold 128 MB per open writer. Suggest only setting it when the field is filled in, recommending 128 MB in the tooltip/docs, and capping the value (`mb * 1024 * 1024` overflows from 2048). Minor: `ui/pom.xml` moves jediterm from 3.57 to 3.56. Intentional? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
