We do and it includes SparkJobServerDriver. I guess the source was needed before just to build a JobServer Docker image for SparkRunner but, starting from 2.20, we publish them in release process [1]. So we just need to update documentation accordingly [2].
[1] https://issues.apache.org/jira/browse/BEAM-9022 [2] https://issues.apache.org/jira/browse/BEAM-9857 > On 22 Jul 2020, at 13:12, Maximilian Michels <m...@apache.org> wrote: > >> On the SparkRunner page, we advise users to download Beam sources and build >> JobService. So I think it would be better just to add a note there about >> this issue with old branches. > > Why is that? Don't we publish the Spark job server jar? > > -Max > > On 21.07.20 18:20, Alexey Romanenko wrote: >> On the SparkRunner page, we advise users to download Beam sources and build >> JobService. So I think it would be better just to add a note there about >> this issue with old branches. >>> On 20 Jul 2020, at 22:29, Kenneth Knowles <k...@apache.org >>> <mailto:k...@apache.org> <mailto:k...@apache.org <mailto:k...@apache.org>>> >>> wrote: >>> >>> I think it is fine to fix it in branches. I do not see too much value in >>> fixing it except in branches you know you are going to use. >>> >>> The "Downloads" page is for users and only mentioned the voted source >>> releases, maven central, and pypi. There is nothing to do with GitHub or >>> ongoing branches there. I don't think un-published cherrypicks to branches >>> matter to users. Did you mean some other place? >>> >>> Kenn >>> >>> On Mon, Jul 20, 2020 at 9:44 AM Alexey Romanenko <aromanenko....@gmail.com >>> <mailto:aromanenko....@gmail.com> <mailto:aromanenko....@gmail.com >>> <mailto:aromanenko....@gmail.com>>> wrote: >>> >>> Then, would it be ok to fix it in branches (the question is how >>> many branches we should fix?) with additional commit and mention >>> that on “Downloads" page? >>> >>>> On 8 Jul 2020, at 21:24, Kenneth Knowles <k...@apache.org >>>> <mailto:k...@apache.org> >>>> <mailto:k...@apache.org <mailto:k...@apache.org>>> wrote: >>>> >>>> >>>> >>>> On Wed, Jul 8, 2020 at 12:07 PM Kyle Weaver <kcwea...@google.com >>>> <mailto:kcwea...@google.com> >>>> <mailto:kcwea...@google.com <mailto:kcwea...@google.com>>> wrote: >>>> >>>> > To fix on previous release branches, we would need to make >>>> a new release, is it not? Since hashes would change.. >>>> >>>> Would it be alright to patch the release branches on Github >>>> and leave the released source as-is? Github release branches >>>> themselves aren't release artifacts, so I think it should be >>>> okay to patch them without making a new release. >>>> >>>> >>>> Yea. There are tags for the exact hashes that RCs were built >>>> from. The release branch is fine to get new commits, and then if >>>> anyone wants to build a patch release they will get those commits. >>>> >>>> Kenn >>>> >>>> On Wed, Jul 8, 2020 at 11:59 AM Pablo Estrada >>>> <pabl...@google.com <mailto:pabl...@google.com> >>>> <mailto:pabl...@google.com <mailto:pabl...@google.com>>> wrote: >>>> >>>> Ah that's annoying that a dependency would be removed >>>> from maven. I thought that was not meant to happen? This >>>> must be an issue happening for many other projects... >>>> Why is errorprone a dependency anyway? >>>> >>>> To fix on previous release branches, we would need to >>>> make a new release, is it not? Since hashes would change.. >>>> >>>> On Wed, Jul 8, 2020 at 10:21 AM Alexey Romanenko >>>> <aromanenko....@gmail.com <mailto:aromanenko....@gmail.com> >>>> <mailto:aromanenko....@gmail.com >>>> <mailto:aromanenko....@gmail.com>>> wrote: >>>> >>>> Hi Max, >>>> >>>> I’m +1 for back porting as well but that seems quite >>>> complicated since we distribute release source code >>>> from https://archive.apache.org/ >>>> <https://archive.apache.org/> >>>> Perhaps, we should just warn users about this issue >>>> and how to workaround it. >>>> >>>> Any other ideas? >>>> >>>> > On 8 Jul 2020, at 11:46, Maximilian Michels >>>> <m...@apache.org <mailto:m...@apache.org> >>>> <mailto:m...@apache.org <mailto:m...@apache.org>>> wrote: >>>> > >>>> > Hi Alexey, >>>> > >>>> > I also came across this issue when building a >>>> custom Beam version. I applied the same fix >>>> (https://github.com/apache/beam/pull/11527) which you >>>> have mentioned. >>>> > >>>> > It appears that the Maven dependencies changed or >>>> are no longer available which causes the missing >>>> class files. >>>> > >>>> > +1 for backporting the fix to the release branches. >>>> > >>>> > Cheers, >>>> > Max >>>> > >>>> > On 08.07.20 11:36, Alexey Romanenko wrote: >>>> >> Hello, >>>> >> Some days ago I noticed that I can’t build the >>>> project from old release branches . For example, I >>>> wanted to build and run Spark Job Server from >>>> “release-2.20.0” branch and it failed: >>>> >> ./gradlew :runners:spark:job-server:runShadow >>>> —stacktrace >>>> >> * Exception is: >>>> >> org.gradle.api.tasks.TaskExecutionException: >>>> Execution failed for task ':model:pipeline:compileJava’. >>>> >> … >>>> >> Caused by: org.gradle.internal.UncheckedException: >>>> java.lang.ClassNotFoundException: >>>> com.google.errorprone.ErrorProneCompiler$Builder >>>> >> … >>>> >> I experienced the same issue for “release-2.19.0” >>>> and “release-2.21.0” branches, I didn’t check older >>>> branches but seems it’s a global issue for >>>> “net.ltgt.gradle:gradle-errorprone-plugin:0.0.13". >>>> >> This is already known issue and it was fixed for >>>> 2.22.0 [1] a while ago. By applying a fix from [2] on >>>> top of previous branch, for example, “release-2.20.0” >>>> branch I’ve managed to build it. Though, the problem >>>> for old branches (<2.22.0) is still there - it’s not >>>> possible to build them right after checkout without >>>> applying the fix. >>>> >> So, there are two questions: >>>> >> 1. Is anyone aware why the old static version of >>>> gradle-errorprone-plugin fails for the branches that >>>> were successfully built before? >>>> >> 2. Do we have to fix it for release branches >>>> <2.22.0 (either cherry-pick the fix for 2.22.0 or >>>> somehow else if it’s possible)? >>>> >> [1] https://issues.apache.org/jira/browse/BEAM-10263 >>>> >> [2] https://github.com/apache/beam/pull/11527
