I'm taking the Spark runner one.
Regards
JB
On 11/09/2018 21:15, Ahmet Altay wrote:
Could anyone else help with looking at these issues earlier?
On Tue, Sep 11, 2018 at 12:03 PM, Romain Manni-Bucau
<[email protected] <mailto:[email protected]>> wrote:
Im running this main [1] through this IT [2]. Was working fine since
~1 year but 2.7.0 broke it. Didnt investigate more but can have a
look later this month if it helps.
[1]
https://github.com/Talend/component-runtime/blob/master/component-runtime-beam/src/it/serialization-over-cluster/src/main/java/org/talend/sdk/component/beam/it/clusterserialization/Main.java
<https://github.com/Talend/component-runtime/blob/master/component-runtime-beam/src/it/serialization-over-cluster/src/main/java/org/talend/sdk/component/beam/it/clusterserialization/Main.java>
[2]
https://github.com/Talend/component-runtime/blob/master/component-runtime-beam/src/it/serialization-over-cluster/src/test/java/org/talend/sdk/component/beam/it/SerializationOverClusterIT.java
<https://github.com/Talend/component-runtime/blob/master/component-runtime-beam/src/it/serialization-over-cluster/src/test/java/org/talend/sdk/component/beam/it/SerializationOverClusterIT.java>
Le mar. 11 sept. 2018 20:54, Charles Chen <[email protected]
<mailto:[email protected]>> a écrit :
Romain: can you give more details on the failure you're
encountering, i.e. how you are performing this validation?
On Tue, Sep 11, 2018 at 9:36 AM Jean-Baptiste Onofré
<[email protected] <mailto:[email protected]>> wrote:
Hi,
weird, I didn't have it on Beam samples. Let me try to
reproduce and I
will create the Jira.
Regards
JB
On 11/09/2018 11:44, Romain Manni-Bucau wrote:
> -1, seems spark integration is broken (tested with spark
2.3.1 and 2.2.1):
>
> 18/09/11 11:33:29 WARN TaskSetManager: Lost task 0.0 in
stage 0.0 (TID 0, RMANNIBUCAU, executor 0):
java.lang.ClassCastException: cannot assign instance of
scala.collection.immutable.List$SerializationProxy to
fieldorg.apache.spark.rdd.RDD.org
<http://fieldorg.apache.spark.rdd.RDD.org>
<http://org.apache.spark.rdd.RDD.org
<http://org.apache.spark.rdd.RDD.org>>$apache$spark$rdd$RDD$$dependencies_
of type scala.collection.Seq in instance of
org.apache.spark.rdd.MapPartitionsRDD
> at
java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2233)
>
>
> Also the issue Lukasz identified is important even if
workarounds can be
> put in place so +1 to fix it as well if possible.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau
<https://twitter.com/rmannibucau>> | Blog
> <https://rmannibucau.metawerx.net/
<https://rmannibucau.metawerx.net/>> | Old Blog
> <http://rmannibucau.wordpress.com
<http://rmannibucau.wordpress.com>> | Github
> <https://github.com/rmannibucau
<https://github.com/rmannibucau>> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau
<https://www.linkedin.com/in/rmannibucau>> | Book
>
<https://www.packtpub.com/application-development/java-ee-8-high-performance
<https://www.packtpub.com/application-development/java-ee-8-high-performance>>
>
>
> Le lun. 10 sept. 2018 à 20:48, Lukasz Cwik
<[email protected] <mailto:[email protected]>
> <mailto:[email protected] <mailto:[email protected]>>> a
écrit :
>
> I found an issue where we are no longer packaging the
pom.xml within
> the artifact jars at
META-INF/maven/groupId/artifactId. More details
> in https://issues.apache.org/jira/browse/BEAM-5351
<https://issues.apache.org/jira/browse/BEAM-5351>. I wouldn't
> consider this a blocker but it was an easy fix
> (https://github.com/apache/beam/pull/6358
<https://github.com/apache/beam/pull/6358>) and users may
rely on the
> pom.xml.
>
> Should we recut the release candidate to include this?
>
> On Mon, Sep 10, 2018 at 4:58 AM Jean-Baptiste Onofré
> <[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>> wrote:
>
> +1 (binding)
>
> Tested successfully on Beam Samples.
>
> Thanks !
>
> Regards
> JB
>
> On 07/09/2018 23:56, Charles Chen wrote:
> > Hi everyone,
> >
> > Please review and vote on the release
candidate #1 for the
> version
> > 2.7.0, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please
provide specific
> comments)
> >
> > The complete staging area is available for
your review, which
> includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be
deployed to
> dist.apache.org <http://dist.apache.org>
<http://dist.apache.org>
> > <http://dist.apache.org> [2], which is signed
with the key with
> > fingerprint 45C60AAAD115F560 [3],
> > * all artifacts to be deployed to the Maven
Central
> Repository [4],
> > * source code tag "v2.7.0-RC1" [5],
> > * website pull request listing the release and
publishing the API
> > reference manual [6].
> > * Java artifacts were built with Gradle 4.8
and OpenJDK
> > 1.8.0_181-8u181-b13-1~deb9u1-b13.
> > * Python artifacts are deployed along with the
source release
> to the
> > dist.apache.org <http://dist.apache.org>
<http://dist.apache.org>
> <http://dist.apache.org> [2].
> >
> > The vote will be open for at least 72 hours.
It is adopted by
> majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Charles
> >
> > [1]
> >
>
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12343654
<https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12343654>
> > [2]
https://dist.apache.org/repos/dist/dev/beam/2.7.0
<https://dist.apache.org/repos/dist/dev/beam/2.7.0>
> > [3]
https://dist.apache.org/repos/dist/dev/beam/KEYS
<https://dist.apache.org/repos/dist/dev/beam/KEYS>
> > [4]
>
https://repository.apache.org/content/repositories/orgapachebeam-1046/
<https://repository.apache.org/content/repositories/orgapachebeam-1046/>
> > [5]
https://github.com/apache/beam/tree/v2.7.0-RC1
<https://github.com/apache/beam/tree/v2.7.0-RC1>
> > [6]
https://github.com/apache/beam-site/pull/549
<https://github.com/apache/beam-site/pull/549>
>
> --
> Jean-Baptiste Onofré
> [email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
