Re: [VOTE] Spark 2.3.0 (RC1)

So looking at http://pgp.mit.edu/pks/lookup?op=vindex&search= 0xA1CEDBA8AD0C022A it seems like Sameer's key isn't in the Apache web of trust yet. This shouldn't block RC process but before we publish it's important to get the key in the Apache web of trust. On Tue, Jan 16, 2018 at 3:00 PM, Sameer

Re: [VOTE] Spark 2.3.0 (RC1)

Yes, I'll cut an RC2 as soon as the remaining blockers are resolved. In the meantime, please continue to report any other issues here. Here's a quick update on progress towards the next RC: - SPARK-22908 (KafkaContiniousSourceSuite) has been rev

Re: Distinct on Map data type -- SPARK-19893

There is a JIRA for making Map types orderable : https://issues.apache.org/jira/browse/SPARK-18134 Given that this is a non-trivial change, it will take time. On Sat, Jan 13, 2018 at 9:50 PM, ckhari4u wrote: > Wan, Thanks a lot,! I see the issue now. > > Do we have any JIRA's open for the future

Re: [VOTE] Spark 2.3.0 (RC1)

Is there going to be another RC ? With KafkaContinuousSourceSuite hanging, it is hard to get the rest of the tests going. Cheers On Sat, Jan 13, 2018 at 7:29 AM, Sean Owen wrote: > The signatures and licenses look OK. Except for the missing k8s package, > the contents look OK. Tests look prett

Re: [build system] currently experiencing git timeouts when building

update: we're seeing about 3% of builds timing out, but today is (so far) a bad one: i've reached out to github about this and am waiting to hear back. $ get_timeouts.py 10 Timeouts by project: 17spark-branch-2.3-lint 10spark-branch-2.3-test-maven-hadoop-2.6 7spark-branch-2.3-comp

Re: [build system] yet another power outage at our colo

ok, we're back up and ready to build. sorry for the inconvenience. On Tue, Jan 16, 2018 at 9:59 AM, shane knapp wrote: > all non-UPS machines (read: all jenkins workers) temporarily lost power a > few minutes ago, and i will need to reconnect them to the master. > > this means no builds for ~2

[build system] yet another power outage at our colo

all non-UPS machines (read: all jenkins workers) temporarily lost power a few minutes ago, and i will need to reconnect them to the master. this means no builds for ~20 mins. i will also be installing a plugin for the spark-on-k8s builds ( https://wiki.jenkins.io/display/JENKINS/Parameterized+Tr

Re: Whole-stage codegen and SparkPlan.newPredicate

Thanks for looking into it, Kazuaki! Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski Mastering Spark SQL https://bit.ly/mastering-spark-sql Spark Structured Streaming https://bit.ly/spark-structured-streaming Mastering Kafka Streams https://bit.ly/mastering-kafka-streams Follow me

How to resolve UnresolvedRelations (to explore FindDataSourceTable)?

Hi, I've been exploring Spark Analyzer's FindDataSourceTable rule and found the following example hard to explain why one of two UnresolvedRelations has not been resolved. Could you help me with the places marked as FIXME? scala> spark.version res0: String = 2.4.0-SNAPSHOT // Create tables // F