Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Thank you so much guys .. it indeed took a long time and it was pretty tough this time :-). It was all possible because of your guys' support. I sincerely appreciate it . 2021년 3월 4일 (목) 오전 2:26, Dongjoon Hyun 님이 작성: > It took a long time. Thank you, Hyukjin and all! > > Bests, > Dongjoon. > >

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

It took a long time. Thank you, Hyukjin and all! Bests, Dongjoon. On Wed, Mar 3, 2021 at 3:23 AM Gabor Somogyi wrote: > Good to hear and great work Hyukjin!  > > On Wed, 3 Mar 2021, 11:15 Jungtaek Lim, > wrote: > >> Thanks Hyukjin for driving the huge release, and thanks everyone for >>

RE: Spark Version 3.0.1 Gui Display Query

Hi Ranju! The UI is built up from events. This is why history server able to show the state of the a finished app as those events are replayed to build a state, for details you can check web UI page and the following section too <

Yaml for google spark kubernetes configmap

Hi Has anyone used kubernetes with spark for configmap. My spark job is not able to find configmap. Can someone pls share the yaml if u have used configmap for google k8s Thanks Rajat

Re: [Spark SQL, intermediate+] possible bug or weird behavior of insertInto

Yep this is the behavior for Insert Into, using the other write apis does schema matching I believe. > On Mar 3, 2021, at 8:29 AM, Sean Owen wrote: > > I don't have any good answer here, but, I seem to recall that this is because > of SQL semantics, which follows column ordering not naming

Re: [Spark SQL, intermediate+] possible bug or weird behavior of insertInto

I don't have any good answer here, but, I seem to recall that this is because of SQL semantics, which follows column ordering not naming when performing operations like this. It may well be as intended. On Tue, Mar 2, 2021 at 6:10 AM Oldrich Vlasic < oldrich.vla...@datasentics.com> wrote: > Hi,

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Great work, Hyukjin ! Bests, Angers Wenchen Fan 于2021年3月3日周三 下午5:02写道： > Great work and congrats! > > On Wed, Mar 3, 2021 at 3:51 PM Kent Yao wrote: > >> Congrats, all! >> >> Bests, >> *Kent Yao * >> @ Data Science Center, Hangzhou Research Institute, NetEase Corp. >> *a spark enthusiast* >>

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Good to hear and great work Hyukjin!  On Wed, 3 Mar 2021, 11:15 Jungtaek Lim, wrote: > Thanks Hyukjin for driving the huge release, and thanks everyone for > contributing the release! > > On Wed, Mar 3, 2021 at 6:54 PM angers zhu wrote: > >> Great work, Hyukjin ! >> >> Bests, >> Angers >> >>

Re: Spark job crashing - Spark Structured Streaming with Kafka

Hi Sachit, Also ensure that you are using the correct Jar files for Spark Structured streaming with Kafka. PLease see my mail on it. HTH LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Spark job crashing - Spark Structured Streaming with Kafka

This looks to be coming from k8s, as WatchConnectionManager looks to be a part of kubernetes-client. If that is intermittent, you may want to check the health of k8s API server or network issues as well. Another bet would be trying out 3.1.1 as Spark announces GA on supporting k8s from this

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Thanks Hyukjin for driving the huge release, and thanks everyone for contributing the release! On Wed, Mar 3, 2021 at 6:54 PM angers zhu wrote: > Great work, Hyukjin ! > > Bests, > Angers > > Wenchen Fan 于2021年3月3日周三 下午5:02写道： > >> Great work and congrats! >> >> On Wed, Mar 3, 2021 at 3:51 PM

Spark 3.1.1 Preliminary results (mainly to do with Spark Structured Streaming)

Hi all, We upgraded a cluster of three nodes previously using Spark *3.0.1 t*o the new release of Spark *3.1.1. *The cluster is using RHES 7.6 with three nodes using yarn etc We tested a job that creates random data and writes to Hive 3.0. This worked fine as before. In version 3.0.1 we could

Re: [ANNOUNCE] Announcing Apache Spark 3.1.1

Great work and congrats! On Wed, Mar 3, 2021 at 3:51 PM Kent Yao wrote: > Congrats, all! > > Bests, > *Kent Yao * > @ Data Science Center, Hangzhou Research Institute, NetEase Corp. > *a spark enthusiast* > *kyuubi is a unified multi-tenant JDBC > interface