Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

2019-04-24 Thread Hyukjin Kwon
Can anyone take a look for this one? OPEN status JIRAs are being rapidly increased (from around 2400 to 2600) 2019년 4월 19일 (금) 오후 8:05, Hyukjin Kwon 님이 작성: > Hi all, > > Looks 'spark/dev/github_jira_sync.py' is not running correctly somewhere. > Usually the JIRA's status should be updated to "IN

Re: queryable state & streaming

2019-04-24 Thread Stavros Kontopoulos
Michael, I have listed used cases above should we proceed with a design doc? Best, Stavros Στις Δευ, 18 Μαρ 2019, 12:21 μ.μ. ο χρήστης Stavros Kontopoulos < stavros.kontopou...@lightbend.com> έγραψε: > Not really, if we agree that we want this, I can put together a design > document and take it

Re: DataFrameWriter does not adjust spark.sql.session.timeZone offset while writing orc files

2019-04-24 Thread Wenchen Fan
Did you re-create your df when you update the timezone conf? On Wed, Apr 24, 2019 at 9:18 PM Shubham Chaurasia wrote: > Writing: > scala> df.write.orc("") > > For looking into contents, I used orc-tools-X.Y.Z-uber.jar ( > https://orc.apache.org/docs/java-tools.html) > > On Wed, Apr 24, 2019 at

Re: DataFrameWriter does not adjust spark.sql.session.timeZone offset while writing orc files

2019-04-24 Thread Wenchen Fan
How did you read/write the timestamp value from/to ORC file? On Wed, Apr 24, 2019 at 6:30 PM Shubham Chaurasia wrote: > Hi All, > > Consider the following(spark v2.4.0): > > Basically I change values of `spark.sql.session.timeZone` and perform an > orc write. Here are 3 samples:- > > 1) >

DataFrameWriter does not adjust spark.sql.session.timeZone offset while writing orc files

2019-04-24 Thread Shubham Chaurasia
Hi All, Consider the following(spark v2.4.0): Basically I change values of `spark.sql.session.timeZone` and perform an orc write. Here are 3 samples:- 1) scala> spark.conf.set("spark.sql.session.timeZone", "Asia/Kolkata") scala> val df = sc.parallelize(Seq("2019-04-23

Re:Unsubscribe

2019-04-24 Thread 刘杰
Unsubscribe