[jira] [Assigned] (SPARK-22713) OOM caused by the memory contention and memory leak in TaskMemoryManager

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22713: Assignee: (was: Apache Spark) > OOM caused by the memory contention and memory leak in

[jira] [Commented] (SPARK-22713) OOM caused by the memory contention and memory leak in TaskMemoryManager

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481475#comment-16481475 ] Apache Spark commented on SPARK-22713: -- User 'eyalfa' has created a pull request for

[jira] [Assigned] (SPARK-22713) OOM caused by the memory contention and memory leak in TaskMemoryManager

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22713: Assignee: Apache Spark > OOM caused by the memory contention and memory leak in TaskMemory

[jira] [Commented] (SPARK-22713) OOM caused by the memory contention and memory leak in TaskMemoryManager

2018-05-18 Thread Eyal Farago (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481468#comment-16481468 ] Eyal Farago commented on SPARK-22713: - [~jerrylead], excellent investigation and desc

[jira] [Resolved] (SPARK-23503) continuous execution should sequence committed epochs

2018-05-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23503. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 20936 [https://g

[jira] [Assigned] (SPARK-23850) We should not redact username|user|url from UI by default

2018-05-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23850: -- Assignee: Marcelo Vanzin > We should not redact username|user|url from UI by default >

[jira] [Resolved] (SPARK-23850) We should not redact username|user|url from UI by default

2018-05-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23850. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 2.

[jira] [Commented] (SPARK-16451) Spark-shell / pyspark should finish gracefully when "SaslException: GSS initiate failed" is hit

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481307#comment-16481307 ] Apache Spark commented on SPARK-16451: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-16451) Spark-shell / pyspark should finish gracefully when "SaslException: GSS initiate failed" is hit

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16451: Assignee: (was: Apache Spark) > Spark-shell / pyspark should finish gracefully when "S

[jira] [Assigned] (SPARK-16451) Spark-shell / pyspark should finish gracefully when "SaslException: GSS initiate failed" is hit

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16451: Assignee: Apache Spark > Spark-shell / pyspark should finish gracefully when "SaslExceptio

[jira] [Assigned] (SPARK-24321) Extract common code from Divide/Remainder to a base trait

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24321: Assignee: (was: Apache Spark) > Extract common code from Divide/Remainder to a base tr

[jira] [Commented] (SPARK-24321) Extract common code from Divide/Remainder to a base trait

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481247#comment-16481247 ] Apache Spark commented on SPARK-24321: -- User 'rednaxelafx' has created a pull reques

[jira] [Assigned] (SPARK-24321) Extract common code from Divide/Remainder to a base trait

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24321: Assignee: Apache Spark > Extract common code from Divide/Remainder to a base trait > -

[jira] [Assigned] (SPARK-18778) Fix the Scala classpath in the spark-shell

2018-05-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-18778: -- Assignee: (was: Marcelo Vanzin) > Fix the Scala classpath in the spark-shell > ---

[jira] [Assigned] (SPARK-18778) Fix the Scala classpath in the spark-shell

2018-05-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-18778: -- Assignee: Marcelo Vanzin > Fix the Scala classpath in the spark-shell > --

[jira] [Created] (SPARK-24321) Extract common code from Divide/Remainder to a base trait

2018-05-18 Thread Kris Mok (JIRA)
Kris Mok created SPARK-24321: Summary: Extract common code from Divide/Remainder to a base trait Key: SPARK-24321 URL: https://issues.apache.org/jira/browse/SPARK-24321 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-17723) "log4j:WARN No appenders could be found for logger" for spark-shell --proxy-user user

2018-05-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17723. Resolution: Cannot Reproduce This should have been fixed by SPARK-21728 (which initializes

[jira] [Assigned] (SPARK-24308) Handle DataReaderFactory to InputPartition renames in left over classes

2018-05-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24308: --- Assignee: Arun Mahadevan > Handle DataReaderFactory to InputPartition renames in left over classes >

[jira] [Resolved] (SPARK-24308) Handle DataReaderFactory to InputPartition renames in left over classes

2018-05-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24308. - Resolution: Fixed Fix Version/s: 2.4.0 > Handle DataReaderFactory to InputPartition renames in lef

[jira] [Assigned] (SPARK-24248) [K8S] Use the Kubernetes cluster as the backing store for the state of pods

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24248: Assignee: Apache Spark > [K8S] Use the Kubernetes cluster as the backing store for the sta

[jira] [Assigned] (SPARK-24248) [K8S] Use the Kubernetes cluster as the backing store for the state of pods

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24248: Assignee: (was: Apache Spark) > [K8S] Use the Kubernetes cluster as the backing store

[jira] [Commented] (SPARK-24248) [K8S] Use the Kubernetes cluster as the backing store for the state of pods

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481217#comment-16481217 ] Apache Spark commented on SPARK-24248: -- User 'mccheah' has created a pull request fo

[jira] [Resolved] (SPARK-23856) Spark jdbc setQueryTimeout option

2018-05-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23856. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.4.0 > Spark jdbc setQueryTi

[jira] [Assigned] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2018-05-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24149: -- Assignee: Marco Gaido > Automatic namespaces discovery in HDFS federation > --

[jira] [Resolved] (SPARK-24149) Automatic namespaces discovery in HDFS federation

2018-05-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24149. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21216 [https:/

[jira] [Resolved] (SPARK-24312) Upgrade to 2.3.3 for Hive Metastore Client 2.3

2018-05-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24312. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.4.0 > Upgrade to 2.3.3 for Hiv

[jira] [Updated] (SPARK-24320) Cannot read file names with spaces

2018-05-18 Thread Zachary Radtka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zachary Radtka updated SPARK-24320: --- Description: I am trying to read from a file on HDFS that has space in the file name, e.g. "

[jira] [Created] (SPARK-24320) Cannot read file names with spaces

2018-05-18 Thread Zachary Radtka (JIRA)
Zachary Radtka created SPARK-24320: -- Summary: Cannot read file names with spaces Key: SPARK-24320 URL: https://issues.apache.org/jira/browse/SPARK-24320 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23850) We should not redact username|user|url from UI by default

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481065#comment-16481065 ] Apache Spark commented on SPARK-23850: -- User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-18 Thread H Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481062#comment-16481062 ] H Lu commented on SPARK-23928: -- OK. I finally got it. I read the code and found classOf[Rand

[jira] [Comment Edited] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-18 Thread H Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481024#comment-16481024 ] H Lu edited comment on SPARK-23928 at 5/18/18 6:43 PM: --- Dear Watche

[jira] [Comment Edited] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-18 Thread H Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481024#comment-16481024 ] H Lu edited comment on SPARK-23928 at 5/18/18 6:40 PM: --- Dear Watche

[jira] [Commented] (SPARK-23928) High-order function: shuffle(x) → array

2018-05-18 Thread H Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481024#comment-16481024 ] H Lu commented on SPARK-23928: -- Dear Watchers ([~mn-mikke]  [~kiszk]), Can someone help me

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2018-05-18 Thread Anthony Cros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481016#comment-16481016 ] Anthony Cros commented on SPARK-14220: -- >> Lack of support for Scala 2.12 is holding

[jira] [Created] (SPARK-24319) run-example can not print usage

2018-05-18 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-24319: Summary: run-example can not print usage Key: SPARK-24319 URL: https://issues.apache.org/jira/browse/SPARK-24319 Project: Spark Issue Type: Bug Com

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2018-05-18 Thread Andy Scott (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480981#comment-16480981 ] Andy Scott commented on SPARK-14220: Lack of support for Scala 2.12 is holding back o

[jira] [Resolved] (SPARK-24159) Enable no-data micro batches for streaming mapGroupswithState

2018-05-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-24159. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21345 [https://git

[jira] [Assigned] (SPARK-24159) Enable no-data micro batches for streaming mapGroupswithState

2018-05-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-24159: Assignee: Tathagata Das > Enable no-data micro batches for streaming mapGroupswithState >

[jira] [Assigned] (SPARK-20538) Dataset.reduce operator should use withNewExecutionId (as foreach or foreachPartition)

2018-05-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-20538: Assignee: Soham Aurangabadkar > Dataset.reduce operator should use withNewExecutionId (as

[jira] [Resolved] (SPARK-20538) Dataset.reduce operator should use withNewExecutionId (as foreach or foreachPartition)

2018-05-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-20538. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21316 [https://git

[jira] [Resolved] (SPARK-24303) Update cloudpickle to v0.4.4

2018-05-18 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-24303. -- Resolution: Fixed Fix Version/s: 2.4.0 > Update cloudpickle to v0.4.4 >

[jira] [Commented] (SPARK-24303) Update cloudpickle to v0.4.4

2018-05-18 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480912#comment-16480912 ] Bryan Cutler commented on SPARK-24303: -- Issue resolved by pull request 21350 https:

[jira] [Assigned] (SPARK-24303) Update cloudpickle to v0.4.4

2018-05-18 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-24303: Assignee: Hyukjin Kwon > Update cloudpickle to v0.4.4 > > >

[jira] [Created] (SPARK-24318) Flaky test: SortShuffleSuite

2018-05-18 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-24318: - Summary: Flaky test: SortShuffleSuite Key: SPARK-24318 URL: https://issues.apache.org/jira/browse/SPARK-24318 Project: Spark Issue Type: Bug Comp

[jira] [Updated] (SPARK-24211) Flaky test: StreamingOuterJoinSuite

2018-05-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24211: -- Description: *windowed left outer join* - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20

[jira] [Commented] (SPARK-24317) Float-point numbers are displayed with different precision in ThriftServer2

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480836#comment-16480836 ] Apache Spark commented on SPARK-24317: -- User 'cxzl25' has created a pull request for

[jira] [Assigned] (SPARK-24317) Float-point numbers are displayed with different precision in ThriftServer2

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24317: Assignee: (was: Apache Spark) > Float-point numbers are displayed with different preci

[jira] [Assigned] (SPARK-24317) Float-point numbers are displayed with different precision in ThriftServer2

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24317: Assignee: Apache Spark > Float-point numbers are displayed with different precision in Thr

[jira] [Created] (SPARK-24317) Float-point numbers are displayed with different precision in ThriftServer2

2018-05-18 Thread dzcxzl (JIRA)
dzcxzl created SPARK-24317: -- Summary: Float-point numbers are displayed with different precision in ThriftServer2 Key: SPARK-24317 URL: https://issues.apache.org/jira/browse/SPARK-24317 Project: Spark

[jira] [Created] (SPARK-24316) Spark sql queries stall for column width more 6k for parquet based table

2018-05-18 Thread Bimalendu Choudhary (JIRA)
Bimalendu Choudhary created SPARK-24316: --- Summary: Spark sql queries stall for column width more 6k for parquet based table Key: SPARK-24316 URL: https://issues.apache.org/jira/browse/SPARK-24316

[jira] [Commented] (SPARK-17557) SQL query on parquet table java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary

2018-05-18 Thread Padma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480723#comment-16480723 ] Padma commented on SPARK-17557: --- I encounter an issue when data resides in Hive as parquet

[jira] [Resolved] (SPARK-23667) Scala version check will fail due to launcher directory doesn't exist

2018-05-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23667. --- Resolution: Not A Problem > Scala version check will fail due to launcher directory doesn't exist > -

[jira] [Commented] (SPARK-19228) inferSchema function processed csv date column as string and "dateFormat" DataSource option is ignored

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480582#comment-16480582 ] Apache Spark commented on SPARK-19228: -- User 'sergey-rubtsov' has created a pull req

[jira] [Commented] (SPARK-24197) add array_sort function

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480514#comment-16480514 ] Apache Spark commented on SPARK-24197: -- User 'mn-mikke' has created a pull request f

[jira] [Updated] (SPARK-19228) inferSchema function processed csv date column as string and "dateFormat" DataSource option is ignored

2018-05-18 Thread Sergey Rubtsov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Rubtsov updated SPARK-19228: --- Description: Current FastDateFormat can't properly parse date and timestamp and does not mee

[jira] [Updated] (SPARK-24313) Collection functions interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Description: Several functions working on collection return incorrect result for complex data types

[jira] [Updated] (SPARK-24313) Collection functions interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Summary: Collection functions interpreted execution doesn't work with complex types (was: array_co

[jira] [Commented] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480482#comment-16480482 ] Kazuaki Ishizaki commented on SPARK-24314: -- I am working for this. > interprete

[jira] [Reopened] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-24314: -- > interpreted element_at or GetMapValue does not work for complex types > -

[jira] [Updated] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-24314: - Summary: interpreted element_at or GetMapValue does not work for complex types (was: int

[jira] [Created] (SPARK-24315) Multiple streaming jobs detected error causing job failure

2018-05-18 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24315: --- Summary: Multiple streaming jobs detected error causing job failure Key: SPARK-24315 URL: https://issues.apache.org/jira/browse/SPARK-24315 Project: Spark Issu

[jira] [Updated] (SPARK-24313) array_contains/array_position interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Description: The functions {{array_contains}} and {{array_position}} return incorrect result for co

[jira] [Updated] (SPARK-24313) array_contains/array_position interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-24313: Summary: array_contains/array_position interpreted execution doesn't work with complex types (was:

[jira] [Resolved] (SPARK-24314) interpreted array_position does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-24314. -- Resolution: Duplicate > interpreted array_position does not work for complex types > --

[jira] [Commented] (SPARK-23614) Union produces incorrect results when caching is used

2018-05-18 Thread Morten Hornbech (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480454#comment-16480454 ] Morten Hornbech commented on SPARK-23614: - In the example provided caching is req

[jira] [Created] (SPARK-24314) interpreted array_position does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24314: Summary: interpreted array_position does not work for complex types Key: SPARK-24314 URL: https://issues.apache.org/jira/browse/SPARK-24314 Project: Spark

[jira] [Commented] (SPARK-23614) Union produces incorrect results when caching is used

2018-05-18 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480438#comment-16480438 ] Yu-Jhe Li commented on SPARK-23614: --- Is this bug happening only when 1) cached datafram

[jira] [Commented] (SPARK-24313) array_contains interpreted execution doesn't work with complex types

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480392#comment-16480392 ] Apache Spark commented on SPARK-24313: -- User 'mgaido91' has created a pull request f

[jira] [Assigned] (SPARK-24313) array_contains interpreted execution doesn't work with complex types

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24313: Assignee: (was: Apache Spark) > array_contains interpreted execution doesn't work with

[jira] [Assigned] (SPARK-24313) array_contains interpreted execution doesn't work with complex types

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24313: Assignee: Apache Spark > array_contains interpreted execution doesn't work with complex ty

[jira] [Created] (SPARK-24313) array_contains interpreted execution doesn't work with complex types

2018-05-18 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-24313: --- Summary: array_contains interpreted execution doesn't work with complex types Key: SPARK-24313 URL: https://issues.apache.org/jira/browse/SPARK-24313 Project: Spark

[jira] [Updated] (SPARK-24302) when using spark persist(),"KryoException:IndexOutOfBoundsException" happens

2018-05-18 Thread yijukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yijukang updated SPARK-24302: - Labels: apache-spark (was: ) > when using spark persist(),"KryoException:IndexOutOfBoundsException" happ

[jira] [Resolved] (SPARK-24277) Code clean up in SQL module: HadoopMapReduceCommitProtocol/FileFormatWriter

2018-05-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24277. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21329 [https://githu

[jira] [Assigned] (SPARK-24277) Code clean up in SQL module: HadoopMapReduceCommitProtocol/FileFormatWriter

2018-05-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24277: --- Assignee: Gengliang Wang > Code clean up in SQL module: HadoopMapReduceCommitProtocol/FileFo