[jira] [Created] (SPARK-26726) The amount of memory used by the broadcast variable is not synchronized to the UI display

2019-01-25 Thread hantiantian (JIRA)
hantiantian created SPARK-26726: --- Summary: The amount of memory used by the broadcast variable is not synchronized to the UI display Key: SPARK-26726 URL: https://issues.apache.org/jira/browse/SPARK-26726

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752044#comment-16752044 ] Sujith commented on SPARK-9: @[~yuvaldeg]  May i know where i can find PR related to

[jira] [Comment Edited] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752044#comment-16752044 ] Sujith edited comment on SPARK-9 at 1/25/19 8:53 AM: - @[~yuv

[jira] [Updated] (SPARK-26726) The amount of memory used by the broadcast variable is not synchronized to the UI display

2019-01-25 Thread hantiantian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hantiantian updated SPARK-26726: Description: The amount of memory used by the broadcast variable is not synchronized to the UI di

[jira] [Updated] (SPARK-26726) The amount of memory used by the broadcast variable is not synchronized to the UI display

2019-01-25 Thread hantiantian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hantiantian updated SPARK-26726: Description: The amount of memory used by the broadcast variable is not synchronized to the UI di

[jira] [Assigned] (SPARK-26720) Remove unused methods from DateTimeUtils

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26720: Assignee: Maxim Gekk > Remove unused methods from DateTimeUtils > ---

[jira] [Resolved] (SPARK-26720) Remove unused methods from DateTimeUtils

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26720. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23643 [https://gi

[jira] [Assigned] (SPARK-26726) The amount of memory used by the broadcast variable is not synchronized to the UI display

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26726: Assignee: Apache Spark > The amount of memory used by the broadcast variable is not sync

[jira] [Assigned] (SPARK-26726) The amount of memory used by the broadcast variable is not synchronized to the UI display

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26726: Assignee: (was: Apache Spark) > The amount of memory used by the broadcast variable

[jira] [Updated] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Srinivas Yarra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srinivas Yarra updated SPARK-26727: --- Description: We experienced that sometimes the Hive query "CREATE OR REPLACE VIEW AS SELECT

[jira] [Updated] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26727: - Description: We experienced that sometimes the Hive query "CREATE OR REPLACE VIEW AS SELECT FR

[jira] [Updated] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26727: - Component/s: (was: Spark Core) SQL > CREATE OR REPLACE VIEW query fails wit

[jira] [Resolved] (SPARK-26724) Non negative coefficients for LinearRegression

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26724. -- Resolution: Invalid questions should better go to mailing list. You could have a better answer

[jira] [Created] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Srinivas Yarra (JIRA)
Srinivas Yarra created SPARK-26727: -- Summary: CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException Key: SPARK-26727 URL: https://issues.apache.org/jira/browse/SPARK-26727 Project: Spar

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752068#comment-16752068 ] Hyukjin Kwon commented on SPARK-26727: -- Does this consistently happen at 11 executi

[jira] [Created] (SPARK-26728) Make rdd.unpersist and broadcast.unpersist blocking configurable

2019-01-25 Thread liupengcheng (JIRA)
liupengcheng created SPARK-26728: Summary: Make rdd.unpersist and broadcast.unpersist blocking configurable Key: SPARK-26728 URL: https://issues.apache.org/jira/browse/SPARK-26728 Project: Spark

[jira] [Commented] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752102#comment-16752102 ] Hyukjin Kwon commented on SPARK-26711: -- Oh, right. Sounds a good lead to follow. We

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Ramgopal N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752104#comment-16752104 ] Ramgopal N commented on SPARK-26727: Not exactly after 11 executions always.It is ra

[jira] [Commented] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752103#comment-16752103 ] Hyukjin Kwon commented on SPARK-26711: -- Just open a PR that replace one line after

[jira] [Updated] (SPARK-26728) Make rdd.unpersist blocking configurable

2019-01-25 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26728: - Summary: Make rdd.unpersist blocking configurable (was: Make rdd.unpersist and broadcast.unpers

[jira] [Assigned] (SPARK-26728) Make rdd.unpersist blocking configurable

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26728: Assignee: Apache Spark > Make rdd.unpersist blocking configurable > -

[jira] [Assigned] (SPARK-26728) Make rdd.unpersist blocking configurable

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26728: Assignee: (was: Apache Spark) > Make rdd.unpersist blocking configurable > --

[jira] [Created] (SPARK-26729) Spark on Kubernetes tooling hardcodes default image names

2019-01-25 Thread Rob Vesse (JIRA)
Rob Vesse created SPARK-26729: - Summary: Spark on Kubernetes tooling hardcodes default image names Key: SPARK-26729 URL: https://issues.apache.org/jira/browse/SPARK-26729 Project: Spark Issue Typ

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Laszlo Rigo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752203#comment-16752203 ] Laszlo Rigo commented on SPARK-26727: - Hi, It can be reproduced with a scala snippe

[jira] [Comment Edited] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Laszlo Rigo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752203#comment-16752203 ] Laszlo Rigo edited comment on SPARK-26727 at 1/25/19 12:09 PM: ---

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752277#comment-16752277 ] Liang-Chi Hsieh commented on SPARK-26727: - I can't reproduce it even continuing

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Peter Rudenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752282#comment-16752282 ] Peter Rudenko commented on SPARK-9: --- [~S71955] - there's no yet PR to include

[jira] [Updated] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Peter Rudenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Rudenko updated SPARK-9: -- Affects Version/s: 3.0.0 2.4.0 > SPIP: RDMA Accelerated Shuffle Engine

[jira] [Created] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread wuyi (JIRA)
wuyi created SPARK-26730: Summary: Strip redundant AssertNotNull expression for ExpressionEncoder's serializer Key: SPARK-26730 URL: https://issues.apache.org/jira/browse/SPARK-26730 Project: Spark

[jira] [Assigned] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26730: Assignee: Apache Spark > Strip redundant AssertNotNull expression for ExpressionEncoder's

[jira] [Assigned] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26730: Assignee: (was: Apache Spark) > Strip redundant AssertNotNull expression for Expressi

[jira] [Commented] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-25 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752304#comment-16752304 ] Bruce Robbins commented on SPARK-26711: --- [~hyukjin.kwon] Ok, that worked. I had in

[jira] [Resolved] (SPARK-26710) ImageSchemaSuite has some errors when running it in local laptop

2019-01-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26710. --- Resolution: Not A Problem > ImageSchemaSuite has some errors when running it in local laptop > -

[jira] [Updated] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-26730: - Description: For types like Product, we've already add AssertNotNull when we construct serializer(pls see the c

[jira] [Updated] (SPARK-26730) Strip redundant AssertNotNull expression for ExpressionEncoder's serializer

2019-01-25 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-26730: - Description: For types like Product, we've already add AssertNotNull when we construct serializer(pls see the c

[jira] [Reopened] (SPARK-26724) Non negative coefficients for LinearRegression

2019-01-25 Thread Alex Chang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Chang reopened SPARK-26724: Please let me know where the "mailing list" is. By the way, if this place is not a good place to ask

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Ryne Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752353#comment-16752353 ] Ryne Yang commented on SPARK-26718: --- [~kabhwan] cool, I will make a PR soon. thank you

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752356#comment-16752356 ] Thomas Graves commented on SPARK-9: --- This is interesting, a few questions * I

[jira] [Updated] (SPARK-26427) Upgrade Apache ORC to 1.5.4

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26427: -- Fix Version/s: 2.4.1 > Upgrade Apache ORC to 1.5.4 > --- > >

[jira] [Updated] (SPARK-23458) Flaky test: OrcQuerySuite

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23458: -- Fix Version/s: 2.4.1 > Flaky test: OrcQuerySuite > -- > >

[jira] [Commented] (SPARK-26410) Support per Pandas UDF configuration

2019-01-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752467#comment-16752467 ] Xiangrui Meng commented on SPARK-26410: --- There are several possible solutions to t

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

2019-01-25 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752462#comment-16752462 ] Xiangrui Meng commented on SPARK-24579: --- [~bryanc] If we limit the scope to model

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752451#comment-16752451 ] Yuval Degani commented on SPARK-9: -- Great questions, [~tgraves] * SparkRDMA (s

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Ryne Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752420#comment-16752420 ] Ryne Yang commented on SPARK-26718: --- [~kabhwan], [~gsomogyi] PR submitted : https://g

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752478#comment-16752478 ] Apache Spark commented on SPARK-26718: -- User 'linehrr' has created a pull request f

[jira] [Assigned] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26718: Assignee: (was: Apache Spark) > structured streaming fetched wrong current offset fro

[jira] [Assigned] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26718: Assignee: Apache Spark > structured streaming fetched wrong current offset from kafka > -

[jira] [Commented] (SPARK-26724) Non negative coefficients for LinearRegression

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752499#comment-16752499 ] Hyukjin Kwon commented on SPARK-26724: -- Take a look at https://spark.apache.org/com

[jira] [Resolved] (SPARK-26724) Non negative coefficients for LinearRegression

2019-01-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26724. -- Resolution: Invalid > Non negative coefficients for LinearRegression > ---

[jira] [Updated] (SPARK-25635) Support selective direct encoding in native ORC write

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25635: -- Fix Version/s: 2.4.1 > Support selective direct encoding in native ORC write > ---

[jira] [Assigned] (SPARK-26694) Console progress bar not showing in 3.0

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26694: -- Assignee: Ankur Gupta > Console progress bar not showing in 3.0 > ---

[jira] [Resolved] (SPARK-26694) Console progress bar not showing in 3.0

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26694. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23618 [https:

[jira] [Created] (SPARK-26731) remove EOLed spark jobs from jenkins

2019-01-25 Thread shane knapp (JIRA)
shane knapp created SPARK-26731: --- Summary: remove EOLed spark jobs from jenkins Key: SPARK-26731 URL: https://issues.apache.org/jira/browse/SPARK-26731 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-26254) Move delegation token providers into a separate project

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26254. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23499 [https:

[jira] [Assigned] (SPARK-26254) Move delegation token providers into a separate project

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26254: -- Assignee: Gabor Somogyi > Move delegation token providers into a separate project > -

[jira] [Updated] (SPARK-26731) remove EOLed spark jobs from jenkins

2019-01-25 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shane knapp updated SPARK-26731: Description: i will disable, but not remove (yet), the branch-specific builds for 1.6, 2.0 and 2.

[jira] [Updated] (SPARK-26731) remove EOLed spark jobs from jenkins

2019-01-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26731: -- Target Version/s: 3.0.0 Priority: Major (was: Critical) Let's, eh, delete them when 3.0 i

[jira] [Commented] (SPARK-26731) remove EOLed spark jobs from jenkins

2019-01-25 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752586#comment-16752586 ] shane knapp commented on SPARK-26731: - ok, should i hold off on disabling them until

[jira] [Commented] (SPARK-26677) Incorrect results of not(eqNullSafe) when data read from Parquet file

2019-01-25 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752606#comment-16752606 ] Ryan Blue commented on SPARK-26677: --- To clarify [~dongjoon]'s comment: All recent vers

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752613#comment-16752613 ] Dongjoon Hyun commented on SPARK-26727: --- Hi, [~yarrasree], [~ramgopalnaali], [~rig

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752607#comment-16752607 ] Dongjoon Hyun commented on SPARK-26727: --- Interesting issue. cc [~cloud_fan] and [~

[jira] [Resolved] (SPARK-22404) Provide an option to use unmanaged AM in yarn-client mode

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22404. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 19616 [https:

[jira] [Commented] (SPARK-26677) Incorrect results of not(eqNullSafe) when data read from Parquet file

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752619#comment-16752619 ] Dongjoon Hyun commented on SPARK-26677: --- Yep. Correct. There were two mixed issues

[jira] [Comment Edited] (SPARK-26677) Incorrect results of not(eqNullSafe) when data read from Parquet file

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752619#comment-16752619 ] Dongjoon Hyun edited comment on SPARK-26677 at 1/25/19 7:56 PM: --

[jira] [Assigned] (SPARK-23385) Allow SparkUITab to be customized adding in SparkConf and loaded when creating SparkUI

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23385: -- Assignee: (was: Marcelo Vanzin) > Allow SparkUITab to be customized adding in Spa

[jira] [Assigned] (SPARK-23385) Allow SparkUITab to be customized adding in SparkConf and loaded when creating SparkUI

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23385: Assignee: Apache Spark > Allow SparkUITab to be customized adding in SparkConf and loaded

[jira] [Assigned] (SPARK-23385) Allow SparkUITab to be customized adding in SparkConf and loaded when creating SparkUI

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23385: Assignee: (was: Apache Spark) > Allow SparkUITab to be customized adding in SparkConf

[jira] [Assigned] (SPARK-22404) Provide an option to use unmanaged AM in yarn-client mode

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22404: -- Assignee: Devaraj K > Provide an option to use unmanaged AM in yarn-client mode > ---

[jira] [Assigned] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26711: Assignee: Apache Spark > JSON Schema inference takes 15 times longer > --

[jira] [Assigned] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26711: Assignee: (was: Apache Spark) > JSON Schema inference takes 15 times longer > ---

[jira] [Resolved] (SPARK-23385) Allow SparkUITab to be customized adding in SparkConf and loaded when creating SparkUI

2019-01-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23385. --- Resolution: Won't Fix > Allow SparkUITab to be customized adding in SparkConf and loaded when > cre

[jira] [Assigned] (SPARK-23385) Allow SparkUITab to be customized adding in SparkConf and loaded when creating SparkUI

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23385: -- Assignee: Marcelo Vanzin > Allow SparkUITab to be customized adding in SparkConf and

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752652#comment-16752652 ] Xiao Li commented on SPARK-26727: - Are you using S3-like object store? These file system

[jira] [Commented] (SPARK-23411) Deprecate SparkContext.getRDDStorageInfo

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752661#comment-16752661 ] Marcelo Vanzin commented on SPARK-23411: I saw the test for this method fail (se

[jira] [Resolved] (SPARK-26170) Add missing metrics in FlatMapGroupsWithState

2019-01-25 Thread Jose Torres (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Torres resolved SPARK-26170. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23142 [https://gith

[jira] [Commented] (SPARK-23411) Deprecate SparkContext.getRDDStorageInfo

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752734#comment-16752734 ] Marcelo Vanzin commented on SPARK-23411: Hmm, removing this method is a little t

[jira] [Created] (SPARK-26732) Flaky test: SparkContextInfoSuite.getRDDStorageInfo only reports on RDDs that actually persist data

2019-01-25 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-26732: -- Summary: Flaky test: SparkContextInfoSuite.getRDDStorageInfo only reports on RDDs that actually persist data Key: SPARK-26732 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Assigned] (SPARK-26732) Flaky test: SparkContextInfoSuite.getRDDStorageInfo only reports on RDDs that actually persist data

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26732: Assignee: (was: Apache Spark) > Flaky test: SparkContextInfoSuite.getRDDStorageInfo o

[jira] [Assigned] (SPARK-26732) Flaky test: SparkContextInfoSuite.getRDDStorageInfo only reports on RDDs that actually persist data

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26732: Assignee: Apache Spark > Flaky test: SparkContextInfoSuite.getRDDStorageInfo only reports

[jira] [Resolved] (SPARK-26379) Structured Streaming - Exception on adding column to Dataset

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26379. --- Resolution: Fixed Assignee: Jungtaek Lim Fix Version/s: 3.0.0 This is resolv

[jira] [Commented] (SPARK-26678) Empty values end up as quoted empty strings in CSV files

2019-01-25 Thread Robert V (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752785#comment-16752785 ] Robert V commented on SPARK-26678: -- Yes, this parameter solved the problem as documente

[jira] [Created] (SPARK-26733) Clean up entrypoint.sh

2019-01-25 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-26733: -- Summary: Clean up entrypoint.sh Key: SPARK-26733 URL: https://issues.apache.org/jira/browse/SPARK-26733 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-26733) Clean up entrypoint.sh

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26733: Assignee: (was: Apache Spark) > Clean up entrypoint.sh > -- > >

[jira] [Updated] (SPARK-26379) Fix issue on adding current_timestamp/current_date to streaming query

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26379: -- Fix Version/s: 2.4.1 > Fix issue on adding current_timestamp/current_date to streaming query >

[jira] [Assigned] (SPARK-26733) Clean up entrypoint.sh

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26733: Assignee: Apache Spark > Clean up entrypoint.sh > -- > >

[jira] [Resolved] (SPARK-23078) Allow Submitting Spark Thrift Server in Cluster Mode

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23078. Resolution: Won't Do No answer to my question, assuming use case is now covered. > Allow

[jira] [Updated] (SPARK-26379) Fix issue on adding current_timestamp/current_date to streaming query

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26379: -- Summary: Fix issue on adding current_timestamp/current_date to streaming query (was: Structur

[jira] [Resolved] (SPARK-25751) Unit Testing for Kerberos Support for Spark on Kubernetes

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25751. Resolution: Won't Do Other changes in this area introduced unit tests already, so this isn

[jira] [Assigned] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25825: -- Assignee: (was: Marcelo Vanzin) > Kerberos Support for Long Running Jobs in Kuber

[jira] [Assigned] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25825: -- Assignee: Marcelo Vanzin > Kerberos Support for Long Running Jobs in Kubernetes > --

[jira] [Resolved] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

2019-01-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25825. Resolution: Won't Do I'm going to close this for now since there isn't a definition of wha

[jira] [Updated] (SPARK-26566) Upgrade apache/arrow to 0.12.0

2019-01-25 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-26566: - Description: Version 0.12.0 includes the following selected fixes/improvements relevant to Spar

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

2019-01-25 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752822#comment-16752822 ] Liang-Chi Hsieh commented on SPARK-26727: - When it asks if a table exists, doesn

[jira] [Updated] (SPARK-26566) Upgrade apache/arrow to 0.12.0

2019-01-25 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-26566: - Description: Version 0.12.0 includes the following selected fixes/improvements relevant to Spar

[jira] [Assigned] (SPARK-26566) Upgrade apache/arrow to 0.12.0

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26566: Assignee: Apache Spark > Upgrade apache/arrow to 0.12.0 > --

[jira] [Assigned] (SPARK-26566) Upgrade apache/arrow to 0.12.0

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26566: Assignee: (was: Apache Spark) > Upgrade apache/arrow to 0.12.0 >

[jira] [Created] (SPARK-26734) StackOverflowError on WAL serialization caused by large receivedBlockQueue

2019-01-25 Thread Ross M. Lodge (JIRA)
Ross M. Lodge created SPARK-26734: - Summary: StackOverflowError on WAL serialization caused by large receivedBlockQueue Key: SPARK-26734 URL: https://issues.apache.org/jira/browse/SPARK-26734 Project:

[jira] [Resolved] (SPARK-26711) JSON Schema inference takes 15 times longer

2019-01-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26711. --- Resolution: Fixed Assignee: Bruce Robbins Fix Version/s: 3.0.0 This is resol

[jira] [Resolved] (SPARK-26709) OptimizeMetadataOnlyQuery does not correctly handle the files with zero record

2019-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-26709. -- Resolution: Fixed Fix Version/s: 3.0.0 2.4.1

[jira] [Created] (SPARK-26735) Verify plan integrity for special expressions

2019-01-25 Thread Kris Mok (JIRA)
Kris Mok created SPARK-26735: Summary: Verify plan integrity for special expressions Key: SPARK-26735 URL: https://issues.apache.org/jira/browse/SPARK-26735 Project: Spark Issue Type: Improvement

  1   2   >