[jira] [Resolved] (SPARK-29258) parity between ml.evaluator and mllib.metrics

2019-09-26 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-29258. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25940

[jira] [Commented] (SPARK-29269) Pyspark ALSModel support getters/setters

2019-09-26 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939109#comment-16939109 ] Huaxin Gao commented on SPARK-29269: Yes. I am happy to work on this. Thanks! [~podongfeng] >

[jira] [Resolved] (SPARK-27715) SQL query details in UI dose not show in correct format.

2019-09-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-27715. -- Fix Version/s: 3.0.0 Assignee: Genmao Yu Resolution: Fixed Resolved by

[jira] [Resolved] (SPARK-29175) Make maven central repository in IsolatedClientLoader configurable

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29175. --- Fix Version/s: 3.0.0 Assignee: Yuanjian Li Resolution: Fixed This is

[jira] [Commented] (SPARK-29245) CCE during creating HiveMetaStoreClient

2019-09-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939099#comment-16939099 ] Hyukjin Kwon commented on SPARK-29245: -- BTW, can we confirm that this is the last one we need for

[jira] [Updated] (SPARK-26431) Update availableSlots by availableCpus for barrier taskset

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26431: -- Fix Version/s: (was: 2.4.0) > Update availableSlots by availableCpus for barrier taskset

[jira] [Resolved] (SPARK-29246) Remove unnecessary imports in `core` module

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29246. --- Fix Version/s: 3.0.0 Resolution: Fixed This is resolved via

[jira] [Created] (SPARK-29269) Pyspark ALSModel support getters/setters

2019-09-26 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-29269: Summary: Pyspark ALSModel support getters/setters Key: SPARK-29269 URL: https://issues.apache.org/jira/browse/SPARK-29269 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-29142) Pyspark clustering models support column setters/getters/predict

2019-09-26 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-29142. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25859

[jira] [Commented] (SPARK-29212) Add common classes without using JVM backend

2019-09-26 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939082#comment-16939082 ] zhengruifeng commented on SPARK-29212: -- [~zero323] I had not notice the base hierarchy without

[jira] [Commented] (SPARK-29257) All Task attempts scheduled to the same executor inevitably access the same bad disk

2019-09-26 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939075#comment-16939075 ] Kent Yao commented on SPARK-29257: -- this issue should be not a problem anymore after this fix - 

[jira] [Commented] (SPARK-29268) Failed to start spark-sql when using Derby metastore and isolatedLoader is enabled

2019-09-26 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939073#comment-16939073 ] Sandeep Katta commented on SPARK-29268: --- I will work on this > Failed to start spark-sql when

[jira] [Updated] (SPARK-29268) Failed to start spark-sql when using Derby metastore and isolatedLoader is enabled

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29268: Description: Failed to start spark-sql when using Derby metastore and isolatedLoader is

[jira] [Updated] (SPARK-29268) Failed to start spark-sql when using Derby metastore and isolatedLoader is enabled

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29268: Description: Failed to start spark-sql when using Derby metastore and isolatedLoader is

[jira] [Created] (SPARK-29268) Failed to start spark-sql when using Derby metastore and isolatedLoader is enabled

2019-09-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29268: --- Summary: Failed to start spark-sql when using Derby metastore and isolatedLoader is enabled Key: SPARK-29268 URL: https://issues.apache.org/jira/browse/SPARK-29268

[jira] [Commented] (SPARK-29254) Failed to include jars passed in through --jars when isolatedLoader is enabled

2019-09-26 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939068#comment-16939068 ] angerszhu commented on SPARK-29254: --- [~cltlfcjin] Not same reason. > Failed to include jars passed in

[jira] [Commented] (SPARK-29262) DataFrameWriter insertIntoPartition function

2019-09-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939067#comment-16939067 ] Wenchen Fan commented on SPARK-29262: - There is a `DataFrameWriterV2`, we can consider adding this

[jira] [Created] (SPARK-29267) rdd.countApprox should stop when 'timeout'

2019-09-26 Thread Kangtian (Jira)
Kangtian created SPARK-29267: Summary: rdd.countApprox should stop when 'timeout' Key: SPARK-29267 URL: https://issues.apache.org/jira/browse/SPARK-29267 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-29266) Optimize Dataset.isEmpty for base relations / unfiltered datasets

2019-09-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-29266: --- Description: SPARK-23627 added a {{Dataset.isEmpty}} method. This is currently implemented as

[jira] [Updated] (SPARK-29246) Remove unnecessary imports in `core` module

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29246: -- Summary: Remove unnecessary imports in `core` module (was: Remove unnecessary imports in

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 10:54 PM:

[jira] [Assigned] (SPARK-29246) Remove unnecessary imports in `core` module

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29246: - Assignee: Jiaqi Li > Remove unnecessary imports in `core` module >

[jira] [Updated] (SPARK-29246) Remove unnecessary imports in `core` module

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29246: -- Component/s: (was: Tests) > Remove unnecessary imports in `core` module >

[jira] [Assigned] (SPARK-29259) Filesystem.exists is called even when not necessary for append save mode

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29259: - Assignee: Rahij Ramsharan > Filesystem.exists is called even when not necessary for

[jira] [Resolved] (SPARK-29259) Filesystem.exists is called even when not necessary for append save mode

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29259. --- Fix Version/s: 3.0.0 Resolution: Fixed This is resolved via

[jira] [Updated] (SPARK-29259) Filesystem.exists is called even when not necessary for append save mode

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29259: -- Affects Version/s: (was: 2.4.4) 3.0.0 > Filesystem.exists is

[jira] [Updated] (SPARK-29266) Optimize Dataset.isEmpty for base relations / unfiltered datasets

2019-09-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-29266: --- Description: SPARK-23627 added a {{Dataset.isEmpty}} method. This is currently implemented as

[jira] [Updated] (SPARK-29266) Optimize Dataset.isEmpty for base relations / unfiltered datasets

2019-09-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-29266: --- Description: SPARK-23627 added a {{Dataset.isEmpty}} method. This is currently implemented as

[jira] [Updated] (SPARK-29266) Optimize Dataset.isEmpty for base relations / unfiltered datasets

2019-09-26 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-29266: --- Description: SPARK-23627 added a {{Dataset.isEmpty}} method. This is currently implemented as

[jira] [Created] (SPARK-29266) Optimize Dataset.isEmpty for base relations / unfiltered datasets

2019-09-26 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-29266: -- Summary: Optimize Dataset.isEmpty for base relations / unfiltered datasets Key: SPARK-29266 URL: https://issues.apache.org/jira/browse/SPARK-29266 Project: Spark

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Attachment: (was: Test.scala) > Window orderBy causing full-DF orderBy >

[jira] [Resolved] (SPARK-29202) --driver-java-options are not passed to driver process in yarn client mode

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29202. --- Fix Version/s: 3.0.0 Assignee: Sandeep Katta Resolution: Fixed This is

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Attachment: (was: TestSpark.zip) > Window orderBy causing full-DF orderBy >

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Affects Version/s: (was: 2.4.4) > Window orderBy causing full-DF orderBy >

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Affects Version/s: 2.4.4 > Window orderBy causing full-DF orderBy >

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 10:18 PM:

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 10:18 PM:

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 10:07 PM:

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 10:07 PM:

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 10:03 PM:

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 10:01 PM:

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Affects Version/s: (was: 2.4.4) (was: 2.4.3)

[jira] [Updated] (SPARK-29242) Check results of expression examples automatically

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29242: -- Issue Type: Improvement (was: Test) > Check results of expression examples automatically >

[jira] [Updated] (SPARK-29242) Check results of expression examples automatically

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29242: -- Component/s: Tests > Check results of expression examples automatically >

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 9:54 PM: ---

[jira] [Comment Edited] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz edited comment on SPARK-29265 at 9/26/19 9:54 PM: ---

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Commented] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938989#comment-16938989 ] Florentino Sainz commented on SPARK-29265: -- Sorry, ima test again... I'm seeing something that

[jira] [Resolved] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz resolved SPARK-29265. -- Resolution: Not A Bug > Window orderBy causing full-DF orderBy >

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Attachment: TestSpark.zip > Window orderBy causing full-DF orderBy >

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Attachment: Test.scala > Window orderBy causing full-DF orderBy >

[jira] [Created] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
Florentino Sainz created SPARK-29265: Summary: Window orderBy causing full-DF orderBy Key: SPARK-29265 URL: https://issues.apache.org/jira/browse/SPARK-29265 Project: Spark Issue Type:

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Updated] (SPARK-29265) Window orderBy causing full-DF orderBy

2019-09-26 Thread Florentino Sainz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florentino Sainz updated SPARK-29265: - Description: Hi,   I had this problem in "real" environments and also made a

[jira] [Resolved] (SPARK-29264) Incorrect example for the RLike expression

2019-09-26 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-29264. Resolution: Won't Fix > Incorrect example for the RLike expression >

[jira] [Commented] (SPARK-29264) Incorrect example for the RLike expression

2019-09-26 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938952#comment-16938952 ] Maxim Gekk commented on SPARK-29264: Remove Rlike from the ignore set after fix: 

[jira] [Updated] (SPARK-29264) Incorrect example for the RLike expression

2019-09-26 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-29264: --- Summary: Incorrect example for the RLike expression (was: Fix examples for the RLike expression)

[jira] [Created] (SPARK-29264) Fix examples for the RLike expression

2019-09-26 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-29264: -- Summary: Fix examples for the RLike expression Key: SPARK-29264 URL: https://issues.apache.org/jira/browse/SPARK-29264 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-28917) Jobs can hang because of race of RDD.dependencies

2019-09-26 Thread Imran Rashid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-28917: - Description: {{RDD.dependencies}} stores the precomputed cache value, but it is not

[jira] [Created] (SPARK-29263) availableSlots in scheduler can change before being checked by barrier taskset

2019-09-26 Thread Juliusz Sompolski (Jira)
Juliusz Sompolski created SPARK-29263: - Summary: availableSlots in scheduler can change before being checked by barrier taskset Key: SPARK-29263 URL: https://issues.apache.org/jira/browse/SPARK-29263

[jira] [Commented] (SPARK-29245) CCE during creating HiveMetaStoreClient

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938857#comment-16938857 ] Dongjoon Hyun commented on SPARK-29245: --- In addition to [~yumwang]'s configuration, in CDH 6.3,

[jira] [Updated] (SPARK-25475) Refactor all benchmark to save the result as a separate file

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25475: -- Target Version/s: 3.0.0 > Refactor all benchmark to save the result as a separate file >

[jira] [Assigned] (SPARK-29242) Check results of expression examples automatically

2019-09-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29242: - Assignee: Maxim Gekk > Check results of expression examples automatically >

[jira] [Commented] (SPARK-29248) Pass in number of partitions to BuildWriter

2019-09-26 Thread Ximo Guanter (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938743#comment-16938743 ] Ximo Guanter commented on SPARK-29248: -- [~kabhwan], I have opened 

[jira] [Commented] (SPARK-29254) Failed to include jars passed in through --jars when isolatedLoader is enabled

2019-09-26 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938725#comment-16938725 ] Lantao Jin commented on SPARK-29254: Does it duplicate to

[jira] [Created] (SPARK-29262) DataFrameWriter insertIntoPartition function

2019-09-26 Thread feiwang (Jira)
feiwang created SPARK-29262: --- Summary: DataFrameWriter insertIntoPartition function Key: SPARK-29262 URL: https://issues.apache.org/jira/browse/SPARK-29262 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-29256) Fix typo in building document

2019-09-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29256. -- Fix Version/s: 3.0.0 Assignee: Tomoko Komiyama Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-29256) Fix typo in building document

2019-09-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938594#comment-16938594 ] Sean R. Owen commented on SPARK-29256: -- There's no need to file a JIRA for this kind of thing. >

[jira] [Updated] (SPARK-29256) Fix typo in building document

2019-09-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29256: - Issue Type: Improvement (was: Bug) > Fix typo in building document >

[jira] [Commented] (SPARK-29059) [SPIP] Support for Hive Materialized Views in Spark SQL.

2019-09-26 Thread Amogh Margoor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938588#comment-16938588 ] Amogh Margoor commented on SPARK-29059: --- Pull Request here with all the optimizer changes: [GitHub

[jira] [Updated] (SPARK-29059) [SPIP] Support for Hive Materialized Views in Spark SQL.

2019-09-26 Thread Amogh Margoor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amogh Margoor updated SPARK-29059: -- Issue Type: New Feature (was: Task) > [SPIP] Support for Hive Materialized Views in Spark

[jira] [Updated] (SPARK-29059) [SPIP] Support for Hive Materialized Views in Spark SQL.

2019-09-26 Thread Amogh Margoor (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amogh Margoor updated SPARK-29059: -- Summary: [SPIP] Support for Hive Materialized Views in Spark SQL. (was: Support for Hive

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2019-09-26 Thread Gary D. Gregory (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938581#comment-16938581 ] Gary D. Gregory commented on SPARK-6305: Log4j 2 provides a clean separation between API and

[jira] [Resolved] (SPARK-28997) Add `spark.sql.dialect`

2019-09-26 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28997. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25697

[jira] [Resolved] (SPARK-29255) Rename package pgSQL to postgreSQL

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-29255. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25936

[jira] [Created] (SPARK-29261) Support recover live entities from KVStore for (SQL)AppStatusListener

2019-09-26 Thread wuyi (Jira)
wuyi created SPARK-29261: Summary: Support recover live entities from KVStore for (SQL)AppStatusListener Key: SPARK-29261 URL: https://issues.apache.org/jira/browse/SPARK-29261 Project: Spark Issue

[jira] [Updated] (SPARK-29203) Reduce shuffle partitions in SQLQueryTestSuite

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29203: Fix Version/s: 2.4.5 > Reduce shuffle partitions in SQLQueryTestSuite >

[jira] [Updated] (SPARK-29203) Reduce shuffle partitions in SQLQueryTestSuite

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29203: Affects Version/s: 2.4.4 > Reduce shuffle partitions in SQLQueryTestSuite >

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2019-09-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938506#comment-16938506 ] Steve Loughran commented on SPARK-6305: --- bq. Steve, apart from the security issue concern, we also

[jira] [Updated] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29260: Description: Hive 3.x is supported currently. Hive 2.2.1 and Hive 2.4.0 have not released. (was:

[jira] [Updated] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29260: Description: Hive 3.0.x and Hive 3.1.x is supported. Hive 2.2.1, 2.4.0 haven't released. >

[jira] [Created] (SPARK-29260) Enable supported Hive metastore versions once it support altering database location

2019-09-26 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29260: --- Summary: Enable supported Hive metastore versions once it support altering database location Key: SPARK-29260 URL: https://issues.apache.org/jira/browse/SPARK-29260

[jira] [Comment Edited] (SPARK-6305) Add support for log4j 2.x to Spark

2019-09-26 Thread Rajiv Bandi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938467#comment-16938467 ] Rajiv Bandi edited comment on SPARK-6305 at 9/26/19 10:20 AM: -- Steve, apart

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2019-09-26 Thread Rajiv Bandi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938467#comment-16938467 ] Rajiv Bandi commented on SPARK-6305: Steve, apart from the security issue concern, we also have the

[jira] [Commented] (SPARK-29212) Add common classes without using JVM backend

2019-09-26 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938465#comment-16938465 ] Maciej Szymkiewicz commented on SPARK-29212: [~podongfeng] First of all thank your for

[jira] [Resolved] (SPARK-28845) Enable spark.sql.execution.sortBeforeRepartition only for retried stages

2019-09-26 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li resolved SPARK-28845. - Resolution: Won't Do After further investigation, I found the objective of performing sort only

[jira] [Commented] (SPARK-29245) CCE during creating HiveMetaStoreClient

2019-09-26 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938439#comment-16938439 ] Yuming Wang commented on SPARK-29245: - How to reproduce: Hive side: {code:sh} export

[jira] [Issue Comment Deleted] (SPARK-29254) Failed to include jars passed in through --jars when isolatedLoader is enabled

2019-09-26 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta updated SPARK-29254: -- Comment: was deleted (was: [~yumwang] I would like to work on it if you have not started )

[jira] [Commented] (SPARK-29254) Failed to include jars passed in through --jars when isolatedLoader is enabled

2019-09-26 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938434#comment-16938434 ] Sandeep Katta commented on SPARK-29254: --- [~yumwang] I would like to work on it if you have not

[jira] [Commented] (SPARK-29254) Failed to include jars passed in through --jars when isolatedLoader is enabled

2019-09-26 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938433#comment-16938433 ] Lantao Jin commented on SPARK-29254: I am looking into this. > Failed to include jars passed in

[jira] [Created] (SPARK-29259) Filesystem.exists is called even when not necessary for append save mode

2019-09-26 Thread Rahij Ramsharan (Jira)
Rahij Ramsharan created SPARK-29259: --- Summary: Filesystem.exists is called even when not necessary for append save mode Key: SPARK-29259 URL: https://issues.apache.org/jira/browse/SPARK-29259

[jira] [Updated] (SPARK-29257) All Task attempts scheduled to the same executor inevitably access the same bad disk

2019-09-26 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-29257: - Attachment: image-2019-09-26-16-44-48-554.png > All Task attempts scheduled to the same executor

[jira] [Updated] (SPARK-29257) All Task attempts scheduled to the same executor inevitably access the same bad disk

2019-09-26 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-29257: - Description: We have an HDFS/YARN cluster with about 2k~3k nodes, each node has 8T * 12 local disks

[jira] [Created] (SPARK-29258) parity between ml.evaluator and mllib.metrics

2019-09-26 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-29258: Summary: parity between ml.evaluator and mllib.metrics Key: SPARK-29258 URL: https://issues.apache.org/jira/browse/SPARK-29258 Project: Spark Issue Type:

[jira] [Created] (SPARK-29257) All Task attempts scheduled to the same executor inevitably access the same bad disk

2019-09-26 Thread Kent Yao (Jira)
Kent Yao created SPARK-29257: Summary: All Task attempts scheduled to the same executor inevitably access the same bad disk Key: SPARK-29257 URL: https://issues.apache.org/jira/browse/SPARK-29257

  1   2   >