[jira] [Resolved] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-23084. - Resolution: Won't Fix Fix Version/s: (was: 2.4.0) This was merged but then reverted d

[jira] [Reopened] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-23084: - > Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark >

[jira] [Created] (SPARK-25849) Improve document for task cancellation.

2018-10-25 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-25849: Summary: Improve document for task cancellation. Key: SPARK-25849 URL: https://issues.apache.org/jira/browse/SPARK-25849 Project: Spark Issue Type: Documenta

[jira] [Commented] (SPARK-20780) Spark Kafka10 Consumer Hangs

2018-10-25 Thread Donghyun Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664686#comment-16664686 ] Donghyun Kim commented on SPARK-20780: -- Hello [~jayadeepj]. I am facing the same pr

[jira] [Updated] (SPARK-25848) Refactor CSVBenchmarks to use main method

2018-10-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25848: Issue Type: Sub-task (was: Improvement) Parent: SPARK-25475 > Refactor CSVBenchmarks to u

[jira] [Updated] (SPARK-25848) Refactor CSVBenchmarks to use main method

2018-10-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25848: Affects Version/s: (was: 2.4.1) 3.0.0 > Refactor CSVBenchmarks to use m

[jira] [Updated] (SPARK-25847) Refactor JSONBenchmarks to use main method

2018-10-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25847: Affects Version/s: (was: 2.4.1) 3.0.0 > Refactor JSONBenchmarks to use

[jira] [Updated] (SPARK-25847) Refactor JSONBenchmarks to use main method

2018-10-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25847: Issue Type: Sub-task (was: Improvement) Parent: SPARK-25475 > Refactor JSONBenchmarks to

[jira] [Resolved] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25842. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22841 [https://gith

[jira] [Resolved] (SPARK-25822) Fix a race condition when releasing a Python worker

2018-10-25 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-25822. --- Resolution: Fixed Fix Version/s: 2.4.0 2.3.3 Issue resolved by pul

[jira] [Assigned] (SPARK-25848) Refactor CSVBenchmarks to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25848: Assignee: Apache Spark > Refactor CSVBenchmarks to use main method >

[jira] [Commented] (SPARK-25848) Refactor CSVBenchmarks to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664637#comment-16664637 ] Apache Spark commented on SPARK-25848: -- User 'heary-cao' has created a pull request

[jira] [Assigned] (SPARK-25848) Refactor CSVBenchmarks to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25848: Assignee: (was: Apache Spark) > Refactor CSVBenchmarks to use main method > -

[jira] [Commented] (SPARK-25848) Refactor CSVBenchmarks to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664636#comment-16664636 ] Apache Spark commented on SPARK-25848: -- User 'heary-cao' has created a pull request

[jira] [Created] (SPARK-25848) Refactor CSVBenchmarks to use main method

2018-10-25 Thread caoxuewen (JIRA)
caoxuewen created SPARK-25848: - Summary: Refactor CSVBenchmarks to use main method Key: SPARK-25848 URL: https://issues.apache.org/jira/browse/SPARK-25848 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-25847) Refactor JSONBenchmarks to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25847: Assignee: Apache Spark > Refactor JSONBenchmarks to use main method > ---

[jira] [Commented] (SPARK-25847) Refactor JSONBenchmarks to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664633#comment-16664633 ] Apache Spark commented on SPARK-25847: -- User 'heary-cao' has created a pull request

[jira] [Assigned] (SPARK-25847) Refactor JSONBenchmarks to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25847: Assignee: (was: Apache Spark) > Refactor JSONBenchmarks to use main method >

[jira] [Resolved] (SPARK-15545) R remove non-exported unused methods, like jsonRDD

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-15545. -- Resolution: Duplicate > R remove non-exported unused methods, like jsonRDD > -

[jira] [Updated] (SPARK-15545) R remove non-exported unused methods, like jsonRDD

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-15545: - Affects Version/s: 2.3.2 External issue ID: SPARK-12172 > R remove non-exported unused metho

[jira] [Created] (SPARK-25847) Refactor JSONBenchmarks to use main method

2018-10-25 Thread caoxuewen (JIRA)
caoxuewen created SPARK-25847: - Summary: Refactor JSONBenchmarks to use main method Key: SPARK-25847 URL: https://issues.apache.org/jira/browse/SPARK-25847 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-12172) Consider removing SparkR internal RDD APIs

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664631#comment-16664631 ] Felix Cheung edited comment on SPARK-12172 at 10/26/18 4:11 AM: --

[jira] [Commented] (SPARK-12172) Consider removing SparkR internal RDD APIs

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664631#comment-16664631 ] Felix Cheung commented on SPARK-12172: -- ok, what's our option for spark.lapply? >

[jira] [Commented] (SPARK-16611) Expose several hidden DataFrame/RDD functions

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664628#comment-16664628 ] Felix Cheung commented on SPARK-16611: -- ping - we are going to consider removing RD

[jira] [Commented] (SPARK-16611) Expose several hidden DataFrame/RDD functions

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664630#comment-16664630 ] Felix Cheung commented on SPARK-16611: -- see SPARK-12172 > Expose several hidden Da

[jira] [Commented] (SPARK-16693) Remove R deprecated methods

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664627#comment-16664627 ] Apache Spark commented on SPARK-16693: -- User 'felixcheung' has created a pull reque

[jira] [Commented] (SPARK-16693) Remove R deprecated methods

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664626#comment-16664626 ] Felix Cheung commented on SPARK-16693: -- rebuilt this on spark 3.0.0 > Remove R dep

[jira] [Commented] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664619#comment-16664619 ] Dongjoon Hyun commented on SPARK-23084: --- Unfortunately, this is going to be revert

[jira] [Updated] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23938: -- Target Version/s: (was: 2.4.0) > High-order function: map_zip_with(map, map, function V3>) →

[jira] [Updated] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23938: -- Fix Version/s: (was: 2.4.0) 3.0.0 > High-order function: map_zip_with(m

[jira] [Commented] (SPARK-23939) High-order function: transform_keys(map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664616#comment-16664616 ] Dongjoon Hyun commented on SPARK-23939: --- I updated the version since this is rever

[jira] [Commented] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664617#comment-16664617 ] Dongjoon Hyun commented on SPARK-23938: --- I updated the version since this is rever

[jira] [Updated] (SPARK-23939) High-order function: transform_keys(map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23939: -- Fix Version/s: (was: 2.4.0) 3.0.0 > High-order function: transform_keys

[jira] [Commented] (SPARK-23940) High-order function: transform_values(map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664615#comment-16664615 ] Dongjoon Hyun commented on SPARK-23940: --- I updated the version since this is rever

[jira] [Updated] (SPARK-23939) High-order function: transform_keys(map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23939: -- Target Version/s: (was: 2.4.0) > High-order function: transform_keys(map, function) → > map

[jira] [Updated] (SPARK-23940) High-order function: transform_values(map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23940: -- Fix Version/s: (was: 2.4.0) 3.0.0 > High-order function: transform_valu

[jira] [Updated] (SPARK-23940) High-order function: transform_values(map, function) → map

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23940: -- Target Version/s: (was: 2.4.0) > High-order function: transform_values(map, function) → > m

[jira] [Updated] (SPARK-23937) High-order function: map_filter(map, function) → MAP

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23937: -- Target Version/s: (was: 2.4.0) > High-order function: map_filter(map, function) → MAP >

[jira] [Updated] (SPARK-23937) High-order function: map_filter(map, function) → MAP

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23937: -- Fix Version/s: (was: 2.4.0) 3.0.0 > High-order function: map_filter(map

[jira] [Commented] (SPARK-23937) High-order function: map_filter(map, function) → MAP

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664614#comment-16664614 ] Dongjoon Hyun commented on SPARK-23937: --- I updated the version since this is rever

[jira] [Updated] (SPARK-23935) High-order function: map_entries(map) → array>

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23935: -- Fix Version/s: (was: 2.4.0) 3.0.0 > High-order function: map_entries(ma

[jira] [Updated] (SPARK-23935) High-order function: map_entries(map) → array>

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23935: -- Target Version/s: (was: 2.4.0) > High-order function: map_entries(map) → array> > --

[jira] [Assigned] (SPARK-25819) Support parse mode option for function `from_avro`

2018-10-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25819: Assignee: Gengliang Wang > Support parse mode option for function `from_avro` > -

[jira] [Resolved] (SPARK-25819) Support parse mode option for function `from_avro`

2018-10-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25819. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22814 [https://gi

[jira] [Updated] (SPARK-16693) Remove R deprecated methods

2018-10-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-16693: - Description: For methods deprecated in Spark 2.0.0, we should remove them in 2.1.0 -> 3.0.0 (wa

[jira] [Updated] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25840: -- Affects Version/s: (was: 3.0.0) 2.4.0 > `make-distribution.sh` shou

[jira] [Resolved] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25840. --- Resolution: Fixed Fix Version/s: 2.4.0 3.0.0 Issue resolved by pul

[jira] [Assigned] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25840: - Assignee: Dongjoon Hyun > `make-distribution.sh` should not fail due to missing LICENSE

[jira] [Updated] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25840: -- Fix Version/s: (was: 3.0.0) > `make-distribution.sh` should not fail due to missing LICENS

[jira] [Assigned] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25793: --- Assignee: Huaxin Gao > Loading model bug in BisectingKMeans > -

[jira] [Resolved] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25793. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22790 [https://gith

[jira] [Resolved] (SPARK-25846) Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method

2018-10-25 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25846. - Resolution: Duplicate > Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method >

[jira] [Assigned] (SPARK-25846) Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25846: Assignee: Apache Spark > Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main m

[jira] [Commented] (SPARK-25846) Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664555#comment-16664555 ] Apache Spark commented on SPARK-25846: -- User 'heary-cao' has created a pull request

[jira] [Assigned] (SPARK-25846) Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25846: Assignee: (was: Apache Spark) > Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to

[jira] [Commented] (SPARK-25846) Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664556#comment-16664556 ] Apache Spark commented on SPARK-25846: -- User 'heary-cao' has created a pull request

[jira] [Created] (SPARK-25846) Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method

2018-10-25 Thread caoxuewen (JIRA)
caoxuewen created SPARK-25846: - Summary: Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method Key: SPARK-25846 URL: https://issues.apache.org/jira/browse/SPARK-25846 Project: Spark

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

2018-10-25 Thread Jackey Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664529#comment-16664529 ] Jackey Lee commented on SPARK-24630: [~shijinkui] [~kabhwan] [~uncleGen] I have rem

[jira] [Commented] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664454#comment-16664454 ] Wenchen Fan commented on SPARK-25841: - Makes sense to me, we need a better API. > R

[jira] [Created] (SPARK-25845) Fix MatchError for calendar interval type in rangeBetween

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25845: --- Summary: Fix MatchError for calendar interval type in rangeBetween Key: SPARK-25845 URL: https://issues.apache.org/jira/browse/SPARK-25845 Project: Spark Issue

[jira] [Created] (SPARK-25844) Implement Python API once we have a new API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25844: --- Summary: Implement Python API once we have a new API Key: SPARK-25844 URL: https://issues.apache.org/jira/browse/SPARK-25844 Project: Spark Issue Type: Sub-tas

[jira] [Commented] (SPARK-21608) Window rangeBetween() API should allow literal boundary

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664439#comment-16664439 ] Apache Spark commented on SPARK-21608: -- User 'rxin' has created a pull request for

[jira] [Commented] (SPARK-21608) Window rangeBetween() API should allow literal boundary

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664440#comment-16664440 ] Apache Spark commented on SPARK-21608: -- User 'rxin' has created a pull request for

[jira] [Assigned] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25842: Assignee: Apache Spark (was: Reynold Xin) > Deprecate APIs introduced in SPARK-21608 > -

[jira] [Commented] (SPARK-21608) Window rangeBetween() API should allow literal boundary

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664438#comment-16664438 ] Apache Spark commented on SPARK-21608: -- User 'rxin' has created a pull request for

[jira] [Assigned] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25842: Assignee: Reynold Xin (was: Apache Spark) > Deprecate APIs introduced in SPARK-21608 > -

[jira] [Commented] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664435#comment-16664435 ] Apache Spark commented on SPARK-25842: -- User 'rxin' has created a pull request for

[jira] [Commented] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664437#comment-16664437 ] Apache Spark commented on SPARK-25842: -- User 'rxin' has created a pull request for

[jira] [Updated] (SPARK-25832) remove newly added map related functions

2018-10-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25832: Summary: remove newly added map related functions (was: remove newly added map related functions

[jira] [Assigned] (SPARK-25832) remove newly added map related functions from FunctionRegistry

2018-10-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25832: --- Assignee: Xiao Li (was: Wenchen Fan) > remove newly added map related functions from Funct

[jira] [Resolved] (SPARK-25832) remove newly added map related functions from FunctionRegistry

2018-10-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25832. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22827 [https://gith

[jira] [Updated] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25842: Target Version/s: 2.4.0 > Deprecate APIs introduced in SPARK-21608 > -

[jira] [Created] (SPARK-25843) Redesign rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25843: --- Summary: Redesign rangeBetween API Key: SPARK-25843 URL: https://issues.apache.org/jira/browse/SPARK-25843 Project: Spark Issue Type: Sub-task Compon

[jira] [Created] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25842: --- Summary: Deprecate APIs introduced in SPARK-21608 Key: SPARK-25842 URL: https://issues.apache.org/jira/browse/SPARK-25842 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25841: Description: As I was reviewing the Spark API changes for 2.4, I found that through organic, ad-h

[jira] [Created] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25841: --- Summary: Redesign window function rangeBetween API Key: SPARK-25841 URL: https://issues.apache.org/jira/browse/SPARK-25841 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25841: Description: As I was reviewing the Spark API changes for 2.4, I found that through organic, ad-h

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-10-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664418#comment-16664418 ] Hyukjin Kwon commented on SPARK-18673: -- Please provide some input in SPARK-20202 or

[jira] [Comment Edited] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-10-25 Thread Dagang Wei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664286#comment-16664286 ] Dagang Wei edited comment on SPARK-18673 at 10/25/18 9:20 PM:

[jira] [Updated] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25656: -- Fix Version/s: 2.4.0 > Add an example section about how to use Parquet/ORC library options > -

[jira] [Updated] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25840: -- Description: We vote for the artifacts. All releases are in the form of the *source* material

[jira] [Assigned] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25840: Assignee: (was: Apache Spark) > `make-distribution.sh` should not fail due to missing

[jira] [Assigned] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25840: Assignee: Apache Spark > `make-distribution.sh` should not fail due to missing LICENSE-bi

[jira] [Commented] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664293#comment-16664293 ] Apache Spark commented on SPARK-25840: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-25840) `make-distribution.sh` should not fail due to missing LICENSE-binary

2018-10-25 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-25840: - Summary: `make-distribution.sh` should not fail due to missing LICENSE-binary Key: SPARK-25840 URL: https://issues.apache.org/jira/browse/SPARK-25840 Project: Spark

[jira] [Resolved] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24787. Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by p

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-10-25 Thread Dagang Wei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664286#comment-16664286 ] Dagang Wei commented on SPARK-18673: Is it possible to fix in org.spark-project.hive

[jira] [Assigned] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24787: -- Assignee: Devaraj K > Events being dropped at an alarming rate due to hsync being slo

[jira] [Resolved] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25803. Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > The -n option to

[jira] [Assigned] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25803: -- Assignee: Steve Larkin > The -n option to docker-image-tool.sh causes other options t

[jira] [Resolved] (SPARK-25665) Refactor ObjectHashAggregateExecBenchmark to use main method

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25665. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22804 [https://

[jira] [Assigned] (SPARK-25665) Refactor ObjectHashAggregateExecBenchmark to use main method

2018-10-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25665: - Assignee: Peter Toth > Refactor ObjectHashAggregateExecBenchmark to use main method > -

[jira] [Commented] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664213#comment-16664213 ] Apache Spark commented on SPARK-25656: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664212#comment-16664212 ] Apache Spark commented on SPARK-25656: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-25839) Implement use of KryoPool in KryoSerializer

2018-10-25 Thread Patrick Brown (JIRA)
Patrick Brown created SPARK-25839: - Summary: Implement use of KryoPool in KryoSerializer Key: SPARK-25839 URL: https://issues.apache.org/jira/browse/SPARK-25839 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-25835) Propagate scala 2.12 profile in k8s integration tests

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664194#comment-16664194 ] Apache Spark commented on SPARK-25835: -- User 'skonto' has created a pull request fo

[jira] [Commented] (SPARK-25835) Propagate scala 2.12 profile in k8s integration tests

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664190#comment-16664190 ] Apache Spark commented on SPARK-25835: -- User 'skonto' has created a pull request fo

[jira] [Assigned] (SPARK-25835) Propagate scala 2.12 profile in k8s integration tests

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25835: Assignee: (was: Apache Spark) > Propagate scala 2.12 profile in k8s integration tests

[jira] [Assigned] (SPARK-25835) Propagate scala 2.12 profile in k8s integration tests

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25835: Assignee: Apache Spark > Propagate scala 2.12 profile in k8s integration tests >

[jira] [Commented] (SPARK-25838) Remove formatVersion from Saveable

2018-10-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664110#comment-16664110 ] Apache Spark commented on SPARK-25838: -- User 'mgaido91' has created a pull request

  1   2   >