[jira] [Assigned] (SPARK-27653) Add max_by() / min_by() SQL aggregate functions

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27653: Assignee: (was: Apache Spark) > Add max_by() / min_by() SQL aggregate functions > ---

[jira] [Assigned] (SPARK-27653) Add max_by() / min_by() SQL aggregate functions

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27653: Assignee: Apache Spark > Add max_by() / min_by() SQL aggregate functions > --

[jira] [Assigned] (SPARK-26412) Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow batches

2019-05-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-26412: - Assignee: Weichen Xu > Allow Pandas UDF to take an iterator of pd.DataFrames or Arrow b

[jira] [Assigned] (SPARK-27658) Catalog API to load functions

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27658: Assignee: (was: Apache Spark) > Catalog API to load functions > -

[jira] [Assigned] (SPARK-27658) Catalog API to load functions

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27658: Assignee: Apache Spark > Catalog API to load functions > - >

[jira] [Assigned] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27661: Assignee: (was: Apache Spark) > Add SupportsNamespaces interface for v2 catalogs > --

[jira] [Assigned] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27661: Assignee: Apache Spark > Add SupportsNamespaces interface for v2 catalogs > -

[jira] [Commented] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836055#comment-16836055 ] Fan Yunbo commented on SPARK-27663: --- [~jerryshao], [~srowen] Could you please take a

[jira] [Comment Edited] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836044#comment-16836044 ] Fan Yunbo edited comment on SPARK-27663 at 5/9/19 3:21 AM: --- Th

[jira] [Resolved] (SPARK-27207) There exists a bug with SortBasedAggregator where merge()/update() operations get invoked on the aggregate buffer without calling initialize

2019-05-08 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27207. - Resolution: Not A Problem > There exists a bug with SortBasedAggregator where merge()/update() o

[jira] [Updated] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Yunbo updated SPARK-27663: -- Attachment: (was: reran-1.png) > Task accomplished incompletely but marked as success > --

[jira] [Commented] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836054#comment-16836054 ] Fan Yunbo commented on SPARK-27663: --- Two thoughts: # The executor shut down hook was

[jira] [Updated] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Yunbo updated SPARK-27663: -- Attachment: reran-1.png reran-0.png > Task accomplished incompletely but marked as suc

[jira] [Commented] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836047#comment-16836047 ] Fan Yunbo commented on SPARK-27663: --- When I reran the query, it was finished as normal

[jira] [Updated] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Yunbo updated SPARK-27663: -- Attachment: reran-1.png > Task accomplished incompletely but marked as success > -

[jira] [Commented] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836044#comment-16836044 ] Fan Yunbo commented on SPARK-27663: --- The incomplete task's id is 17.0 in tage 98517.0

[jira] [Updated] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Yunbo updated SPARK-27663: -- Attachment: incomplte-task-0.png > Task accomplished incompletely but marked as success >

[jira] [Updated] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Yunbo updated SPARK-27663: -- Attachment: image-2019-05-09-11-10-04-602.png > Task accomplished incompletely but marked as success >

[jira] [Updated] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Yunbo updated SPARK-27663: -- Attachment: incomplte-task-2.png incomplte-task-1.png > Task accomplished incompletely

[jira] [Created] (SPARK-27663) Task accomplished incompletely but marked as success

2019-05-08 Thread Fan Yunbo (JIRA)
Fan Yunbo created SPARK-27663: - Summary: Task accomplished incompletely but marked as success Key: SPARK-27663 URL: https://issues.apache.org/jira/browse/SPARK-27663 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-27359) Joins on some array functions can be optimized

2019-05-08 Thread Nikolas Vanderhoof (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolas Vanderhoof updated SPARK-27359: --- Description: I encounter these cases frequently, and implemented the optimization ma

[jira] [Assigned] (SPARK-27359) Joins on some array functions can be optimized

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27359: Assignee: Apache Spark > Joins on some array functions can be optimized > ---

[jira] [Assigned] (SPARK-27359) Joins on some array functions can be optimized

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27359: Assignee: (was: Apache Spark) > Joins on some array functions can be optimized >

[jira] [Assigned] (SPARK-27662) SQL tab shows two jobs for one SQL command

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27662: Assignee: (was: Apache Spark) > SQL tab shows two jobs for one SQL command >

[jira] [Assigned] (SPARK-27662) SQL tab shows two jobs for one SQL command

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27662: Assignee: Apache Spark > SQL tab shows two jobs for one SQL command > ---

[jira] [Resolved] (SPARK-27627) Make option "pathGlobFilter" as a general option for all file sources

2019-05-08 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27627. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24518 [https://gi

[jira] [Assigned] (SPARK-27627) Make option "pathGlobFilter" as a general option for all file sources

2019-05-08 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27627: Assignee: Gengliang Wang > Make option "pathGlobFilter" as a general option for all file

[jira] [Updated] (SPARK-27662) SQL tab shows two jobs for one SQL command

2019-05-08 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27662: Description: !screenshot-1.png! > SQL tab shows two jobs for one SQL command > -

[jira] [Updated] (SPARK-27662) SQL tab shows two jobs for one SQL command

2019-05-08 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27662: Attachment: screenshot-1.png > SQL tab shows two jobs for one SQL command > --

[jira] [Created] (SPARK-27662) SQL tab shows two jobs for one SQL command

2019-05-08 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-27662: --- Summary: SQL tab shows two jobs for one SQL command Key: SPARK-27662 URL: https://issues.apache.org/jira/browse/SPARK-27662 Project: Spark Issue Type: Improvem

[jira] [Assigned] (SPARK-26130) Change Event Timeline Display Functionality on the Stages Page to use either REST API or data from other tables

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26130: Assignee: (was: Apache Spark) > Change Event Timeline Display Functionality on the St

[jira] [Assigned] (SPARK-26130) Change Event Timeline Display Functionality on the Stages Page to use either REST API or data from other tables

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26130: Assignee: Apache Spark > Change Event Timeline Display Functionality on the Stages Page t

[jira] [Resolved] (SPARK-27624) Fix CalenderInterval to show an empty interval correctly

2019-05-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27624. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.4.4

[jira] [Commented] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

2019-05-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835921#comment-16835921 ] Dongjoon Hyun commented on SPARK-27661: --- Ur, it seems that something went wrong. I

[jira] [Assigned] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

2019-05-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-27661: - Assignee: (was: Dongjoon Hyun) > Add SupportsNamespaces interface for v2 catalogs >

[jira] [Assigned] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

2019-05-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-27661: - Assignee: Dongjoon Hyun > Add SupportsNamespaces interface for v2 catalogs > --

[jira] [Updated] (SPARK-25139) PythonRunner#WriterThread released block after TaskRunner finally block which invoke BlockManager#releaseAllLocksForTask

2019-05-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25139: -- Fix Version/s: 2.4.4 > PythonRunner#WriterThread released block after TaskRunner finally block

[jira] [Resolved] (SPARK-27660) Allow PySpark toLocalIterator to pre-fetch data

2019-05-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-27660. -- Resolution: Duplicate Somehow this issue got created twice > Allow PySpark toLocalIterator to

[jira] [Updated] (SPARK-27659) Allow PySpark toLocalIterator to prefetch data

2019-05-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27659: - Summary: Allow PySpark toLocalIterator to prefetch data (was: Allow PySpark toLocalIterator to

[jira] [Closed] (SPARK-27660) Allow PySpark toLocalIterator to pre-fetch data

2019-05-08 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler closed SPARK-27660. > Allow PySpark toLocalIterator to pre-fetch data > --- >

[jira] [Created] (SPARK-27661) Add SupportsNamespaces interface for v2 catalogs

2019-05-08 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-27661: - Summary: Add SupportsNamespaces interface for v2 catalogs Key: SPARK-27661 URL: https://issues.apache.org/jira/browse/SPARK-27661 Project: Spark Issue Type: Improv

[jira] [Commented] (SPARK-27657) ml.util.Instrumentation.logFailure doesn't log error message

2019-05-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835780#comment-16835780 ] Xiangrui Meng commented on SPARK-27657: --- This is how JDK format the error string:

[jira] [Comment Edited] (SPARK-27657) ml.util.Instrumentation.logFailure doesn't log error message

2019-05-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835780#comment-16835780 ] Xiangrui Meng edited comment on SPARK-27657 at 5/8/19 5:42 PM: ---

[jira] [Commented] (SPARK-27657) ml.util.Instrumentation.logFailure doesn't log error message

2019-05-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835779#comment-16835779 ] Xiangrui Meng commented on SPARK-27657: --- [~mrbago] Can you send a PR to fix it? >

[jira] [Created] (SPARK-27660) Allow PySpark toLocalIterator to pre-fetch data

2019-05-08 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-27660: Summary: Allow PySpark toLocalIterator to pre-fetch data Key: SPARK-27660 URL: https://issues.apache.org/jira/browse/SPARK-27660 Project: Spark Issue Type: I

[jira] [Created] (SPARK-27659) Allow PySpark toLocalIterator to pre-fetch data

2019-05-08 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-27659: Summary: Allow PySpark toLocalIterator to pre-fetch data Key: SPARK-27659 URL: https://issues.apache.org/jira/browse/SPARK-27659 Project: Spark Issue Type: I

[jira] [Created] (SPARK-27658) Catalog API to load functions

2019-05-08 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-27658: - Summary: Catalog API to load functions Key: SPARK-27658 URL: https://issues.apache.org/jira/browse/SPARK-27658 Project: Spark Issue Type: Improvement Com

[jira] [Created] (SPARK-27657) ml.util.Instrumentation.logFailure doesn't log error message

2019-05-08 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-27657: - Summary: ml.util.Instrumentation.logFailure doesn't log error message Key: SPARK-27657 URL: https://issues.apache.org/jira/browse/SPARK-27657 Project: Spark

[jira] [Commented] (SPARK-27648) In Spark2.4 Structured Streaming:The executor storage memory increasing over time

2019-05-08 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835751#comment-16835751 ] Jungtaek Lim commented on SPARK-27648: -- [~yy3b2007com] FYI, you can monitor the (a

[jira] [Commented] (SPARK-23098) Migrate Kafka batch source to v2

2019-05-08 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835700#comment-16835700 ] Ryan Blue commented on SPARK-23098: --- I don't think there's a DSv2-related obstacle to

[jira] [Commented] (SPARK-23098) Migrate Kafka batch source to v2

2019-05-08 Thread Dylan Guedes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835693#comment-16835693 ] Dylan Guedes commented on SPARK-23098: -- [~gsomogyi] No, I didn't picked this one. W

[jira] [Commented] (SPARK-23098) Migrate Kafka batch source to v2

2019-05-08 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835684#comment-16835684 ] Gabor Somogyi commented on SPARK-23098: --- [~rdblue] since you're heavily involved i

[jira] [Commented] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2019-05-08 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835598#comment-16835598 ] shahid commented on SPARK-25861: [~drKalko]Normally Browser supports refresh page. This

[jira] [Assigned] (SPARK-27641) Unregistering a single Metrics Source with no metrics leads to removing all the metrics from other sources with the same name

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27641: Assignee: Apache Spark > Unregistering a single Metrics Source with no metrics leads to r

[jira] [Assigned] (SPARK-27641) Unregistering a single Metrics Source with no metrics leads to removing all the metrics from other sources with the same name

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27641: Assignee: (was: Apache Spark) > Unregistering a single Metrics Source with no metrics

[jira] [Commented] (SPARK-25861) Remove unused refreshInterval parameter from the headerSparkPage method.

2019-05-08 Thread Artem Kalchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835565#comment-16835565 ] Artem Kalchenko commented on SPARK-25861: - why remove it instead of adding refre

[jira] [Commented] (SPARK-20166) Use XXX for ISO timezone instead of ZZ which is FastDateFormat specific in CSV/JSON time related options

2019-05-08 Thread Shyama (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1683#comment-1683 ] Shyama commented on SPARK-20166: [~srowen],  any clue of this ?  [https://stackoverflow

[jira] [Assigned] (SPARK-27649) Unify the way you use 'spark.network.timeout'

2019-05-08 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27649: --- Assignee: jiaan.geng > Unify the way you use 'spark.network.timeout' >

[jira] [Assigned] (SPARK-27555) cannot create table by using the hive default fileformat in both hive-site.xml and spark-defaults.conf

2019-05-08 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27555: Assignee: Sandeep Katta (was: Hyukjin Kwon) > cannot create table by using the hive defa

[jira] [Resolved] (SPARK-17906) MulticlassClassificationEvaluator support target label

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-17906. -- Resolution: Not A Problem > MulticlassClassificationEvaluator support target label > -

[jira] [Commented] (SPARK-27641) Unregistering a single Metrics Source with no metrics leads to removing all the metrics from other sources with the same name

2019-05-08 Thread chunpinghe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835504#comment-16835504 ] chunpinghe commented on SPARK-27641: i am working on this! > Unregistering a single

[jira] [Resolved] (SPARK-22320) ORC should support VectorUDT/MatrixUDT

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-22320. -- Resolution: Not A Problem > ORC should support VectorUDT/MatrixUDT > -

[jira] [Resolved] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-7008. - Resolution: Not A Problem > An implementation of Factorization Machine (LibFM) >

[jira] [Resolved] (SPARK-16872) Include Gaussian Naive Bayes Classifier

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-16872. -- Resolution: Not A Problem > Include Gaussian Naive Bayes Classifier >

[jira] [Resolved] (SPARK-13677) Support Tree-Based Feature Transformation for ML

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-13677. -- Resolution: Not A Problem > Support Tree-Based Feature Transformation for ML > ---

[jira] [Resolved] (SPARK-13385) Enable AssociationRules to generate consequents with user-defined lengths

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-13385. -- Resolution: Not A Problem > Enable AssociationRules to generate consequents with user-defined

[jira] [Resolved] (SPARK-14174) Implement the Mini-Batch KMeans

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-14174. -- Resolution: Not A Problem > Implement the Mini-Batch KMeans > ---

[jira] [Resolved] (SPARK-19208) MultivariateOnlineSummarizer performance optimization

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-19208. -- Resolution: Not A Problem > MultivariateOnlineSummarizer performance optimization > --

[jira] [Resolved] (SPARK-18518) HasSolver should support allowed values

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-18518. -- Resolution: Not A Problem > HasSolver should support allowed values >

[jira] [Resolved] (SPARK-20711) MultivariateOnlineSummarizer/Summarizer incorrect min/max for NaN value

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-20711. -- Resolution: Won't Fix > MultivariateOnlineSummarizer/Summarizer incorrect min/max for NaN valu

[jira] [Resolved] (SPARK-21879) Should Scalers handel NaN values?

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-21879. -- Resolution: Won't Fix > Should Scalers handel NaN values? > -

[jira] [Resolved] (SPARK-18757) Models in Pyspark support column setters

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-18757. -- Resolution: Not A Problem > Models in Pyspark support column setters > ---

[jira] [Resolved] (SPARK-23805) support vector-size validation and Inference

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-23805. -- Resolution: Not A Problem > support vector-size validation and Inference > ---

[jira] [Resolved] (SPARK-24664) Column support name getter

2019-05-08 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-24664. -- Resolution: Not A Problem > Column support name getter > -- > >

[jira] [Assigned] (SPARK-27656) Safely register class for GraphX

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27656: Assignee: (was: Apache Spark) > Safely register class for GraphX > --

[jira] [Assigned] (SPARK-27656) Safely register class for GraphX

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27656: Assignee: Apache Spark > Safely register class for GraphX > -

[jira] [Created] (SPARK-27656) Safely register class for GraphX

2019-05-08 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-27656: Summary: Safely register class for GraphX Key: SPARK-27656 URL: https://issues.apache.org/jira/browse/SPARK-27656 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-27600) Unable to start Spark Hive Thrift Server when multiple hive server server share the same metastore

2019-05-08 Thread pin_zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835437#comment-16835437 ] pin_zhang edited comment on SPARK-27600 at 5/8/19 9:02 AM: --- [~

[jira] [Commented] (SPARK-27600) Unable to start Spark Hive Thrift Server when multiple hive server server share the same metastore

2019-05-08 Thread pin_zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835437#comment-16835437 ] pin_zhang commented on SPARK-27600: --- I think this is relate to a hive bug  https://iss

[jira] [Resolved] (SPARK-27649) Unify the way you use 'spark.network.timeout'

2019-05-08 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi resolved SPARK-27649. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24545 https://g

[jira] [Assigned] (SPARK-27622) Avoid the network when block manager fetches disk persisted RDD blocks from the same host

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27622: Assignee: (was: Apache Spark) > Avoid the network when block manager fetches disk per

[jira] [Assigned] (SPARK-27622) Avoid the network when block manager fetches disk persisted RDD blocks from the same host

2019-05-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27622: Assignee: Apache Spark > Avoid the network when block manager fetches disk persisted RDD

[jira] [Commented] (SPARK-27648) In Spark2.4 Structured Streaming:The executor storage memory increasing over time

2019-05-08 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835420#comment-16835420 ] Gabor Somogyi commented on SPARK-27648: --- [~yy3b2007com] you've mentioned this: {qu