[jira] [Commented] (SPARK-21943) When I use rest Api (/ applications / [app-id] / jobs / [job-id]) to view some of the jobs that are running jobs, the returned json information is missing the “descrip

[ https://issues.apache.org/jira/browse/SPARK-21943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160814#comment-16160814 ] Saisai Shao commented on SPARK-21943: - >From the code, it says that job description i

[jira] [Closed] (SPARK-21906) No need to runAsSparkUser to switch UserGroupInformation in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-21906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-21906. --- Resolution: Not A Problem > No need to runAsSparkUser to switch UserGroupInformation in YARN mode > -

[jira] [Commented] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160758#comment-16160758 ] Apache Spark commented on SPARK-21972: -- User 'zhengruifeng' has created a pull reque

[jira] [Assigned] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21972: Assignee: (was: Apache Spark) > Allow users to control input data persistence in ML Es

[jira] [Assigned] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21972: Assignee: Apache Spark > Allow users to control input data persistence in ML Estimators vi

[jira] [Updated] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Murching updated SPARK-21972: --- Description: Several Spark ML algorithms (LogisticRegression, LinearRegression, KMean

[jira] [Commented] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160624#comment-16160624 ] Siddharth Murching commented on SPARK-21972: Work has already begun on this i

[jira] [Updated] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Murching updated SPARK-21972: --- Description: Several Spark ML algorithms (LogisticRegression, LinearRegression, KMean

[jira] [Comment Edited] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160624#comment-16160624 ] Siddharth Murching edited comment on SPARK-21972 at 9/11/17 3:46 AM: --

[jira] [Updated] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Murching updated SPARK-21972: --- Description: Several Spark ML algorithms (LogisticRegression, LinearRegression, KMean

[jira] [Updated] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Murching updated SPARK-21972: --- Description: Several Spark ML algorithms (LogisticRegression, LinearRegression, KMean

[jira] [Updated] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Murching updated SPARK-21972: --- Description: Several Spark ML algorithms (LogisticRegression, LinearRegression, KMean

[jira] [Updated] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

[ https://issues.apache.org/jira/browse/SPARK-21972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Murching updated SPARK-21972: --- Description: Several Spark ML algorithms (LogisticRegression, LinearRegression, KMean

[jira] [Created] (SPARK-21972) Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param

Siddharth Murching created SPARK-21972: -- Summary: Allow users to control input data persistence in ML Estimators via a handlePersistence ml.Param Key: SPARK-21972 URL: https://issues.apache.org/jira/browse/SP

[jira] [Updated] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

[ https://issues.apache.org/jira/browse/SPARK-21955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] poseidon updated SPARK-21955: - Description: just in my way to know how stream , chunk , block works in netty found some nasty case. p

[jira] [Updated] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

[ https://issues.apache.org/jira/browse/SPARK-21955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] poseidon updated SPARK-21955: - Description: just in my way to know how stream , chunk , block works in netty found some nasty case. p

[jira] [Updated] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

[ https://issues.apache.org/jira/browse/SPARK-21955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] poseidon updated SPARK-21955: - Description: just in my way to know how stream , chunk , block works in netty found some nasty case. p

[jira] [Updated] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

[ https://issues.apache.org/jira/browse/SPARK-21955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] poseidon updated SPARK-21955: - Attachment: screenshot-1.png > OneForOneStreamManager may leak memory when network is poor >

[jira] [Updated] (SPARK-21971) Too many open files in Spark due to concurrent files being opened

[ https://issues.apache.org/jira/browse/SPARK-21971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21971: -- Affects Version/s: 2.2.0 > Too many open files in Spark due to concurrent files being opened >

[jira] [Assigned] (SPARK-21854) Python interface for MLOR summary

[ https://issues.apache.org/jira/browse/SPARK-21854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21854: Assignee: Apache Spark > Python interface for MLOR summary > -

[jira] [Assigned] (SPARK-21854) Python interface for MLOR summary

[ https://issues.apache.org/jira/browse/SPARK-21854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21854: Assignee: (was: Apache Spark) > Python interface for MLOR summary > --

[jira] [Commented] (SPARK-21854) Python interface for MLOR summary

[ https://issues.apache.org/jira/browse/SPARK-21854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160616#comment-16160616 ] Apache Spark commented on SPARK-21854: -- User 'jmwdpk' has created a pull request for

[jira] [Updated] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

[ https://issues.apache.org/jira/browse/SPARK-21955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] poseidon updated SPARK-21955: - Description: just in my way to know how stream , chunk , block works in netty found some nasty case. p

[jira] [Updated] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

[ https://issues.apache.org/jira/browse/SPARK-21955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] poseidon updated SPARK-21955: - Description: just in my way to know how stream , chunk , block works in netty found some nasty case. p

[jira] [Comment Edited] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160550#comment-16160550 ] xinzhang edited comment on SPARK-21067 at 9/11/17 2:02 AM: --- [~d

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160550#comment-16160550 ] xinzhang commented on SPARK-21067: -- [~dricard] Thanks for your reply. So do we . Use th

[jira] [Assigned] (SPARK-21971) Too many open files in Spark due to concurrent files being opened

[ https://issues.apache.org/jira/browse/SPARK-21971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21971: Assignee: (was: Apache Spark) > Too many open files in Spark due to concurrent files b

[jira] [Assigned] (SPARK-21971) Too many open files in Spark due to concurrent files being opened

[ https://issues.apache.org/jira/browse/SPARK-21971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21971: Assignee: Apache Spark > Too many open files in Spark due to concurrent files being opened

[jira] [Commented] (SPARK-21971) Too many open files in Spark due to concurrent files being opened

[ https://issues.apache.org/jira/browse/SPARK-21971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160547#comment-16160547 ] Apache Spark commented on SPARK-21971: -- User 'rajeshbalamohan' has created a pull re

[jira] [Assigned] (SPARK-20098) DataType's typeName method returns with 'StructF' in case of StructField

[ https://issues.apache.org/jira/browse/SPARK-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-20098: --- Assignee: Peter Szalai > DataType's typeName method returns with 'StructF' in case of Struct

[jira] [Created] (SPARK-21971) Too many open files in Spark due to concurrent files being opened

Rajesh Balamohan created SPARK-21971: Summary: Too many open files in Spark due to concurrent files being opened Key: SPARK-21971 URL: https://issues.apache.org/jira/browse/SPARK-21971 Project: Sp

[jira] [Commented] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

[ https://issues.apache.org/jira/browse/SPARK-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160491#comment-16160491 ] Apache Spark commented on SPARK-21960: -- User 'karth295' has created a pull request f

[jira] [Assigned] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

[ https://issues.apache.org/jira/browse/SPARK-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21960: Assignee: Apache Spark > Spark Streaming Dynamic Allocation should respect spark.executor.

[jira] [Assigned] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

[ https://issues.apache.org/jira/browse/SPARK-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21960: Assignee: (was: Apache Spark) > Spark Streaming Dynamic Allocation should respect spar

[jira] [Assigned] (SPARK-21970) Do a Project Wide Sweep for Redundant Throws Declarations

[ https://issues.apache.org/jira/browse/SPARK-21970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21970: Assignee: (was: Apache Spark) > Do a Project Wide Sweep for Redundant Throws Declarati

[jira] [Assigned] (SPARK-21970) Do a Project Wide Sweep for Redundant Throws Declarations

[ https://issues.apache.org/jira/browse/SPARK-21970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21970: Assignee: Apache Spark > Do a Project Wide Sweep for Redundant Throws Declarations > -

[jira] [Commented] (SPARK-21970) Do a Project Wide Sweep for Redundant Throws Declarations

[ https://issues.apache.org/jira/browse/SPARK-21970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160470#comment-16160470 ] Apache Spark commented on SPARK-21970: -- User 'original-brownbear' has created a pull

[jira] [Created] (SPARK-21970) Do a Project Wide Sweep for Redundant Throws Declarations

Armin Braun created SPARK-21970: --- Summary: Do a Project Wide Sweep for Redundant Throws Declarations Key: SPARK-21970 URL: https://issues.apache.org/jira/browse/SPARK-21970 Project: Spark Issue

[jira] [Updated] (SPARK-21969) CommandUtils.updateTableStats should call refreshTable

[ https://issues.apache.org/jira/browse/SPARK-21969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bogdan Raducanu updated SPARK-21969: Description: The table is cached so even though statistics are removed, they will still be

[jira] [Updated] (SPARK-21969) CommandUtils.updateTableStats should call refreshTable

[ https://issues.apache.org/jira/browse/SPARK-21969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bogdan Raducanu updated SPARK-21969: Description: The table is cached so even though statistics are removed, they will still be

[jira] [Created] (SPARK-21969) CommandUtils.updateTableStats should call refreshTable

Bogdan Raducanu created SPARK-21969: --- Summary: CommandUtils.updateTableStats should call refreshTable Key: SPARK-21969 URL: https://issues.apache.org/jira/browse/SPARK-21969 Project: Spark

[jira] [Created] (SPARK-21968) Improved KernelDensity support

Brian created SPARK-21968: - Summary: Improved KernelDensity support Key: SPARK-21968 URL: https://issues.apache.org/jira/browse/SPARK-21968 Project: Spark Issue Type: Improvement Components

[jira] [Commented] (SPARK-20684) expose createOrReplaceGlobalTempView/createGlobalTempView and dropGlobalTempView in SparkR

[ https://issues.apache.org/jira/browse/SPARK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160420#comment-16160420 ] Felix Cheung commented on SPARK-20684: -- I"m making this primary JIRA for tracking th

[jira] [Updated] (SPARK-20684) expose createOrReplaceGlobalTempView/createGlobalTempView and dropGlobalTempView in SparkR

[ https://issues.apache.org/jira/browse/SPARK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20684: - Summary: expose createOrReplaceGlobalTempView/createGlobalTempView and dropGlobalTempView in Spar

[jira] [Reopened] (SPARK-20684) expose createGlobalTempView and dropGlobalTempView in SparkR

[ https://issues.apache.org/jira/browse/SPARK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reopened SPARK-20684: -- > expose createGlobalTempView and dropGlobalTempView in SparkR > --

[jira] [Commented] (SPARK-18591) Replace hash-based aggregates with sort-based ones if inputs already sorted

[ https://issues.apache.org/jira/browse/SPARK-18591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160388#comment-16160388 ] Takeshi Yamamuro commented on SPARK-18591: -- I just kindly give a head-up for the

[jira] [Assigned] (SPARK-21907) NullPointerException in UnsafeExternalSorter.spill()

[ https://issues.apache.org/jira/browse/SPARK-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21907: Assignee: Apache Spark > NullPointerException in UnsafeExternalSorter.spill() > --

[jira] [Assigned] (SPARK-21907) NullPointerException in UnsafeExternalSorter.spill()

[ https://issues.apache.org/jira/browse/SPARK-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21907: Assignee: (was: Apache Spark) > NullPointerException in UnsafeExternalSorter.spill() >

[jira] [Commented] (SPARK-21907) NullPointerException in UnsafeExternalSorter.spill()

[ https://issues.apache.org/jira/browse/SPARK-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160364#comment-16160364 ] Apache Spark commented on SPARK-21907: -- User 'eyalfa' has created a pull request for

[jira] [Commented] (SPARK-21907) NullPointerException in UnsafeExternalSorter.spill()

[ https://issues.apache.org/jira/browse/SPARK-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160366#comment-16160366 ] Eyal Farago commented on SPARK-21907: - opened PR: https://github.com/apache/spark/pul

[jira] [Assigned] (SPARK-21967) org.apache.spark.unsafe.types.UTF8String#compareTo Should Compare 8 Bytes at a Time for Better Performance

[ https://issues.apache.org/jira/browse/SPARK-21967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21967: Assignee: (was: Apache Spark) > org.apache.spark.unsafe.types.UTF8String#compareTo Sho

[jira] [Commented] (SPARK-21967) org.apache.spark.unsafe.types.UTF8String#compareTo Should Compare 8 Bytes at a Time for Better Performance

[ https://issues.apache.org/jira/browse/SPARK-21967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160338#comment-16160338 ] Apache Spark commented on SPARK-21967: -- User 'original-brownbear' has created a pull

[jira] [Assigned] (SPARK-21967) org.apache.spark.unsafe.types.UTF8String#compareTo Should Compare 8 Bytes at a Time for Better Performance

[ https://issues.apache.org/jira/browse/SPARK-21967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21967: Assignee: Apache Spark > org.apache.spark.unsafe.types.UTF8String#compareTo Should Compare

[jira] [Created] (SPARK-21967) org.apache.spark.unsafe.types.UTF8String#compareTo Should Compare 8 Bytes at a Time for Better Performance

Armin Braun created SPARK-21967: --- Summary: org.apache.spark.unsafe.types.UTF8String#compareTo Should Compare 8 Bytes at a Time for Better Performance Key: SPARK-21967 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Assigned] (SPARK-21966) ResolveMissingReference rule should not ignore the Union operator

[ https://issues.apache.org/jira/browse/SPARK-21966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21966: Assignee: (was: Apache Spark) > ResolveMissingReference rule should not ignore the Uni

[jira] [Commented] (SPARK-21966) ResolveMissingReference rule should not ignore the Union operator

[ https://issues.apache.org/jira/browse/SPARK-21966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160279#comment-16160279 ] Apache Spark commented on SPARK-21966: -- User 'DonnyZone' has created a pull request

[jira] [Assigned] (SPARK-21966) ResolveMissingReference rule should not ignore the Union operator

[ https://issues.apache.org/jira/browse/SPARK-21966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21966: Assignee: Apache Spark > ResolveMissingReference rule should not ignore the Union operator

[jira] [Created] (SPARK-21966) ResolveMissingReference rule should not ignore the Union operator

Feng Zhu created SPARK-21966: Summary: ResolveMissingReference rule should not ignore the Union operator Key: SPARK-21966 URL: https://issues.apache.org/jira/browse/SPARK-21966 Project: Spark Is

[jira] [Closed] (SPARK-21965) Add createOrReplaceGlobalTempView and dropGlobalTempView for SparkR

[ https://issues.apache.org/jira/browse/SPARK-21965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang closed SPARK-21965. --- Resolution: Duplicate > Add createOrReplaceGlobalTempView and dropGlobalTempView for SparkR > ---

[jira] [Commented] (SPARK-20098) DataType's typeName method returns with 'StructF' in case of StructField

[ https://issues.apache.org/jira/browse/SPARK-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160276#comment-16160276 ] Hyukjin Kwon commented on SPARK-20098: -- [~jerryshao], I am sorry for asking this aga

[jira] [Resolved] (SPARK-20098) DataType's typeName method returns with 'StructF' in case of StructField

[ https://issues.apache.org/jira/browse/SPARK-20098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-20098. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by pull r