[jira] [Closed] (SPARK-15795) Enable more optimizations in whole stage codegen when isNull is a compile-time constant

[ https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hiroshi Inoue closed SPARK-15795. - Resolution: Invalid Since this implementation is hacky and we need other solutions. > Enable mor

[jira] [Updated] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14220: Target Version/s: (was: 2.3.0) > Build and test Spark against Scala 2.12 > --

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060469#comment-16060469 ] Reynold Xin commented on SPARK-14220: - I just removed the target version given the am

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060468#comment-16060468 ] Vaclav Svejcar commented on SPARK-14220: Hi, I understand it's not an easy task t

[jira] [Commented] (SPARK-21076) R dapply doesn't return array or raw columns when array have different length

[ https://issues.apache.org/jira/browse/SPARK-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060460#comment-16060460 ] Xu Yang commented on SPARK-21076: - i get test1 from a sparkDataFrame created by scala sch

[jira] [Commented] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060455#comment-16060455 ] Reynold Xin commented on SPARK-21187: - Does Pandas support array / struct / map? >

[jira] [Updated] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13534: Issue Type: Sub-task (was: New Feature) Parent: SPARK-21187 > Implement Apache Arrow seria

[jira] [Commented] (SPARK-21159) Cluster mode, driver throws connection refused exception submitted by SparkLauncher

[ https://issues.apache.org/jira/browse/SPARK-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060453#comment-16060453 ] niefei commented on SPARK-21159: Great, thank you. > Cluster mode, driver throws connect

[jira] [Updated] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-21187: - Description: This is to track adding the remaining type support in Arrow Converters. Currently,

[jira] [Updated] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-21187: - Description: This is to track adding the remaining type support in Arrow Converters. Currently,

[jira] [Updated] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-21187: - Description: This is to track adding the remaining type support in Arrow Converters. Currently,

[jira] [Assigned] (SPARK-21188) releaseAllLocksForTask should synchronize the whole method

[ https://issues.apache.org/jira/browse/SPARK-21188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21188: Assignee: (was: Apache Spark) > releaseAllLocksForTask should synchronize the whole me

[jira] [Commented] (SPARK-21188) releaseAllLocksForTask should synchronize the whole method

[ https://issues.apache.org/jira/browse/SPARK-21188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060422#comment-16060422 ] Apache Spark commented on SPARK-21188: -- User 'liufengdb' has created a pull request

[jira] [Assigned] (SPARK-21188) releaseAllLocksForTask should synchronize the whole method

[ https://issues.apache.org/jira/browse/SPARK-21188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21188: Assignee: Apache Spark > releaseAllLocksForTask should synchronize the whole method >

[jira] [Updated] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-21187: - Summary: Complete support for remaining Spark data types in Arrow Converters (was: Complete supp

[jira] [Created] (SPARK-21188) releaseAllLocksForTask should synchronize the whole method

Feng Liu created SPARK-21188: Summary: releaseAllLocksForTask should synchronize the whole method Key: SPARK-21188 URL: https://issues.apache.org/jira/browse/SPARK-21188 Project: Spark Issue Type

[jira] [Commented] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060413#comment-16060413 ] Bryan Cutler commented on SPARK-13534: -- That is correct [~rxin], this did not have s

[jira] [Closed] (SPARK-20738) Option to turn off building docs in sbt build.

[ https://issues.apache.org/jira/browse/SPARK-20738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma closed SPARK-20738. --- Resolution: Won't Fix > Option to turn off building docs in sbt build. >

[jira] [Created] (SPARK-21187) Complete support for remaining Spark data type in Arrow Converters

Bryan Cutler created SPARK-21187: Summary: Complete support for remaining Spark data type in Arrow Converters Key: SPARK-21187 URL: https://issues.apache.org/jira/browse/SPARK-21187 Project: Spark

[jira] [Commented] (SPARK-21066) LibSVM load just one input file

[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060381#comment-16060381 ] Yan Facai (颜发才) commented on SPARK-21066: - Downgrade to Trivial since `numFeature

[jira] [Updated] (SPARK-21066) LibSVM load just one input file

[ https://issues.apache.org/jira/browse/SPARK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Facai (颜发才) updated SPARK-21066: Priority: Trivial (was: Major) > LibSVM load just one input file > ---

[jira] [Updated] (SPARK-21186) PySpark with --packages fails to import library due to lack of pythonpath to .ivy2/jars/*.jar

[ https://issues.apache.org/jira/browse/SPARK-21186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HanCheol Cho updated SPARK-21186: - Description: I experienced "ImportError: No module named sparkdl" exception while trying to use

[jira] [Created] (SPARK-21186) PySpark with --packages fails to import library due to lack of pythonpath to .ivy2/jars/*.jar

HanCheol Cho created SPARK-21186: Summary: PySpark with --packages fails to import library due to lack of pythonpath to .ivy2/jars/*.jar Key: SPARK-21186 URL: https://issues.apache.org/jira/browse/SPARK-21186

[jira] [Commented] (SPARK-21171) Speculate task scheduling block dirve handle normal task when a job task number more than one hundred thousand

[ https://issues.apache.org/jira/browse/SPARK-21171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060348#comment-16060348 ] wangminfeng commented on SPARK-21171: - We have modified some code for this feature, i

[jira] [Commented] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060328#comment-16060328 ] Reynold Xin commented on SPARK-13534: - Was this done? I thought there are still other

[jira] [Assigned] (SPARK-20599) ConsoleSink should work with write (batch)

[ https://issues.apache.org/jira/browse/SPARK-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-20599: Assignee: Lubo Zhang > ConsoleSink should work with write (batch) > --

[jira] [Commented] (SPARK-21185) Spurious errors in unidoc causing PRs to fail

[ https://issues.apache.org/jira/browse/SPARK-21185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060296#comment-16060296 ] Hyukjin Kwon commented on SPARK-21185: -- Yea, I believe this is a duplicate of https

[jira] [Assigned] (SPARK-21174) Validate sampling fraction in logical operator level

[ https://issues.apache.org/jira/browse/SPARK-21174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21174: --- Assignee: Gengliang Wang > Validate sampling fraction in logical operator level > --

[jira] [Resolved] (SPARK-21174) Validate sampling fraction in logical operator level

[ https://issues.apache.org/jira/browse/SPARK-21174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21174. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18387 [https://githu

[jira] [Commented] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060283#comment-16060283 ] Wenchen Fan commented on SPARK-20923: - This patch changes the public behavior and we

[jira] [Resolved] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20923. - Resolution: Fixed Fix Version/s: 2.3.0 > TaskMetrics._updatedBlockStatuses uses a lot of m

[jira] [Updated] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20923: Labels: releasenotes (was: ) > TaskMetrics._updatedBlockStatuses uses a lot of memory > --

[jira] [Assigned] (SPARK-20923) TaskMetrics._updatedBlockStatuses uses a lot of memory

[ https://issues.apache.org/jira/browse/SPARK-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20923: --- Assignee: Thomas Graves > TaskMetrics._updatedBlockStatuses uses a lot of memory > -

[jira] [Updated] (SPARK-21185) Spurious errors in unidoc causing PRs to fail

[ https://issues.apache.org/jira/browse/SPARK-21185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-21185: -- Description: Some PRs are failing because of unidoc throwing random errors. When GenJavaDoc ge

[jira] [Created] (SPARK-21185) Spurious errors in unidoc causing PRs to fail

Tathagata Das created SPARK-21185: - Summary: Spurious errors in unidoc causing PRs to fail Key: SPARK-21185 URL: https://issues.apache.org/jira/browse/SPARK-21185 Project: Spark Issue Type: B

[jira] [Assigned] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-13534: --- Assignee: Bryan Cutler > Implement Apache Arrow serializer for Spark DataFrame for use in >

[jira] [Resolved] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-13534. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 15821 [https://githu

[jira] [Commented] (SPARK-21159) Cluster mode, driver throws connection refused exception submitted by SparkLauncher

[ https://issues.apache.org/jira/browse/SPARK-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060208#comment-16060208 ] Apache Spark commented on SPARK-21159: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-21159) Cluster mode, driver throws connection refused exception submitted by SparkLauncher

[ https://issues.apache.org/jira/browse/SPARK-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21159: Assignee: (was: Apache Spark) > Cluster mode, driver throws connection refused excepti

[jira] [Assigned] (SPARK-21159) Cluster mode, driver throws connection refused exception submitted by SparkLauncher

[ https://issues.apache.org/jira/browse/SPARK-21159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21159: Assignee: Apache Spark > Cluster mode, driver throws connection refused exception submitte

[jira] [Commented] (SPARK-18343) FileSystem$Statistics$StatisticsDataReferenceCleaner hangs on s3 write

[ https://issues.apache.org/jira/browse/SPARK-18343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060203#comment-16060203 ] Kanagha Pradha commented on SPARK-18343: I am getting the same error with spark 2

[jira] [Commented] (SPARK-21145) Restarted queries reuse same StateStoreProvider, causing multiple concurrent tasks to update same StateStore

[ https://issues.apache.org/jira/browse/SPARK-21145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060196#comment-16060196 ] Apache Spark commented on SPARK-21145: -- User 'tdas' has created a pull request for t

[jira] [Commented] (SPARK-20391) Properly rename the memory related fields in ExecutorSummary REST API

[ https://issues.apache.org/jira/browse/SPARK-20391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060097#comment-16060097 ] Jose Soltren commented on SPARK-20391: -- So, this is months old now and irrelevant, b

[jira] [Commented] (SPARK-20655) In-memory key-value store implementation

[ https://issues.apache.org/jira/browse/SPARK-20655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060067#comment-16060067 ] Apache Spark commented on SPARK-20655: -- User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-20379) Allow setting SSL-related passwords through env variables

[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060065#comment-16060065 ] Apache Spark commented on SPARK-20379: -- User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

[ https://issues.apache.org/jira/browse/SPARK-20342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060063#comment-16060063 ] Apache Spark commented on SPARK-20342: -- User 'vanzin' has created a pull request for

[jira] [Resolved] (SPARK-19937) Collect metrics of block sizes when shuffle.

[ https://issues.apache.org/jira/browse/SPARK-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19937. Resolution: Fixed Assignee: jin xing Fix Version/s: 2.3.0 > Collect metrics

[jira] [Created] (SPARK-21184) QuantileSummaries implementation is wrong and QuantileSummariesSuite fails with larger n

Andrew Ray created SPARK-21184: -- Summary: QuantileSummaries implementation is wrong and QuantileSummariesSuite fails with larger n Key: SPARK-21184 URL: https://issues.apache.org/jira/browse/SPARK-21184

[jira] [Created] (SPARK-21183) Unable to return Google BigQuery INTEGER data type into Spark via google BigQuery JDBC driver: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value to

Matthew Walton created SPARK-21183: -- Summary: Unable to return Google BigQuery INTEGER data type into Spark via google BigQuery JDBC driver: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value to long. Key: SP

[jira] [Created] (SPARK-21182) Structured streaming on Spark-shell on windows

Vijay created SPARK-21182: - Summary: Structured streaming on Spark-shell on windows Key: SPARK-21182 URL: https://issues.apache.org/jira/browse/SPARK-21182 Project: Spark Issue Type: Bug Co

[jira] [Updated] (SPARK-21179) Unable to return Hive INT data type into Spark via Hive JDBC driver: Caused by: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value to int.

[ https://issues.apache.org/jira/browse/SPARK-21179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Walton updated SPARK-21179: --- Affects Version/s: 2.1.1 Summary: Unable to return Hive INT data type into Spark

[jira] [Updated] (SPARK-21168) KafkaRDD should always set kafka clientId.

[ https://issues.apache.org/jira/browse/SPARK-21168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21168: - Component/s: (was: Structured Streaming) DStreams > KafkaRDD should always s

[jira] [Commented] (SPARK-21167) Path is not decoded correctly when reading output of FileSink

[ https://issues.apache.org/jira/browse/SPARK-21167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059816#comment-16059816 ] Apache Spark commented on SPARK-21167: -- User 'dijingran' has created a pull request

[jira] [Issue Comment Deleted] (SPARK-21167) Path is not decoded correctly when reading output of FileSink

[ https://issues.apache.org/jira/browse/SPARK-21167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21167: - Comment: was deleted (was: User 'dijingran' has created a pull request for this issue: https://gi

[jira] [Resolved] (SPARK-20599) ConsoleSink should work with write (batch)

[ https://issues.apache.org/jira/browse/SPARK-20599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-20599. -- Resolution: Fixed Fix Version/s: 2.3.0 > ConsoleSink should work with write (batch) > --

[jira] [Updated] (SPARK-21155) Add (? running tasks) into Spark UI progress

[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21155: Attachment: Screen Shot 2017-06-22 at 9.58.08 AM.png > Add (? running tasks) into Spark UI

[jira] [Assigned] (SPARK-21181) Suppress memory leak errors reported by netty

[ https://issues.apache.org/jira/browse/SPARK-21181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21181: Assignee: (was: Apache Spark) > Suppress memory leak errors reported by netty > --

[jira] [Commented] (SPARK-21181) Suppress memory leak errors reported by netty

[ https://issues.apache.org/jira/browse/SPARK-21181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059671#comment-16059671 ] Apache Spark commented on SPARK-21181: -- User 'dhruve' has created a pull request for

[jira] [Assigned] (SPARK-21181) Suppress memory leak errors reported by netty

[ https://issues.apache.org/jira/browse/SPARK-21181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21181: Assignee: Apache Spark > Suppress memory leak errors reported by netty > -

[jira] [Created] (SPARK-21181) Suppress memory leak errors reported by netty

Dhruve Ashar created SPARK-21181: Summary: Suppress memory leak errors reported by netty Key: SPARK-21181 URL: https://issues.apache.org/jira/browse/SPARK-21181 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21110) Structs should be usable in inequality filters

[ https://issues.apache.org/jira/browse/SPARK-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059536#comment-16059536 ] Nicholas Chammas commented on SPARK-21110: -- cc [~marmbrus] - Assuming this is a

[jira] [Commented] (SPARK-21180) Remove conf from stats functions since now we have conf in LogicalPlan

[ https://issues.apache.org/jira/browse/SPARK-21180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059525#comment-16059525 ] Apache Spark commented on SPARK-21180: -- User 'wzhfy' has created a pull request for

[jira] [Assigned] (SPARK-21180) Remove conf from stats functions since now we have conf in LogicalPlan

[ https://issues.apache.org/jira/browse/SPARK-21180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21180: Assignee: (was: Apache Spark) > Remove conf from stats functions since now we have con

[jira] [Assigned] (SPARK-21180) Remove conf from stats functions since now we have conf in LogicalPlan

[ https://issues.apache.org/jira/browse/SPARK-21180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21180: Assignee: Apache Spark > Remove conf from stats functions since now we have conf in Logica

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Description: In reverse proxy mode, Sparks exhausts the Jetty thread pool if the master node h

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Description: In reverse proxy mode, Sparks exhausts the Jetty thread pool if the master node h

[jira] [Created] (SPARK-21180) Remove conf from stats functions since now we have conf in LogicalPlan

Zhenhua Wang created SPARK-21180: Summary: Remove conf from stats functions since now we have conf in LogicalPlan Key: SPARK-21180 URL: https://issues.apache.org/jira/browse/SPARK-21180 Project: Spark

[jira] [Commented] (SPARK-21166) Automated ML persistence

[ https://issues.apache.org/jira/browse/SPARK-21166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059481#comment-16059481 ] Mark Hamilton commented on SPARK-21166: --- The code is currently being developed here

[jira] [Created] (SPARK-21179) Unable to return Hive INT data type into Spark SQL via Hive JDBC driver: Caused by: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value to int.

Matthew Walton created SPARK-21179: -- Summary: Unable to return Hive INT data type into Spark SQL via Hive JDBC driver: Caused by: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value to int. Key: SPARK-21179

[jira] [Resolved] (SPARK-20832) Standalone master should explicitly inform drivers of worker deaths and invalidate external shuffle service outputs

[ https://issues.apache.org/jira/browse/SPARK-20832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20832. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18362 [https://githu

[jira] [Assigned] (SPARK-20832) Standalone master should explicitly inform drivers of worker deaths and invalidate external shuffle service outputs

[ https://issues.apache.org/jira/browse/SPARK-20832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20832: --- Assignee: Jiang Xingbo > Standalone master should explicitly inform drivers of worker deaths

[jira] [Commented] (SPARK-20295) when spark.sql.adaptive.enabled is enabled, have conflict with Exchange Resue

[ https://issues.apache.org/jira/browse/SPARK-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059273#comment-16059273 ] cen yuhai commented on SPARK-20295: --- I hit this bug... > when spark.sql.adaptive.enab

[jira] [Commented] (SPARK-21178) Add support for label specific metrics in MulticlassClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-21178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059251#comment-16059251 ] Apache Spark commented on SPARK-21178: -- User 'rawataaryan9' has created a pull reque

[jira] [Assigned] (SPARK-21178) Add support for label specific metrics in MulticlassClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-21178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21178: Assignee: Apache Spark > Add support for label specific metrics in MulticlassClassificatio

[jira] [Assigned] (SPARK-21178) Add support for label specific metrics in MulticlassClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-21178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21178: Assignee: (was: Apache Spark) > Add support for label specific metrics in MulticlassCl

[jira] [Updated] (SPARK-21178) Add support for label specific metrics in MulticlassClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-21178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Rawat updated SPARK-21178: --- Target Version/s: (was: 2.1.2) > Add support for label specific metrics in MulticlassClassification

[jira] [Created] (SPARK-21178) Add support for label specific metrics in MulticlassClassificationEvaluator

Aman Rawat created SPARK-21178: -- Summary: Add support for label specific metrics in MulticlassClassificationEvaluator Key: SPARK-21178 URL: https://issues.apache.org/jira/browse/SPARK-21178 Project: Spar

[jira] [Updated] (SPARK-21177) df.saveAsTable slows down linearly, with number of appends

[ https://issues.apache.org/jira/browse/SPARK-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-21177: Summary: df.saveAsTable slows down linearly, with number of appends (was: df.SaveAsTable s

[jira] [Updated] (SPARK-21177) df.SaveAsTable slows down linearly, with number of appends.

[ https://issues.apache.org/jira/browse/SPARK-21177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-21177: Summary: df.SaveAsTable slows down linearly, with number of appends. (was: Append to hive

[jira] [Created] (SPARK-21177) Append to hive slows down linearly, with number of appends.

Prashant Sharma created SPARK-21177: --- Summary: Append to hive slows down linearly, with number of appends. Key: SPARK-21177 URL: https://issues.apache.org/jira/browse/SPARK-21177 Project: Spark

[jira] [Commented] (SPARK-11373) Add metrics to the History Server and providers

[ https://issues.apache.org/jira/browse/SPARK-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059141#comment-16059141 ] Steve Loughran commented on SPARK-11373: metrics might help with understanding th

[jira] [Resolved] (SPARK-21080) Workaround for HDFS delegation token expiry broken with some Hadoop versions

[ https://issues.apache.org/jira/browse/SPARK-21080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21080. --- Resolution: Not A Problem > Workaround for HDFS delegation token expiry broken with some Hadoop versi

[jira] [Commented] (SPARK-21080) Workaround for HDFS delegation token expiry broken with some Hadoop versions

[ https://issues.apache.org/jira/browse/SPARK-21080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059022#comment-16059022 ] Lukasz Raszka commented on SPARK-21080: --- Update: I think it might be a mistake on o

[jira] [Commented] (SPARK-21167) Path is not decoded correctly when reading output of FileSink

[ https://issues.apache.org/jira/browse/SPARK-21167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059019#comment-16059019 ] Apache Spark commented on SPARK-21167: -- User 'dijingran' has created a pull request

[jira] [Commented] (SPARK-21171) Speculate task scheduling block dirve handle normal task when a job task number more than one hundred thousand

[ https://issues.apache.org/jira/browse/SPARK-21171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059018#comment-16059018 ] Sean Owen commented on SPARK-21171: --- There's no real detail here. I'd have to close thi

[jira] [Resolved] (SPARK-21161) SparkContext stopped when execute a query on Solr

[ https://issues.apache.org/jira/browse/SPARK-21161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21161. --- Resolution: Not A Problem > SparkContext stopped when execute a query on Solr > -

[jira] [Resolved] (SPARK-21173) There are several configuration about SSL displayed in configuration.md but never be used.

[ https://issues.apache.org/jira/browse/SPARK-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21173. --- Resolution: Not A Problem > There are several configuration about SSL displayed in configuration.md b

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Description: In reverse proxy mode, Sparks exhausts the Jetty thread pool if the master node h

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Description: In reverse proxy mode, Sparks exhausts the Jetty thread pool if the master node h

[jira] [Commented] (SPARK-21080) Workaround for HDFS delegation token expiry broken with some Hadoop versions

[ https://issues.apache.org/jira/browse/SPARK-21080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058985#comment-16058985 ] Sean Owen commented on SPARK-21080: --- This really isn't my area; maybe [~vanzin]? > Wor

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Affects Version/s: 2.2.1 2.2.0 2.1.1 > Master UI

[jira] [Created] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

Ingo Schuster created SPARK-21176: - Summary: Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs Key: SPARK-21176 URL: https://issues.apache.org/jira/browse/SPARK-21176 Pr

[jira] [Commented] (SPARK-14174) Implement the Mini-Batch KMeans

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058982#comment-16058982 ] zhengruifeng commented on SPARK-14174: -- [~mlnick] I send a new PR for MiniBatch KMea

[jira] [Commented] (SPARK-14174) Implement the Mini-Batch KMeans

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058975#comment-16058975 ] Apache Spark commented on SPARK-14174: -- User 'zhengruifeng' has created a pull reque

[jira] [Resolved] (SPARK-21163) DataFrame.toPandas should respect the data type

[ https://issues.apache.org/jira/browse/SPARK-21163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21163. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18378 [https://githu

[jira] [Updated] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14174: - Description: The MiniBatchKMeans is a variant of the KMeans algorithm which uses mini-batches to

[jira] [Updated] (SPARK-14174) Implement the Mini-Batch KMeans

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14174: - Summary: Implement the Mini-Batch KMeans (was: Accelerate KMeans via Mini-Batch EM) > Implement

[jira] [Updated] (SPARK-14174) Implement the Mini-Batch KMeans

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14174: - Attachment: MBKM.xlsx > Implement the Mini-Batch KMeans > --- > >

[jira] [Updated] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14174: - Attachment: (was: MiniBatchKMeans_Performance.pdf) > Accelerate KMeans via Mini-Batch EM > --

[jira] [Updated] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-14174: - Attachment: (was: MiniBatchKMeans_Performance_II.pdf) > Accelerate KMeans via Mini-Batch EM >

  1   2   >