[jira] [Commented] (SPARK-21122) Address starvation issues when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052729#comment-16052729 ] Sean Owen commented on SPARK-21122: --- Yes, resource managers don't know which resource y

[jira] [Commented] (SPARK-19490) Hive partition columns are case-sensitive

[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052712#comment-16052712 ] Taklon Stephen Wu commented on SPARK-19490: --- Spark 2.1.1 and you mentioned anot

[jira] [Resolved] (SPARK-21058) potential SVD optimization

[ https://issues.apache.org/jira/browse/SPARK-21058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21058. --- Resolution: Not A Problem If there's a specific optimization for large, sparse matrices to discuss, I

[jira] [Resolved] (SPARK-21120) Increasing the master's metric is conducive to the spark cluster management system monitoring.

[ https://issues.apache.org/jira/browse/SPARK-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21120. --- Resolution: Invalid > Increasing the master's metric is conducive to the spark cluster management >

[jira] [Resolved] (SPARK-21081) Throw specific IllegalStateException subtype when asserting that SparkContext not stopped

[ https://issues.apache.org/jira/browse/SPARK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21081. --- Resolution: Won't Fix > Throw specific IllegalStateException subtype when asserting that SparkContext

[jira] [Updated] (SPARK-21126) The configuration which named "spark.core.connection.auth.wait.timeout" hasn't been used in spark

[ https://issues.apache.org/jira/browse/SPARK-21126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21126: -- Priority: Trivial (was: Major) Component/s: Documentation [~liuzhaokun] please fill out JIRAs m

[jira] [Commented] (SPARK-19490) Hive partition columns are case-sensitive

[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052633#comment-16052633 ] cen yuhai commented on SPARK-19490: --- I don't know, what is your spark version? > Hive

[jira] [Assigned] (SPARK-21126) The configuration which named "spark.core.connection.auth.wait.timeout" hasn't been used in spark

[ https://issues.apache.org/jira/browse/SPARK-21126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21126: Assignee: (was: Apache Spark) > The configuration which named "spark.core.connection.a

[jira] [Commented] (SPARK-21126) The configuration which named "spark.core.connection.auth.wait.timeout" hasn't been used in spark

[ https://issues.apache.org/jira/browse/SPARK-21126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052626#comment-16052626 ] Apache Spark commented on SPARK-21126: -- User 'liu-zhaokun' has created a pull reques

[jira] [Assigned] (SPARK-21126) The configuration which named "spark.core.connection.auth.wait.timeout" hasn't been used in spark

[ https://issues.apache.org/jira/browse/SPARK-21126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21126: Assignee: Apache Spark > The configuration which named "spark.core.connection.auth.wait.ti

[jira] [Created] (SPARK-21126) The configuration which named "spark.core.connection.auth.wait.timeout" hasn't been used in spark

liuzhaokun created SPARK-21126: -- Summary: The configuration which named "spark.core.connection.auth.wait.timeout" hasn't been used in spark Key: SPARK-21126 URL: https://issues.apache.org/jira/browse/SPARK-21126

[jira] [Resolved] (SPARK-2971) Orphaned YARN ApplicationMaster lingers forever

[ https://issues.apache.org/jira/browse/SPARK-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-2971. --- Resolution: Not A Problem Pretty sure this has been fixed some point after 1.0; there's both t

[jira] [Resolved] (SPARK-6393) Extra RPC to the AM during killExecutor invocation

[ https://issues.apache.org/jira/browse/SPARK-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6393. --- Resolution: Won't Fix I read the PR again and I'm not really sure what this is talking about,

[jira] [Resolved] (SPARK-7252) Add support for creating new Hive and HBase delegation tokens

[ https://issues.apache.org/jira/browse/SPARK-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-7252. --- Resolution: Duplicate > Add support for creating new Hive and HBase delegation tokens > --

[jira] [Resolved] (SPARK-8929) [Windows] Application with Appname including whiteSpace fails in Yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-8929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-8929. --- Resolution: Unresolved This doesn't even have the Spark version that's affected... since this

[jira] [Updated] (SPARK-21125) PySpark context missing function to set Job Description.

[ https://issues.apache.org/jira/browse/SPARK-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Jarvie updated SPARK-21125: - Shepherd: Alex Bozarth > PySpark context missing function to set Job Description. > -

[jira] [Reopened] (SPARK-11058) failed spark job reports on YARN as successful

[ https://issues.apache.org/jira/browse/SPARK-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-11058: > failed spark job reports on YARN as successful >

[jira] [Resolved] (SPARK-11058) failed spark job reports on YARN as successful

[ https://issues.apache.org/jira/browse/SPARK-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11058. Resolution: Won't Fix > failed spark job reports on YARN as successful > --

[jira] [Resolved] (SPARK-11058) failed spark job reports on YARN as successful

[ https://issues.apache.org/jira/browse/SPARK-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11058. Resolution: Not A Problem This is a limitation of client mode in Spark. The YARN applicatio

[jira] [Updated] (SPARK-11170) ​ EOFException on History server reading in progress lz4

[ https://issues.apache.org/jira/browse/SPARK-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-11170: --- Component/s: (was: YARN) > ​ EOFException on History server reading in progress lz4 > ---

[jira] [Resolved] (SPARK-12279) Requesting a HBase table with kerberos is not working

[ https://issues.apache.org/jira/browse/SPARK-12279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-12279. Resolution: Cannot Reproduce I'm going to close this because I'm pretty sure this works (we

[jira] [Assigned] (SPARK-21125) PySpark context missing function to set Job Description.

[ https://issues.apache.org/jira/browse/SPARK-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21125: Assignee: (was: Apache Spark) > PySpark context missing function to set Job Descriptio

[jira] [Commented] (SPARK-21125) PySpark context missing function to set Job Description.

[ https://issues.apache.org/jira/browse/SPARK-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052516#comment-16052516 ] Apache Spark commented on SPARK-21125: -- User 'sjarvie' has created a pull request fo

[jira] [Assigned] (SPARK-21125) PySpark context missing function to set Job Description.

[ https://issues.apache.org/jira/browse/SPARK-21125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21125: Assignee: Apache Spark > PySpark context missing function to set Job Description. > --

[jira] [Created] (SPARK-21125) PySpark context missing function to set Job Description.

Shane Jarvie created SPARK-21125: Summary: PySpark context missing function to set Job Description. Key: SPARK-21125 URL: https://issues.apache.org/jira/browse/SPARK-21125 Project: Spark Issu

[jira] [Resolved] (SPARK-20458) support getting Yarn Tracking URL in code

[ https://issues.apache.org/jira/browse/SPARK-20458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20458. Resolution: Won't Fix > support getting Yarn Tracking URL in code > ---

[jira] [Commented] (SPARK-20458) support getting Yarn Tracking URL in code

[ https://issues.apache.org/jira/browse/SPARK-20458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052504#comment-16052504 ] Marcelo Vanzin commented on SPARK-20458: You can do this by reading the {{yarn.re

[jira] [Commented] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052441#comment-16052441 ] Apache Spark commented on SPARK-21124: -- User 'vanzin' has created a pull request for

[jira] [Assigned] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21124: Assignee: Apache Spark > Wrong user shown in UI when using kerberos >

[jira] [Assigned] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21124: Assignee: (was: Apache Spark) > Wrong user shown in UI when using kerberos > -

[jira] [Comment Edited] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052409#comment-16052409 ] Marcelo Vanzin edited comment on SPARK-21124 at 6/16/17 9:35 PM: --

[jira] [Commented] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052409#comment-16052409 ] Marcelo Vanzin commented on SPARK-21124: Also, the SPARK-14483 also masks which i

[jira] [Comment Edited] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052409#comment-16052409 ] Marcelo Vanzin edited comment on SPARK-21124 at 6/16/17 9:21 PM: --

[jira] [Commented] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052404#comment-16052404 ] Marcelo Vanzin commented on SPARK-21124: Hmm, yeah, it seems like that code would

[jira] [Commented] (SPARK-21124) Wrong user shown in UI when using kerberos

[ https://issues.apache.org/jira/browse/SPARK-21124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052394#comment-16052394 ] Alex Bozarth commented on SPARK-21124: -- Though a separate issue, I believe the open

[jira] [Created] (SPARK-21124) Wrong user shown in UI when using kerberos

Marcelo Vanzin created SPARK-21124: -- Summary: Wrong user shown in UI when using kerberos Key: SPARK-21124 URL: https://issues.apache.org/jira/browse/SPARK-21124 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-21122) Address starvation issues when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052362#comment-16052362 ] Craig Ingram commented on SPARK-21122: -- Thanks for the feedback, Sean. This ticket s

[jira] [Commented] (SPARK-15689) Data source API v2

[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052321#comment-16052321 ] Russell Spitzer commented on SPARK-15689: - I've been trying to work with making C

[jira] [Commented] (SPARK-21065) Spark Streaming concurrentJobs + StreamingJobProgressListener conflict

[ https://issues.apache.org/jira/browse/SPARK-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052242#comment-16052242 ] Shixiong Zhu commented on SPARK-21065: -- Please don't use `spark.streaming.concurrent

[jira] [Resolved] (SPARK-21065) Spark Streaming concurrentJobs + StreamingJobProgressListener conflict

[ https://issues.apache.org/jira/browse/SPARK-21065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21065. -- Resolution: Won't Fix > Spark Streaming concurrentJobs + StreamingJobProgressListener conflict

[jira] [Updated] (SPARK-21123) Options for file stream source are in a wrong table

[ https://issues.apache.org/jira/browse/SPARK-21123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21123: - Labels: starter (was: ) > Options for file stream source are in a wrong table >

[jira] [Updated] (SPARK-21123) Options for file stream source are in a wrong table

[ https://issues.apache.org/jira/browse/SPARK-21123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21123: - Affects Version/s: 2.2.0 > Options for file stream source are in a wrong table >

[jira] [Updated] (SPARK-21123) Options for file stream source are in a wrong table

[ https://issues.apache.org/jira/browse/SPARK-21123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21123: - Description: Right now options for file stream source are documented with file sink. We should c

[jira] [Updated] (SPARK-21123) Options for file stream source are in a wrong table

[ https://issues.apache.org/jira/browse/SPARK-21123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21123: - Description: !Screen Shot 2017-06-15 at 11.25.49 AM.png|thumbnail! was:!attachment-name.jpg|

[jira] [Updated] (SPARK-21123) Options for file stream source are in a wrong table

[ https://issues.apache.org/jira/browse/SPARK-21123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21123: - Description: !attachment-name.jpg|thumbnail! > Options for file stream source are in a wrong tabl

[jira] [Created] (SPARK-21123) Options for file stream source are in a wrong table

Shixiong Zhu created SPARK-21123: Summary: Options for file stream source are in a wrong table Key: SPARK-21123 URL: https://issues.apache.org/jira/browse/SPARK-21123 Project: Spark Issue Typ

[jira] [Updated] (SPARK-21123) Options for file stream source are in a wrong table

[ https://issues.apache.org/jira/browse/SPARK-21123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21123: - Attachment: Screen Shot 2017-06-15 at 11.25.49 AM.png > Options for file stream source are in a w

[jira] [Commented] (SPARK-21122) Address starvation issues when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-21122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052199#comment-16052199 ] Sean Owen commented on SPARK-21122: --- How is this different from https://issues.apache.o

[jira] [Resolved] (SPARK-20338) Spaces in spark.eventLog.dir are not correctly handled

[ https://issues.apache.org/jira/browse/SPARK-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20338. Resolution: Fixed Assignee: zuotingbing Fix Version/s: 2.3.0 > Spaces in sp

[jira] [Created] (SPARK-21122) Address starvation issues when dynamic allocation is enabled

Craig Ingram created SPARK-21122: Summary: Address starvation issues when dynamic allocation is enabled Key: SPARK-21122 URL: https://issues.apache.org/jira/browse/SPARK-21122 Project: Spark

[jira] [Resolved] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21089. - Resolution: Fixed > Table properties are not shown in DESC EXTENDED/FORMATTED > -

[jira] [Updated] (SPARK-21089) Table properties are not shown in DESC EXTENDED/FORMATTED

[ https://issues.apache.org/jira/browse/SPARK-21089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21089: Fix Version/s: 2.2.0 > Table properties are not shown in DESC EXTENDED/FORMATTED >

[jira] [Comment Edited] (SPARK-20977) NPE in CollectionAccumulator

[ https://issues.apache.org/jira/browse/SPARK-20977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052104#comment-16052104 ] Leonardo Zanivan edited comment on SPARK-20977 at 6/16/17 4:35 PM:

[jira] [Commented] (SPARK-20977) NPE in CollectionAccumulator

[ https://issues.apache.org/jira/browse/SPARK-20977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052104#comment-16052104 ] Leonardo Zanivan commented on SPARK-20977: -- I have the exactly same issue Spark

[jira] [Commented] (SPARK-20749) Built-in SQL Function Support - all variants of LEN[GTH]

[ https://issues.apache.org/jira/browse/SPARK-20749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052058#comment-16052058 ] Apache Spark commented on SPARK-20749: -- User 'wangyum' has created a pull request fo

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052057#comment-16052057 ] Wenchen Fan commented on SPARK-18016: - cc [~aeskilson] do you wanna send a new PR to

[jira] [Resolved] (SPARK-21119) unset table properties should keep the table comment

[ https://issues.apache.org/jira/browse/SPARK-21119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21119. - Resolution: Fixed Fix Version/s: 2.3.0 > unset table properties should keep the table comment > --

[jira] [Commented] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051981#comment-16051981 ] Apache Spark commented on SPARK-19909: -- User 'mgaido91' has created a pull request f

[jira] [Commented] (SPARK-19909) Batches will fail in case that temporary checkpoint dir is on local file system while metadata dir is on HDFS

[ https://issues.apache.org/jira/browse/SPARK-19909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051963#comment-16051963 ] Marco Gaido commented on SPARK-19909: - IMHO the best option to deal with this problem

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051950#comment-16051950 ] Divya commented on SPARK-18016: --- Is a backport fix available for 2.1.0? > Code Generation:

[jira] [Commented] (SPARK-20760) Memory Leak of RDD blocks

[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051942#comment-16051942 ] Jose Soltren commented on SPARK-20760: -- Hello Ravi - current thinking is that this i

[jira] [Assigned] (SPARK-21121) Set up StorageLevel for CACHE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-21121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21121: Assignee: (was: Apache Spark) > Set up StorageLevel for CACHE TABLE command >

[jira] [Commented] (SPARK-21121) Set up StorageLevel for CACHE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-21121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051910#comment-16051910 ] Apache Spark commented on SPARK-21121: -- User 'dosoft' has created a pull request for

[jira] [Assigned] (SPARK-21121) Set up StorageLevel for CACHE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-21121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21121: Assignee: Apache Spark > Set up StorageLevel for CACHE TABLE command > ---

[jira] [Updated] (SPARK-11083) insert overwrite table failed when beeline reconnect

[ https://issues.apache.org/jira/browse/SPARK-11083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11083: -- Fix Version/s: (was: 1.6.0) > insert overwrite table failed when beeline reconnect > --

[jira] [Updated] (SPARK-21054) Reset Command support reset specific property which is compatible with Hive

[ https://issues.apache.org/jira/browse/SPARK-21054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21054: -- Target Version/s: (was: 2.1.1) > Reset Command support reset specific property which is compatible wi

[jira] [Updated] (SPARK-21054) Reset Command support reset specific property which is compatible with Hive

[ https://issues.apache.org/jira/browse/SPARK-21054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21054: -- Fix Version/s: (was: 2.1.1) > Reset Command support reset specific property which is compatible wit

[jira] [Updated] (SPARK-21121) Set up StorageLevel for CACHE TABLE command

[ https://issues.apache.org/jira/browse/SPARK-21121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Danilov updated SPARK-21121: - Description: Currently, "CACHE TABLE" always uses the default MEMORY_AND_DISK storage level. We

[jira] [Created] (SPARK-21121) Set up StorageLevel for CACHE TABLE command

Oleg Danilov created SPARK-21121: Summary: Set up StorageLevel for CACHE TABLE command Key: SPARK-21121 URL: https://issues.apache.org/jira/browse/SPARK-21121 Project: Spark Issue Type: New F

[jira] [Commented] (SPARK-20760) Memory Leak of RDD blocks

[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051897#comment-16051897 ] Ravi Teja commented on SPARK-20760: --- We are also facing similar issue but in spark stre

[jira] [Commented] (SPARK-20984) Reading back from ORC format gives error on big endian systems.

[ https://issues.apache.org/jira/browse/SPARK-20984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051878#comment-16051878 ] Adam Roberts commented on SPARK-20984: -- For the Spark binary package we ship in IBM,

[jira] [Commented] (SPARK-20984) Reading back from ORC format gives error on big endian systems.

[ https://issues.apache.org/jira/browse/SPARK-20984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051861#comment-16051861 ] Mahesh commented on SPARK-20984: [~aroberts] Mentioned that snappy 0.2 version needs to b

[jira] [Comment Edited] (SPARK-18649) sc.textFile(my_file).collect() raises socket.timeout on large files

[ https://issues.apache.org/jira/browse/SPARK-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050940#comment-16050940 ] Serge Vilvovsky edited comment on SPARK-18649 at 6/16/17 12:13 PM:

[jira] [Assigned] (SPARK-20994) Alleviate memory pressure in StreamManager

[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20994: --- Assignee: jin xing > Alleviate memory pressure in StreamManager > --

[jira] [Resolved] (SPARK-20994) Alleviate memory pressure in StreamManager

[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20994. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18231 [https://githu

[jira] [Commented] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051769#comment-16051769 ] jin xing commented on SPARK-21047: -- [~kiszk] Would you mind if I make a try for this JIR

[jira] [Assigned] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21047: Assignee: (was: Apache Spark) > Add test suites for complicated cases in ColumnarBatch

[jira] [Commented] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051768#comment-16051768 ] Apache Spark commented on SPARK-21047: -- User 'jinxing64' has created a pull request

[jira] [Assigned] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21047: Assignee: Apache Spark > Add test suites for complicated cases in ColumnarBatchSuite > ---

[jira] [Commented] (SPARK-21118) OOM with 2 handred million vertex when mitrx multply

[ https://issues.apache.org/jira/browse/SPARK-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051766#comment-16051766 ] tao commented on SPARK-21118: - ok,thanks > OOM with 2 handred million vertex when mitrx mul

[jira] [Commented] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051726#comment-16051726 ] Ilyes Hachani commented on SPARK-12606: --- [~srowen] Can I rewrite it ? What does set

[jira] [Resolved] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12606. --- Resolution: Invalid This wasn't really a well-formed JIRA to begin with. [~ihachani] I don't think th

[jira] [Commented] (SPARK-21120) Increasing the master's metric is conducive to the spark cluster management system monitoring.

[ https://issues.apache.org/jira/browse/SPARK-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051699#comment-16051699 ] Sean Owen commented on SPARK-21120: --- There's no detail about the motivation here. Pleas

[jira] [Comment Edited] (SPARK-12606) Scala/Java compatibility issue Re: how to extend java transformer from Scala UnaryTransformer ?

[ https://issues.apache.org/jira/browse/SPARK-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044508#comment-16044508 ] Ilyes Hachani edited comment on SPARK-12606 at 6/16/17 9:52 AM: ---

[jira] [Resolved] (SPARK-21118) OOM with 2 handred million vertex when mitrx multply

[ https://issues.apache.org/jira/browse/SPARK-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21118. --- Resolution: Invalid [~icesxrun] do not reopen this. This doesn't describe a problem with Spark, but

[jira] [Closed] (SPARK-21118) OOM with 2 handred million vertex when mitrx multply

[ https://issues.apache.org/jira/browse/SPARK-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-21118. - > OOM with 2 handred million vertex when mitrx multply >

[jira] [Resolved] (SPARK-21077) Cannot access public files over S3 protocol

[ https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21077. --- Resolution: Not A Problem > Cannot access public files over S3 protocol > ---

[jira] [Updated] (SPARK-21116) Support map_contains function

[ https://issues.apache.org/jira/browse/SPARK-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated SPARK-21116: -- Summary: Support map_contains function (was: Support MapKeyContains function) > Support map_c

[jira] [Updated] (SPARK-21116) Support MapKeyContains function

[ https://issues.apache.org/jira/browse/SPARK-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated SPARK-21116: -- Description: map_contains(map , key) Check whether the map contains the key . Returns true if

[jira] [Commented] (SPARK-21077) Cannot access public files over S3 protocol

[ https://issues.apache.org/jira/browse/SPARK-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051680#comment-16051680 ] Steve Loughran commented on SPARK-21077: like people say, this is inevitably a co

[jira] [Commented] (SPARK-21120) Increasing the master's metric is conducive to the spark cluster management system monitoring.

[ https://issues.apache.org/jira/browse/SPARK-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051660#comment-16051660 ] Apache Spark commented on SPARK-21120: -- User 'guoxiaolongzte' has created a pull req

[jira] [Assigned] (SPARK-21120) Increasing the master's metric is conducive to the spark cluster management system monitoring.

[ https://issues.apache.org/jira/browse/SPARK-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21120: Assignee: Apache Spark > Increasing the master's metric is conducive to the spark cluster

[jira] [Assigned] (SPARK-21120) Increasing the master's metric is conducive to the spark cluster management system monitoring.

[ https://issues.apache.org/jira/browse/SPARK-21120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21120: Assignee: (was: Apache Spark) > Increasing the master's metric is conducive to the spa

[jira] [Updated] (SPARK-21116) Support MapKeyContains function

[ https://issues.apache.org/jira/browse/SPARK-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] darion yaphet updated SPARK-21116: -- Description: map_contains(map , key) Check whether the map contains the key . Returns true if

[jira] [Commented] (SPARK-21118) OOM with 2 handred million vertex when mitrx multply

[ https://issues.apache.org/jira/browse/SPARK-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051641#comment-16051641 ] Lorenz Bühmann commented on SPARK-21118: The first point would be to use a subjec

[jira] [Created] (SPARK-21120) Increasing the master's metric is conducive to the spark cluster management system monitoring.

guoxiaolongzte created SPARK-21120: -- Summary: Increasing the master's metric is conducive to the spark cluster management system monitoring. Key: SPARK-21120 URL: https://issues.apache.org/jira/browse/SPARK-21120

[jira] [Comment Edited] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051607#comment-16051607 ] JiYeon OH edited comment on SPARK-12009 at 6/16/17 8:50 AM: I

[jira] [Updated] (SPARK-21118) OOM with 2 handred million vertex when mitrx multply

[ https://issues.apache.org/jira/browse/SPARK-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao updated SPARK-21118: Description: i have 2 matrix each one is 200milions*200milions. i want to multiply them ,but run out with oom . fin

[jira] [Commented] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16051607#comment-16051607 ] JiYeon OH commented on SPARK-12009: --- I'm having the same problem with Spark 2.1.0 I hav

[jira] [Updated] (SPARK-21118) OOM with 2 handred million vertex when mitrx multply

[ https://issues.apache.org/jira/browse/SPARK-21118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao updated SPARK-21118: Description: i have 2 matrix each one is 200milions*200milions. i want to multiply them ,but run out with oom . fin

  1   2   >