[jira] [Created] (SPARK-19383) Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option

Brent Dorsey created SPARK-19383: Summary: Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option Key: SPARK-19383 URL: https://issues.apache.org/jira/browse/SPARK-19383 Project:

[jira] [Resolved] (SPARK-18929) Add Tweedie distribution in GLM

[ https://issues.apache.org/jira/browse/SPARK-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-18929. - Resolution: Fixed Fix Version/s: 2.2.0 > Add Tweedie distribution in GLM >

[jira] [Created] (SPARK-19382) Test sparse vectors in LinearSVCSuite

Joseph K. Bradley created SPARK-19382: - Summary: Test sparse vectors in LinearSVCSuite Key: SPARK-19382 URL: https://issues.apache.org/jira/browse/SPARK-19382 Project: Spark Issue Type:

[jira] [Commented] (SPARK-9215) Implement WAL-free Kinesis receiver that give at-least once guarantee

[ https://issues.apache.org/jira/browse/SPARK-9215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15841011#comment-15841011 ] Gaurav Shah commented on SPARK-9215: [~tdas] ping > Implement WAL-free Kinesis receiver that give

[jira] [Commented] (SPARK-19304) Kinesis checkpoint recovery is 10x slow

[ https://issues.apache.org/jira/browse/SPARK-19304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15841009#comment-15841009 ] Gaurav Shah commented on SPARK-19304: - There are two issues in `KinesisSequenceRangeIterator.getNext`

[jira] [Resolved] (SPARK-18788) Add getNumPartitions() to SparkR

[ https://issues.apache.org/jira/browse/SPARK-18788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-18788. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Target

[jira] [Commented] (SPARK-18821) Bisecting k-means wrapper in SparkR

[ https://issues.apache.org/jira/browse/SPARK-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15841004#comment-15841004 ] Felix Cheung commented on SPARK-18821: -- Need to follow up with programming guide, example and

[jira] [Updated] (SPARK-18821) Bisecting k-means wrapper in SparkR

[ https://issues.apache.org/jira/browse/SPARK-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18821: - Fix Version/s: 2.2.0 > Bisecting k-means wrapper in SparkR > ---

[jira] [Resolved] (SPARK-18821) Bisecting k-means wrapper in SparkR

[ https://issues.apache.org/jira/browse/SPARK-18821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-18821. -- Resolution: Fixed Assignee: Miao Wang > Bisecting k-means wrapper in SparkR >

[jira] [Updated] (SPARK-18218) Optimize BlockMatrix multiplication, which may cause OOM and low parallelism usage problem in several cases

[ https://issues.apache.org/jira/browse/SPARK-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-18218: Assignee: Weichen Xu > Optimize BlockMatrix multiplication, which may cause OOM and low

[jira] [Resolved] (SPARK-18218) Optimize BlockMatrix multiplication, which may cause OOM and low parallelism usage problem in several cases

[ https://issues.apache.org/jira/browse/SPARK-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-18218. - Resolution: Implemented Fix Version/s: 2.2.0 Resolved by

[jira] [Commented] (SPARK-14480) Remove meaningless StringIteratorReader for CSV data source for better performance

[ https://issues.apache.org/jira/browse/SPARK-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840872#comment-15840872 ] Hyukjin Kwon commented on SPARK-14480: -- removed `StringIteratorReader` concatenates the lines in

[jira] [Assigned] (SPARK-14480) Remove meaningless StringIteratorReader for CSV data source for better performance

[ https://issues.apache.org/jira/browse/SPARK-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14480: Assignee: Apache Spark (was: Hyukjin Kwon) > Remove meaningless StringIteratorReader for

[jira] [Created] (SPARK-19381) spark 2.1.0 raises unrelated (unhelpful) error for parquet files beginning with '_'

Paul Pearce created SPARK-19381: --- Summary: spark 2.1.0 raises unrelated (unhelpful) error for parquet files beginning with '_' Key: SPARK-19381 URL: https://issues.apache.org/jira/browse/SPARK-19381

[jira] [Updated] (SPARK-19381) spark 2.1.0 raises unrelated (unhelpful) error for parquet filenames beginning with '_'

[ https://issues.apache.org/jira/browse/SPARK-19381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pearce updated SPARK-19381: Summary: spark 2.1.0 raises unrelated (unhelpful) error for parquet filenames beginning with '_'

[jira] [Assigned] (SPARK-14480) Remove meaningless StringIteratorReader for CSV data source for better performance

[ https://issues.apache.org/jira/browse/SPARK-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14480: Assignee: Hyukjin Kwon (was: Apache Spark) > Remove meaningless StringIteratorReader for

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840857#comment-15840857 ] Liang-Chi Hsieh commented on SPARK-18539: - [~lian cheng] Yea, I see. The term {{optional}} is

[jira] [Reopened] (SPARK-14480) Remove meaningless StringIteratorReader for CSV data source for better performance

[ https://issues.apache.org/jira/browse/SPARK-14480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-14480: This patch have a regression: A column that have escaped newline can't be correctly parsed anymore.

[jira] [Updated] (SPARK-18218) Optimize BlockMatrix multiplication, which may cause OOM and low parallelism usage problem in several cases

[ https://issues.apache.org/jira/browse/SPARK-18218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18218: -- Shepherd: Burak Yavuz (was: Yanbo Liang) > Optimize BlockMatrix multiplication, which

[jira] [Updated] (SPARK-19380) YARN - Dynamic allocation should use configured number of executors as max number of executors

[ https://issues.apache.org/jira/browse/SPARK-19380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated SPARK-19380: -- Description: SPARK-13723 only uses user's number of executors as the initial number of executors

[jira] [Created] (SPARK-19380) YARN - Dynamic allocation should use configured number of executors as max number of executors

Zhe Zhang created SPARK-19380: - Summary: YARN - Dynamic allocation should use configured number of executors as max number of executors Key: SPARK-19380 URL: https://issues.apache.org/jira/browse/SPARK-19380

[jira] [Commented] (SPARK-19220) SSL redirect handler only redirects the server's root

[ https://issues.apache.org/jira/browse/SPARK-19220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840785#comment-15840785 ] Apache Spark commented on SPARK-19220: -- User 'vanzin' has created a pull request for this issue:

[jira] [Updated] (SPARK-19379) SparkAppHandle.getState not registering FAILED state upon Spark app failure in Local mode

[ https://issues.apache.org/jira/browse/SPARK-19379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Kramer updated SPARK-19379: Summary: SparkAppHandle.getState not registering FAILED state upon Spark app failure in Local mode

[jira] [Updated] (SPARK-19220) SSL redirect handler only redirects the server's root

[ https://issues.apache.org/jira/browse/SPARK-19220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-19220: --- Fix Version/s: 2.1.1 > SSL redirect handler only redirects the server's root >

[jira] [Updated] (SPARK-19379) SparkAppHandle.getState not registered FAILED state upon Spark app failure in Local mode

[ https://issues.apache.org/jira/browse/SPARK-19379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Kramer updated SPARK-19379: Summary: SparkAppHandle.getState not registered FAILED state upon Spark app failure in Local mode

[jira] [Updated] (SPARK-19379) SparkAppHandle.getState not registered FAILED state upon Spark app failure in Local mode

[ https://issues.apache.org/jira/browse/SPARK-19379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Kramer updated SPARK-19379: Affects Version/s: 2.1.0 > SparkAppHandle.getState not registered FAILED state upon Spark app

[jira] [Created] (SPARK-19379) SparkAppHandle.getState not registered FAILED state upon Spark app failure

Adam Kramer created SPARK-19379: --- Summary: SparkAppHandle.getState not registered FAILED state upon Spark app failure Key: SPARK-19379 URL: https://issues.apache.org/jira/browse/SPARK-19379 Project:

[jira] [Updated] (SPARK-19378) StateOperator metrics should still return the total number of rows in state even if there was no data for a trigger

[ https://issues.apache.org/jira/browse/SPARK-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-19378: Description: If you have a StreamingDataFrame with an aggregation, we report a metric called

[jira] [Assigned] (SPARK-19378) StateOperator metrics should still return the total number of rows in state even if there was no data for a trigger

[ https://issues.apache.org/jira/browse/SPARK-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19378: Assignee: Burak Yavuz (was: Apache Spark) > StateOperator metrics should still return

[jira] [Assigned] (SPARK-19378) StateOperator metrics should still return the total number of rows in state even if there was no data for a trigger

[ https://issues.apache.org/jira/browse/SPARK-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19378: Assignee: Apache Spark (was: Burak Yavuz) > StateOperator metrics should still return

[jira] [Commented] (SPARK-19378) StateOperator metrics should still return the total number of rows in state even if there was no data for a trigger

[ https://issues.apache.org/jira/browse/SPARK-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840757#comment-15840757 ] Apache Spark commented on SPARK-19378: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Commented] (SPARK-4638) Spark's MLlib SVM classification to include Kernels like Gaussian / (RBF) to find non linear boundaries

[ https://issues.apache.org/jira/browse/SPARK-4638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840711#comment-15840711 ] Joseph K. Bradley commented on SPARK-4638: -- Commenting here b/c of the recent dev list thread:

[jira] [Created] (SPARK-19378) StateOperator metrics should still return the total number of rows in state even if there was no data for a trigger

Burak Yavuz created SPARK-19378: --- Summary: StateOperator metrics should still return the total number of rows in state even if there was no data for a trigger Key: SPARK-19378 URL:

[jira] [Updated] (SPARK-18080) Locality Sensitive Hashing (LSH) Python API

[ https://issues.apache.org/jira/browse/SPARK-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18080: -- Assignee: Yun Ni (was: Yanbo Liang) > Locality Sensitive Hashing (LSH) Python API >

[jira] [Updated] (SPARK-19354) Killed tasks are getting marked as FAILED

[ https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-19354: -- Description: When we enable speculation, we can see there are multiple attempts running for the same

[jira] [Updated] (SPARK-19377) Killed tasks should have the status as KILLED

[ https://issues.apache.org/jira/browse/SPARK-19377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated SPARK-19377: -- Description: |143|10 |0 |SUCCESS|NODE_LOCAL |6 / x.xx.x.x stdout stderr

[jira] [Created] (SPARK-19377) Killed tasks should have the status as KILLED

Devaraj K created SPARK-19377: - Summary: Killed tasks should have the status as KILLED Key: SPARK-19377 URL: https://issues.apache.org/jira/browse/SPARK-19377 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-19067) mapWithState Style API

[ https://issues.apache.org/jira/browse/SPARK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-19067: - Assignee: Tathagata Das > mapWithState Style API > -- > >

[jira] [Commented] (SPARK-19371) Cannot spread cached partitions evenly across executors

[ https://issues.apache.org/jira/browse/SPARK-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840569#comment-15840569 ] Thunder Stumpges commented on SPARK-19371: -- Thanks Sean. I don't think I can (always) rely on

[jira] [Resolved] (SPARK-19376) CLONE - CheckAnalysis rejects TPCDS query 32

[ https://issues.apache.org/jira/browse/SPARK-19376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Shahdadi resolved SPARK-19376. -- Resolution: Fixed > CLONE - CheckAnalysis rejects TPCDS query 32 >

[jira] [Created] (SPARK-19376) CLONE - CheckAnalysis rejects TPCDS query 32

Mostafa Shahdadi created SPARK-19376: Summary: CLONE - CheckAnalysis rejects TPCDS query 32 Key: SPARK-19376 URL: https://issues.apache.org/jira/browse/SPARK-19376 Project: Spark Issue

[jira] [Updated] (SPARK-19364) Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled and an exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Milkowski updated SPARK-19364: - Priority: Blocker (was: Major) > Stream Blocks in Storage Persists Forever when Kinesis

[jira] [Updated] (SPARK-19364) Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled and an exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Milkowski updated SPARK-19364: - Description: -- update --- we found that below situation occurs when we encounter

[jira] [Updated] (SPARK-19364) Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled and an exception is thrown

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Milkowski updated SPARK-19364: - Summary: Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled

[jira] [Commented] (SPARK-19111) S3 Mesos history upload fails silently if too large

[ https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840419#comment-15840419 ] Charles Allen commented on SPARK-19111: --- While switching to s3a helped the logs upload, it made the

[jira] [Commented] (SPARK-19111) S3 Mesos history upload fails silently if too large

[ https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840418#comment-15840418 ] Charles Allen commented on SPARK-19111: --- We have a patch https://github.com/apache/spark/pull/16714

[jira] [Updated] (SPARK-19364) Some Stream Blocks in Storage Persists Forever

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Milkowski updated SPARK-19364: - Description: -- update --- we found that below situation occurs when we encounter

[jira] [Updated] (SPARK-19364) Some Stream Blocks in Storage Persists Forever

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Milkowski updated SPARK-19364: - Description: -- update --- we found that below situation occurs when we encounter

[jira] [Updated] (SPARK-19364) Some Stream Blocks in Storage Persists Forever

[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Milkowski updated SPARK-19364: - Description: *** update *** we found that below situation occurs when we encounter

[jira] [Commented] (SPARK-15505) Explode nested Array in DF Column into Multiple Columns

[ https://issues.apache.org/jira/browse/SPARK-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840390#comment-15840390 ] Herman van Hovell commented on SPARK-15505: --- I am closing this as a won't fix. Thanks for the

[jira] [Closed] (SPARK-19375) na.fill() should not change the data type of column

[ https://issues.apache.org/jira/browse/SPARK-19375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-19375. -- Resolution: Duplicate > na.fill() should not change the data type of column >

[jira] [Closed] (SPARK-15505) Explode nested Array in DF Column into Multiple Columns

[ https://issues.apache.org/jira/browse/SPARK-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-15505. - Resolution: Won't Fix > Explode nested Array in DF Column into Multiple Columns >

[jira] [Commented] (SPARK-18080) Locality Sensitive Hashing (LSH) Python API

[ https://issues.apache.org/jira/browse/SPARK-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840388#comment-15840388 ] Apache Spark commented on SPARK-18080: -- User 'Yunni' has created a pull request for this issue:

[jira] [Updated] (SPARK-19374) java.security.KeyManagementException: Default SSLContext is initialized automatically

[ https://issues.apache.org/jira/browse/SPARK-19374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Derek M Miller updated SPARK-19374: --- Description: I am currently getting an SSL error when turning on ssl. I have confirmed

[jira] [Updated] (SPARK-19374) java.security.KeyManagementException: Default SSLContext is initialized automatically

[ https://issues.apache.org/jira/browse/SPARK-19374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Derek M Miller updated SPARK-19374: --- Description: I am currently getting an SSL error when turning on ssl. I have confirmed

[jira] [Updated] (SPARK-19374) java.security.KeyManagementException: Default SSLContext is initialized automatically

[ https://issues.apache.org/jira/browse/SPARK-19374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Derek M Miller updated SPARK-19374: --- Description: I am currently getting an SSL error when turning on ssl. I have confirmed

[jira] [Created] (SPARK-19375) na.fill() should not change the data type of column

Davies Liu created SPARK-19375: -- Summary: na.fill() should not change the data type of column Key: SPARK-19375 URL: https://issues.apache.org/jira/browse/SPARK-19375 Project: Spark Issue Type:

[jira] [Created] (SPARK-19374) java.security.KeyManagementException: Default SSLContext is initialized automatically

Derek M Miller created SPARK-19374: -- Summary: java.security.KeyManagementException: Default SSLContext is initialized automatically Key: SPARK-19374 URL: https://issues.apache.org/jira/browse/SPARK-19374

[jira] [Commented] (SPARK-19316) Spark event logs are huge compared to 1.5.2

[ https://issues.apache.org/jira/browse/SPARK-19316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840348#comment-15840348 ] Jisoo Kim commented on SPARK-19316: --- Found the duplicate and made a PR to resolve the issue

[jira] [Assigned] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16333: Assignee: (was: Apache Spark) > Excessive Spark history event/json data size (5GB

[jira] [Assigned] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16333: Assignee: Apache Spark > Excessive Spark history event/json data size (5GB each) >

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840340#comment-15840340 ] Apache Spark commented on SPARK-16333: -- User 'jisookim0513' has created a pull request for this

[jira] [Commented] (SPARK-15505) Explode nested Array in DF Column into Multiple Columns

[ https://issues.apache.org/jira/browse/SPARK-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840227#comment-15840227 ] Jorge Machado commented on SPARK-15505: --- Sometimes you read from a database that has an array in

[jira] [Assigned] (SPARK-18873) New test cases for scalar subquery

[ https://issues.apache.org/jira/browse/SPARK-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18873: Assignee: Apache Spark > New test cases for scalar subquery >

[jira] [Commented] (SPARK-18873) New test cases for scalar subquery

[ https://issues.apache.org/jira/browse/SPARK-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840197#comment-15840197 ] Apache Spark commented on SPARK-18873: -- User 'nsyca' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18873) New test cases for scalar subquery

[ https://issues.apache.org/jira/browse/SPARK-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18873: Assignee: (was: Apache Spark) > New test cases for scalar subquery >

[jira] [Created] (SPARK-19373) Mesos implementation of spark.scheduler.minRegisteredResourcesRatio looks at acquired cores rather than registerd cores

Michael Gummelt created SPARK-19373: --- Summary: Mesos implementation of spark.scheduler.minRegisteredResourcesRatio looks at acquired cores rather than registerd cores Key: SPARK-19373 URL:

[jira] [Commented] (SPARK-18539) Cannot filter by nonexisting column in parquet file

[ https://issues.apache.org/jira/browse/SPARK-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840186#comment-15840186 ] Cheng Lian commented on SPARK-18539: [~viirya], sorry for the (super) late reply. What I mentioned

[jira] [Commented] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840151#comment-15840151 ] Ilya Matiach commented on SPARK-17975: -- [~josephkb] I was able to verify that this issue is now

[jira] [Commented] (SPARK-19220) SSL redirect handler only redirects the server's root

[ https://issues.apache.org/jira/browse/SPARK-19220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840138#comment-15840138 ] Apache Spark commented on SPARK-19220: -- User 'vanzin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-19338) Always Identical Name for UDF in the EXPLAIN output

[ https://issues.apache.org/jira/browse/SPARK-19338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19338. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Always Identical Name for UDF

[jira] [Updated] (SPARK-19338) Always Identical Name for UDF in the EXPLAIN output

[ https://issues.apache.org/jira/browse/SPARK-19338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19338: Assignee: Takeshi Yamamuro > Always Identical Name for UDF in the EXPLAIN output >

[jira] [Commented] (SPARK-19371) Cannot spread cached partitions evenly across executors

[ https://issues.apache.org/jira/browse/SPARK-19371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840113#comment-15840113 ] Sean Owen commented on SPARK-19371: --- There's a tension between waiting for locality and taking a free

[jira] [Assigned] (SPARK-18872) New test cases for EXISTS subquery

[ https://issues.apache.org/jira/browse/SPARK-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18872: Assignee: Apache Spark > New test cases for EXISTS subquery >

[jira] [Commented] (SPARK-18872) New test cases for EXISTS subquery

[ https://issues.apache.org/jira/browse/SPARK-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840098#comment-15840098 ] Apache Spark commented on SPARK-18872: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-18872) New test cases for EXISTS subquery

[ https://issues.apache.org/jira/browse/SPARK-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18872: Assignee: (was: Apache Spark) > New test cases for EXISTS subquery >

[jira] [Created] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

Jay Pranavamurthi created SPARK-19372: - Summary: Code generation for Filter predicate including many OR conditions exceeds JVM method size limit Key: SPARK-19372 URL:

[jira] [Updated] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Pranavamurthi updated SPARK-19372: -- Attachment: wide400cols.csv > Code generation for Filter predicate including many OR

[jira] [Created] (SPARK-19371) Cannot spread cached partitions evenly across executors

Thunder Stumpges created SPARK-19371: Summary: Cannot spread cached partitions evenly across executors Key: SPARK-19371 URL: https://issues.apache.org/jira/browse/SPARK-19371 Project: Spark

[jira] [Resolved] (SPARK-19369) SparkConf not getting properly initialized in PySpark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19369. Resolution: Duplicate Try setting those in the command line for now; this will be fixed in

[jira] [Updated] (SPARK-19370) Flaky test: MetadataCacheSuite

[ https://issues.apache.org/jira/browse/SPARK-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-19370: --- Affects Version/s: 2.1.0 > Flaky test: MetadataCacheSuite > -- > >

[jira] [Updated] (SPARK-19370) Flaky test: MetadataCacheSuite

[ https://issues.apache.org/jira/browse/SPARK-19370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-19370: --- Description:

[jira] [Created] (SPARK-19370) Flaky test: MetadataCacheSuite

Davies Liu created SPARK-19370: -- Summary: Flaky test: MetadataCacheSuite Key: SPARK-19370 URL: https://issues.apache.org/jira/browse/SPARK-19370 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-19368) Very bad performance in BlockMatrix.toIndexedRowMatrix()

[ https://issues.apache.org/jira/browse/SPARK-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19368: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) Do you see ways to optimize

[jira] [Commented] (SPARK-15505) Explode nested Array in DF Column into Multiple Columns

[ https://issues.apache.org/jira/browse/SPARK-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839908#comment-15839908 ] Herman van Hovell commented on SPARK-15505: --- H This is will be quite bad in terms of

[jira] [Created] (SPARK-19369) SparkConf not getting properly initialized in PySpark 2.1.0

Sidney Feiner created SPARK-19369: - Summary: SparkConf not getting properly initialized in PySpark 2.1.0 Key: SPARK-19369 URL: https://issues.apache.org/jira/browse/SPARK-19369 Project: Spark

[jira] [Updated] (SPARK-19369) SparkConf not getting properly initialized in PySpark 2.1.0

[ https://issues.apache.org/jira/browse/SPARK-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sidney Feiner updated SPARK-19369: -- Description: Trying to migrate from Spark 1.6 to 2.1, I've stumbled upon a small problem - my

[jira] [Commented] (SPARK-19122) Unnecessary shuffle+sort added if join predicates ordering differ from bucketing and sorting order

[ https://issues.apache.org/jira/browse/SPARK-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839845#comment-15839845 ] Herman van Hovell commented on SPARK-19122: --- [~tejasp] We should fix this. The rules make

[jira] [Commented] (SPARK-5786) Documentation of Narrow Dependencies

[ https://issues.apache.org/jira/browse/SPARK-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839788#comment-15839788 ] Hyukjin Kwon commented on SPARK-5786: - It seems they are documented, at least, in API docs, e.g.,

[jira] [Resolved] (SPARK-2687) after receving allocated containers,amClient should remove ContainerRequest.

[ https://issues.apache.org/jira/browse/SPARK-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-2687. - Resolution: Duplicate I am resolving this per

[jira] [Commented] (SPARK-18839) Executor is active on web, but actually is dead

[ https://issues.apache.org/jira/browse/SPARK-18839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839719#comment-15839719 ] Hyukjin Kwon commented on SPARK-18839: -- [~uncleGen] Would you mind elaborating why you think it is

[jira] [Commented] (SPARK-18579) spark-csv strips whitespace (pyspark)

[ https://issues.apache.org/jira/browse/SPARK-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839704#comment-15839704 ] Hyukjin Kwon commented on SPARK-18579: -- Can we just strip them within the dataframe/dataset?

[jira] [Resolved] (SPARK-19361) kafka.maxRatePerPartition for compacted topic cause exception

[ https://issues.apache.org/jira/browse/SPARK-19361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger resolved SPARK-19361. Resolution: Duplicate > kafka.maxRatePerPartition for compacted topic cause exception >

[jira] [Commented] (SPARK-19361) kafka.maxRatePerPartition for compacted topic cause exception

[ https://issues.apache.org/jira/browse/SPARK-19361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839683#comment-15839683 ] Cody Koeninger commented on SPARK-19361: Compacted topics in general don't work with direct

[jira] [Commented] (SPARK-19368) Very bad performance in BlockMatrix.toIndexedRowMatrix()

[ https://issues.apache.org/jira/browse/SPARK-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839673#comment-15839673 ] Ohad Raviv commented on SPARK-19368: caused by.. > Very bad performance in

[jira] [Updated] (SPARK-19368) Very bad performance in BlockMatrix.toIndexedRowMatrix()

[ https://issues.apache.org/jira/browse/SPARK-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Raviv updated SPARK-19368: --- Attachment: profiler snapshot.png > Very bad performance in BlockMatrix.toIndexedRowMatrix() >

[jira] [Resolved] (SPARK-17734) inner equi-join shorthand that returns Datasets, like DataFrame already has

[ https://issues.apache.org/jira/browse/SPARK-17734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17734. -- Resolution: Won't Fix We have {{joinWith}} to return {{Dataset}}. Also, we have {{join}} and

[jira] [Updated] (SPARK-19368) Very bad performance in BlockMatrix.toIndexedRowMatrix()

[ https://issues.apache.org/jira/browse/SPARK-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ohad Raviv updated SPARK-19368: --- Description: In SPARK-12869, this function was optimized for the case of dense matrices using

[jira] [Created] (SPARK-19368) Very bad performance in BlockMatrix.toIndexedRowMatrix()

Ohad Raviv created SPARK-19368: -- Summary: Very bad performance in BlockMatrix.toIndexedRowMatrix() Key: SPARK-19368 URL: https://issues.apache.org/jira/browse/SPARK-19368 Project: Spark Issue

[jira] [Created] (SPARK-19367) Hive metastore temporary configuration doesn't specify default filesystem

Jacek Lewandowski created SPARK-19367: - Summary: Hive metastore temporary configuration doesn't specify default filesystem Key: SPARK-19367 URL: https://issues.apache.org/jira/browse/SPARK-19367

  1   2   >