[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15617506#comment-15617506 ] Liwei Lin commented on SPARK-16845: --- [~dondrake] yea I would expect it to work as long

[jira] [Updated] (SPARK-18164) ForeachSink should fail the Spark job if `process` throws exception

[ https://issues.apache.org/jira/browse/SPARK-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18164: - Issue Type: Sub-task (was: Bug) Parent: SPARK-8360 > ForeachSink should fail the Spark j

[jira] [Resolved] (SPARK-18164) ForeachSink should fail the Spark job if `process` throws exception

[ https://issues.apache.org/jira/browse/SPARK-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18164. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.3 > ForeachSink should fai

[jira] [Updated] (SPARK-18108) Partition discovery fails with explicitly written long partitions

[ https://issues.apache.org/jira/browse/SPARK-18108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Moorhead updated SPARK-18108: - Issue Type: Bug (was: Question) > Partition discovery fails with explicitly written long

[jira] [Commented] (SPARK-16312) Docs for Kafka 0.10 consumer integration

[ https://issues.apache.org/jira/browse/SPARK-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15617150#comment-15617150 ] Apache Spark commented on SPARK-16312: -- User 'lw-lin' has created a pull request for

[jira] [Commented] (SPARK-17963) Add examples (extend) in each function and improve documentation with arguments

[ https://issues.apache.org/jira/browse/SPARK-17963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15617070#comment-15617070 ] Apache Spark commented on SPARK-17963: -- User 'HyukjinKwon' has created a pull reques

[jira] [Commented] (SPARK-17344) Kafka 0.8 support for Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15617004#comment-15617004 ] Michael Allman commented on SPARK-17344: We (at VideoAmp) would love to use struc

[jira] [Commented] (SPARK-18168) Revert the change of SPARK-18167

[ https://issues.apache.org/jira/browse/SPARK-18168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616980#comment-15616980 ] Yin Huai commented on SPARK-18168: -- cc [~ekhliang] > Revert the change of SPARK-18167 >

[jira] [Updated] (SPARK-18168) Revert the change of SPARK-18167

[ https://issues.apache.org/jira/browse/SPARK-18168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-18168: - Target Version/s: 2.1.0 > Revert the change of SPARK-18167 > > >

[jira] [Created] (SPARK-18168) Revert the change of SPARK-18167

Yin Huai created SPARK-18168: Summary: Revert the change of SPARK-18167 Key: SPARK-18168 URL: https://issues.apache.org/jira/browse/SPARK-18168 Project: Spark Issue Type: Bug Components

[jira] [Updated] (SPARK-18168) Revert the change of SPARK-18167

[ https://issues.apache.org/jira/browse/SPARK-18168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-18168: - Priority: Blocker (was: Major) > Revert the change of SPARK-18167 > > >

[jira] [Assigned] (SPARK-18167) Flaky test when hive partition pruning is enabled

[ https://issues.apache.org/jira/browse/SPARK-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18167: Assignee: (was: Apache Spark) > Flaky test when hive partition pruning is enabled > --

[jira] [Assigned] (SPARK-18167) Flaky test when hive partition pruning is enabled

[ https://issues.apache.org/jira/browse/SPARK-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18167: Assignee: Apache Spark > Flaky test when hive partition pruning is enabled > -

[jira] [Commented] (SPARK-18167) Flaky test when hive partition pruning is enabled

[ https://issues.apache.org/jira/browse/SPARK-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616845#comment-15616845 ] Apache Spark commented on SPARK-18167: -- User 'ericl' has created a pull request for

[jira] [Created] (SPARK-18167) Flaky test when hive partition pruning is enabled

Eric Liang created SPARK-18167: -- Summary: Flaky test when hive partition pruning is enabled Key: SPARK-18167 URL: https://issues.apache.org/jira/browse/SPARK-18167 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-18144) StreamingQueryListener.QueryStartedEvent is not written to event log

[ https://issues.apache.org/jira/browse/SPARK-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18144: - Priority: Minor (was: Major) > StreamingQueryListener.QueryStartedEvent is not written to event

[jira] [Created] (SPARK-18166) GeneralizedLinearRegression Wrong Value Range for Poisson Distribution

Wayne Zhang created SPARK-18166: --- Summary: GeneralizedLinearRegression Wrong Value Range for Poisson Distribution Key: SPARK-18166 URL: https://issues.apache.org/jira/browse/SPARK-18166 Project: Spark

[jira] [Updated] (SPARK-18081) Locality Sensitive Hashing (LSH) User Guide

[ https://issues.apache.org/jira/browse/SPARK-18081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18081: -- Target Version/s: 2.1.0 > Locality Sensitive Hashing (LSH) User Guide > ---

[jira] [Resolved] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-5992. -- Resolution: Fixed Fix Version/s: 2.1.0 Target Version/s: (was: ) Issu

[jira] [Updated] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5992: - Component/s: (was: MLlib) ML > Locality Sensitive Hashing (LSH) for M

[jira] [Updated] (SPARK-18143) History Server is broken because of the refactoring work in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18143: - Priority: Blocker (was: Major) > History Server is broken because of the refactoring work in Str

[jira] [Updated] (SPARK-17791) Join reordering using star schema detection

[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17791: Assignee: Ioana Delaney > Join reordering using star schema detection > ---

[jira] [Updated] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17626: Target Version/s: 2.2.0 > TPC-DS performance improvements using star-schema heuristics > --

[jira] [Updated] (SPARK-17791) Join reordering using star schema detection

[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17791: Target Version/s: 2.2.0 > Join reordering using star schema detection > ---

[jira] [Commented] (SPARK-17791) Join reordering using star schema detection

[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616590#comment-15616590 ] Ron Hu commented on SPARK-17791: This JIRA is indeed complementary to Cost-Based Optimize

[jira] [Commented] (SPARK-18123) org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils.saveTable the case senstivity issue

[ https://issues.apache.org/jira/browse/SPARK-18123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616584#comment-15616584 ] Dongjoon Hyun commented on SPARK-18123: --- Hi, [~zwu@gmail.com]. Do you think you

[jira] [Commented] (SPARK-17612) Support `DESCRIBE table PARTITION` SQL syntax

[ https://issues.apache.org/jira/browse/SPARK-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616575#comment-15616575 ] Dongjoon Hyun commented on SPARK-17612: --- Ah, I see. For that question, this issue i

[jira] [Commented] (SPARK-17612) Support `DESCRIBE table PARTITION` SQL syntax

[ https://issues.apache.org/jira/browse/SPARK-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616542#comment-15616542 ] Franck Tago commented on SPARK-17612: - Hi Thanks for the quick reply. So I was only

[jira] [Created] (SPARK-18165) Kinesis support in Structured Streaming

Lauren Moos created SPARK-18165: --- Summary: Kinesis support in Structured Streaming Key: SPARK-18165 URL: https://issues.apache.org/jira/browse/SPARK-18165 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-18144) StreamingQueryListener.QueryStartedEvent is not written to event log

[ https://issues.apache.org/jira/browse/SPARK-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616503#comment-15616503 ] Apache Spark commented on SPARK-18144: -- User 'CodingCat' has created a pull request

[jira] [Assigned] (SPARK-18144) StreamingQueryListener.QueryStartedEvent is not written to event log

[ https://issues.apache.org/jira/browse/SPARK-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18144: Assignee: (was: Apache Spark) > StreamingQueryListener.QueryStartedEvent is not writte

[jira] [Assigned] (SPARK-18144) StreamingQueryListener.QueryStartedEvent is not written to event log

[ https://issues.apache.org/jira/browse/SPARK-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18144: Assignee: Apache Spark > StreamingQueryListener.QueryStartedEvent is not written to event

[jira] [Commented] (SPARK-17612) Support `DESCRIBE table PARTITION` SQL syntax

[ https://issues.apache.org/jira/browse/SPARK-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616422#comment-15616422 ] Dongjoon Hyun commented on SPARK-17612: --- Hi, [~tafra...@gmail.com]. I think you are

[jira] [Commented] (SPARK-15616) Metastore relation should fallback to HDFS size of partitions that are involved in Query if statistics are not available.

[ https://issues.apache.org/jira/browse/SPARK-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616385#comment-15616385 ] Franck Tago commented on SPARK-15616: - So I tried the changed that were made for this

[jira] [Commented] (SPARK-17612) Support `DESCRIBE table PARTITION` SQL syntax

[ https://issues.apache.org/jira/browse/SPARK-17612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616376#comment-15616376 ] Franck Tago commented on SPARK-17612: - Hi Basically I have an issue where I am per

[jira] [Commented] (SPARK-882) Have link for feedback/suggestions in docs

[ https://issues.apache.org/jira/browse/SPARK-882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616159#comment-15616159 ] Sean Owen commented on SPARK-882: - You're right, I was thinking of the API docs rather than

[jira] [Assigned] (SPARK-18164) ForeachSink should fail the Spark job if `process` throws exception

[ https://issues.apache.org/jira/browse/SPARK-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18164: Assignee: Apache Spark (was: Shixiong Zhu) > ForeachSink should fail the Spark job if `pr

[jira] [Assigned] (SPARK-18164) ForeachSink should fail the Spark job if `process` throws exception

[ https://issues.apache.org/jira/browse/SPARK-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18164: Assignee: Shixiong Zhu (was: Apache Spark) > ForeachSink should fail the Spark job if `pr

[jira] [Commented] (SPARK-18164) ForeachSink should fail the Spark job if `process` throws exception

[ https://issues.apache.org/jira/browse/SPARK-18164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616143#comment-15616143 ] Apache Spark commented on SPARK-18164: -- User 'zsxwing' has created a pull request fo

[jira] [Created] (SPARK-18164) ForeachSink should fail the Spark job if `process` throws exception

Shixiong Zhu created SPARK-18164: Summary: ForeachSink should fail the Spark job if `process` throws exception Key: SPARK-18164 URL: https://issues.apache.org/jira/browse/SPARK-18164 Project: Spark

[jira] [Assigned] (SPARK-17992) HiveClient.getPartitionsByFilter throws an exception for some unsupported filters when hive.metastore.try.direct.sql=false

[ https://issues.apache.org/jira/browse/SPARK-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17992: Assignee: (was: Apache Spark) > HiveClient.getPartitionsByFilter throws an exception f

[jira] [Assigned] (SPARK-17992) HiveClient.getPartitionsByFilter throws an exception for some unsupported filters when hive.metastore.try.direct.sql=false

[ https://issues.apache.org/jira/browse/SPARK-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17992: Assignee: Apache Spark > HiveClient.getPartitionsByFilter throws an exception for some uns

[jira] [Commented] (SPARK-17992) HiveClient.getPartitionsByFilter throws an exception for some unsupported filters when hive.metastore.try.direct.sql=false

[ https://issues.apache.org/jira/browse/SPARK-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616068#comment-15616068 ] Apache Spark commented on SPARK-17992: -- User 'mallman' has created a pull request fo

[jira] [Commented] (SPARK-882) Have link for feedback/suggestions in docs

[ https://issues.apache.org/jira/browse/SPARK-882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615975#comment-15615975 ] Deron Eriksson commented on SPARK-882: -- It looks like the More menu in the docs (http:

[jira] [Updated] (SPARK-18014) Filters are incorrectly being grouped together when there is processing in between

[ https://issues.apache.org/jira/browse/SPARK-18014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Patterson updated SPARK-18014: -- Environment: Pyspark 2.0.1, Ipython 4.2 (was: Pyspark 2.0.0, Ipython 4.2) > Filters ar

[jira] [Commented] (SPARK-13331) AES support for over-the-wire encryption

[ https://issues.apache.org/jira/browse/SPARK-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615888#comment-15615888 ] Marcelo Vanzin commented on SPARK-13331: You need to be a little patient. People

[jira] [Commented] (SPARK-18162) SparkEnv.get.metricsSystem in spark-shell results in error: missing or invalid dependency detected while loading class file 'MetricsSystem.class'

[ https://issues.apache.org/jira/browse/SPARK-18162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615419#comment-15615419 ] Sean Owen commented on SPARK-18162: --- I don't observe this on master right now. Are you

[jira] [Closed] (SPARK-18163) Union unexpected behaviour when generating data frames programatically

[ https://issues.apache.org/jira/browse/SPARK-18163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ulrich zink closed SPARK-18163. --- Resolution: Invalid > Union unexpected behaviour when generating data frames programatically > --

[jira] [Assigned] (SPARK-18148) Misleading Error Message for Aggregation Without Window/GroupBy

[ https://issues.apache.org/jira/browse/SPARK-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18148: Assignee: (was: Apache Spark) > Misleading Error Message for Aggregation Without Windo

[jira] [Assigned] (SPARK-18148) Misleading Error Message for Aggregation Without Window/GroupBy

[ https://issues.apache.org/jira/browse/SPARK-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18148: Assignee: Apache Spark > Misleading Error Message for Aggregation Without Window/GroupBy >

[jira] [Commented] (SPARK-18148) Misleading Error Message for Aggregation Without Window/GroupBy

[ https://issues.apache.org/jira/browse/SPARK-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615121#comment-15615121 ] Apache Spark commented on SPARK-18148: -- User 'jiangxb1987' has created a pull reques

[jira] [Created] (SPARK-18163) Union unexpected behaviour when generating data frames programatically

Ulrich zink created SPARK-18163: --- Summary: Union unexpected behaviour when generating data frames programatically Key: SPARK-18163 URL: https://issues.apache.org/jira/browse/SPARK-18163 Project: Spark

[jira] [Commented] (SPARK-18125) Spark generated code causes CompileException when groupByKey, reduceGroups and map(_._2) are used

[ https://issues.apache.org/jira/browse/SPARK-18125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615034#comment-15615034 ] Kazuaki Ishizaki commented on SPARK-18125: -- I confirmed this code can reproduce

[jira] [Commented] (SPARK-18159) Stand-alone cluster, supervised app: restart of worker hosting the driver causes app to run twice

[ https://issues.apache.org/jira/browse/SPARK-18159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615025#comment-15615025 ] Stephan Kepser commented on SPARK-18159: I saw the old executors kept running for

[jira] [Commented] (SPARK-18159) Stand-alone cluster, supervised app: restart of worker hosting the driver causes app to run twice

[ https://issues.apache.org/jira/browse/SPARK-18159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615024#comment-15615024 ] Stephan Kepser commented on SPARK-18159: I saw the old executors kept running for

[jira] [Issue Comment Deleted] (SPARK-18159) Stand-alone cluster, supervised app: restart of worker hosting the driver causes app to run twice

[ https://issues.apache.org/jira/browse/SPARK-18159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Kepser updated SPARK-18159: --- Comment: was deleted (was: I saw the old executors kept running for several hours (more than

[jira] [Commented] (SPARK-14567) Add instrumentation logs to MLlib training algorithms

[ https://issues.apache.org/jira/browse/SPARK-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615017#comment-15615017 ] Apache Spark commented on SPARK-14567: -- User 'zhengruifeng' has created a pull reque

[jira] [Assigned] (SPARK-14567) Add instrumentation logs to MLlib training algorithms

[ https://issues.apache.org/jira/browse/SPARK-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14567: Assignee: Apache Spark (was: Timothy Hunter) > Add instrumentation logs to MLlib training

[jira] [Assigned] (SPARK-14567) Add instrumentation logs to MLlib training algorithms

[ https://issues.apache.org/jira/browse/SPARK-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14567: Assignee: Timothy Hunter (was: Apache Spark) > Add instrumentation logs to MLlib training

[jira] [Resolved] (SPARK-18150) Spark 2.* failes to create partitions for avro files

[ https://issues.apache.org/jira/browse/SPARK-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18150. --- Resolution: Invalid Please start on the mailing list with a more detailed question, and after review

[jira] [Resolved] (SPARK-18151) CLONE - MetadataLog should support purging old logs

[ https://issues.apache.org/jira/browse/SPARK-18151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18151. --- Resolution: Invalid > CLONE - MetadataLog should support purging old logs > -

[jira] [Resolved] (SPARK-18157) CLONE - Support purging aged file entry for FileStreamSource metadata log

[ https://issues.apache.org/jira/browse/SPARK-18157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18157. --- Resolution: Invalid > CLONE - Support purging aged file entry for FileStreamSource metadata log > ---

[jira] [Resolved] (SPARK-18154) CLONE - Change Source API so that sources do not need to keep unbounded state

[ https://issues.apache.org/jira/browse/SPARK-18154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18154. --- Resolution: Invalid > CLONE - Change Source API so that sources do not need to keep unbounded state >

[jira] [Resolved] (SPARK-18155) CLONE - HDFSMetadataLog should not leak CRC files

[ https://issues.apache.org/jira/browse/SPARK-18155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18155. --- Resolution: Invalid > CLONE - HDFSMetadataLog should not leak CRC files > ---

[jira] [Resolved] (SPARK-18152) CLONE - FileStreamSource should not track the list of seen files indefinitely

[ https://issues.apache.org/jira/browse/SPARK-18152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18152. --- Resolution: Invalid > CLONE - FileStreamSource should not track the list of seen files indefinitely >

[jira] [Resolved] (SPARK-18153) CLONE - Ability to remove old metadata for structure streaming MetadataLog

[ https://issues.apache.org/jira/browse/SPARK-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18153. --- Resolution: Invalid > CLONE - Ability to remove old metadata for structure streaming MetadataLog > --

[jira] [Resolved] (SPARK-18156) CLONE - StreamExecution should discard unneeded metadata

[ https://issues.apache.org/jira/browse/SPARK-18156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18156. --- Resolution: Invalid > CLONE - StreamExecution should discard unneeded metadata >

[jira] [Commented] (SPARK-18147) Broken Spark SQL Codegen

[ https://issues.apache.org/jira/browse/SPARK-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614959#comment-15614959 ] Kazuaki Ishizaki commented on SPARK-18147: -- This also cause the same exception.

[jira] [Commented] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614954#comment-15614954 ] Vivek Gupta commented on SPARK-11278: - We are facing an issue with Spark 1.6.0 whereb

[jira] [Commented] (SPARK-18159) Stand-alone cluster, supervised app: restart of worker hosting the driver causes app to run twice

[ https://issues.apache.org/jira/browse/SPARK-18159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614955#comment-15614955 ] Sean Owen commented on SPARK-18159: --- Yes, that is at least the intended behavior. The n

[jira] [Resolved] (SPARK-17940) Typo in LAST function error message

[ https://issues.apache.org/jira/browse/SPARK-17940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17940. --- Resolution: Duplicate > Typo in LAST function error message > --- > >

[jira] [Commented] (SPARK-18150) Spark 2.* failes to create partitions for avro files

[ https://issues.apache.org/jira/browse/SPARK-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614937#comment-15614937 ] Sean Owen commented on SPARK-18150: --- Whoa, [~sunilsbjoshi], I don't understand why you

[jira] [Resolved] (SPARK-18133) Python ML Pipeline Example has syntax errors

[ https://issues.apache.org/jira/browse/SPARK-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-18133. - Resolution: Fixed Assignee: Jagadeesan A S Fix Version/s: 2.1.0 > Python ML Pipel

[jira] [Created] (SPARK-18162) SparkEnv.get.metricsSystem in spark-shell results in error: missing or invalid dependency detected while loading class file 'MetricsSystem.class'

Jacek Laskowski created SPARK-18162: --- Summary: SparkEnv.get.metricsSystem in spark-shell results in error: missing or invalid dependency detected while loading class file 'MetricsSystem.class' Key: SPARK-18162

[jira] [Commented] (SPARK-18148) Misleading Error Message for Aggregation Without Window/GroupBy

[ https://issues.apache.org/jira/browse/SPARK-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614773#comment-15614773 ] Jiang Xingbo commented on SPARK-18148: -- [~pat.mcdono...@databricks.com] I've reprodu

[jira] [Commented] (SPARK-18161) Default PickleSerializer pickle protocol doesn't handle > 4GB objects

[ https://issues.apache.org/jira/browse/SPARK-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614752#comment-15614752 ] Sloane Simmons commented on SPARK-18161: I changed the importance from minor to m

[jira] [Commented] (SPARK-18161) Default PickleSerializer pickle protocol doesn't handle > 4GB objects

[ https://issues.apache.org/jira/browse/SPARK-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614741#comment-15614741 ] Apache Spark commented on SPARK-18161: -- User 'singularperturbation' has created a pu

[jira] [Assigned] (SPARK-18161) Default PickleSerializer pickle protocol doesn't handle > 4GB objects

[ https://issues.apache.org/jira/browse/SPARK-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18161: Assignee: Apache Spark > Default PickleSerializer pickle protocol doesn't handle > 4GB obj

[jira] [Assigned] (SPARK-18161) Default PickleSerializer pickle protocol doesn't handle > 4GB objects

[ https://issues.apache.org/jira/browse/SPARK-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18161: Assignee: (was: Apache Spark) > Default PickleSerializer pickle protocol doesn't handl

[jira] [Assigned] (SPARK-18160) SparkContext.addFile doesn't work in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18160: Assignee: Apache Spark > SparkContext.addFile doesn't work in yarn-cluster mode >

[jira] [Assigned] (SPARK-18160) SparkContext.addFile doesn't work in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18160: Assignee: (was: Apache Spark) > SparkContext.addFile doesn't work in yarn-cluster mode

[jira] [Commented] (SPARK-18160) SparkContext.addFile doesn't work in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614668#comment-15614668 ] Apache Spark commented on SPARK-18160: -- User 'zjffdu' has created a pull request for

[jira] [Updated] (SPARK-18161) Default PickleSerializer pickle protocol doesn't handle > 4GB objects

[ https://issues.apache.org/jira/browse/SPARK-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sloane Simmons updated SPARK-18161: --- Priority: Major (was: Minor) > Default PickleSerializer pickle protocol doesn't handle > 4GB

[jira] [Created] (SPARK-18161) Default PickleSerializer pickle protocol doesn't handle > 4GB objects

Sloane Simmons created SPARK-18161: -- Summary: Default PickleSerializer pickle protocol doesn't handle > 4GB objects Key: SPARK-18161 URL: https://issues.apache.org/jira/browse/SPARK-18161 Project: Sp

[jira] [Created] (SPARK-18160) SparkContext.addFile doesn't work in yarn-cluster mode

Jeff Zhang created SPARK-18160: -- Summary: SparkContext.addFile doesn't work in yarn-cluster mode Key: SPARK-18160 URL: https://issues.apache.org/jira/browse/SPARK-18160 Project: Spark Issue Type

[jira] [Resolved] (SPARK-18109) Log instrumentation in GMM

[ https://issues.apache.org/jira/browse/SPARK-18109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-18109. - Resolution: Fixed Fix Version/s: 2.1.0 > Log instrumentation in GMM >

[jira] [Created] (SPARK-18159) Stand-alone cluster, supervised app: restart of worker hosting the driver causes app to run twice

Stephan Kepser created SPARK-18159: -- Summary: Stand-alone cluster, supervised app: restart of worker hosting the driver causes app to run twice Key: SPARK-18159 URL: https://issues.apache.org/jira/browse/SPARK-18

[jira] [Commented] (SPARK-18150) Spark 2.* failes to create partitions for avro files

[ https://issues.apache.org/jira/browse/SPARK-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614587#comment-15614587 ] Sunil Kumar commented on SPARK-18150: - I am using Spark : 2.0.0.24 and spark-avro :

[jira] [Updated] (SPARK-18150) Spark 2.* failes to create partitions for avro files

[ https://issues.apache.org/jira/browse/SPARK-18150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Kumar updated SPARK-18150: Description: I am using Apache Spark 2.0.1 for processing the Grid HDFS Avro file, however I don't

[jira] [Assigned] (SPARK-18124) Implement watermarking for handling late data

[ https://issues.apache.org/jira/browse/SPARK-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-18124: Assignee: Michael Armbrust (was: Tathagata Das) > Implement watermarking for hand

[jira] [Commented] (SPARK-17055) add labelKFold to CrossValidator

[ https://issues.apache.org/jira/browse/SPARK-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15614576#comment-15614576 ] Rémi Delassus commented on SPARK-17055: --- I had an issue that could be solved by thi

[jira] [Created] (SPARK-18158) Submit app in standalone cluster mode supervised with HA: all masters have to be up and running

Stephan Kepser created SPARK-18158: -- Summary: Submit app in standalone cluster mode supervised with HA: all masters have to be up and running Key: SPARK-18158 URL: https://issues.apache.org/jira/browse/SPARK-1815

[jira] [Created] (SPARK-18151) CLONE - MetadataLog should support purging old logs

Sunil Kumar created SPARK-18151: --- Summary: CLONE - MetadataLog should support purging old logs Key: SPARK-18151 URL: https://issues.apache.org/jira/browse/SPARK-18151 Project: Spark Issue Type:

[jira] [Created] (SPARK-18154) CLONE - Change Source API so that sources do not need to keep unbounded state

Sunil Kumar created SPARK-18154: --- Summary: CLONE - Change Source API so that sources do not need to keep unbounded state Key: SPARK-18154 URL: https://issues.apache.org/jira/browse/SPARK-18154 Project:

[jira] [Created] (SPARK-18155) CLONE - HDFSMetadataLog should not leak CRC files

Sunil Kumar created SPARK-18155: --- Summary: CLONE - HDFSMetadataLog should not leak CRC files Key: SPARK-18155 URL: https://issues.apache.org/jira/browse/SPARK-18155 Project: Spark Issue Type: S

[jira] [Created] (SPARK-18152) CLONE - FileStreamSource should not track the list of seen files indefinitely

Sunil Kumar created SPARK-18152: --- Summary: CLONE - FileStreamSource should not track the list of seen files indefinitely Key: SPARK-18152 URL: https://issues.apache.org/jira/browse/SPARK-18152 Project:

[jira] [Created] (SPARK-18150) Spark 2.* failes to create partitions for avro files

Sunil Kumar created SPARK-18150: --- Summary: Spark 2.* failes to create partitions for avro files Key: SPARK-18150 URL: https://issues.apache.org/jira/browse/SPARK-18150 Project: Spark Issue Typ

[jira] [Created] (SPARK-18157) CLONE - Support purging aged file entry for FileStreamSource metadata log

Sunil Kumar created SPARK-18157: --- Summary: CLONE - Support purging aged file entry for FileStreamSource metadata log Key: SPARK-18157 URL: https://issues.apache.org/jira/browse/SPARK-18157 Project: Spar

[jira] [Created] (SPARK-18156) CLONE - StreamExecution should discard unneeded metadata

Sunil Kumar created SPARK-18156: --- Summary: CLONE - StreamExecution should discard unneeded metadata Key: SPARK-18156 URL: https://issues.apache.org/jira/browse/SPARK-18156 Project: Spark Issue

[jira] [Created] (SPARK-18153) CLONE - Ability to remove old metadata for structure streaming MetadataLog

Sunil Kumar created SPARK-18153: --- Summary: CLONE - Ability to remove old metadata for structure streaming MetadataLog Key: SPARK-18153 URL: https://issues.apache.org/jira/browse/SPARK-18153 Project: Spa

  1   2   >