[jira] [Commented] (SPARK-5204) Column case need to be consistent with Hive

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273417#comment-14273417 ] Apache Spark commented on SPARK-5204: - User 'OopsOutOfMemory' has created a pull

[jira] [Created] (SPARK-5204) Column case need to be consistent with Hive

2015-01-12 Thread shengli (JIRA)
shengli created SPARK-5204: -- Summary: Column case need to be consistent with Hive Key: SPARK-5204 URL: https://issues.apache.org/jira/browse/SPARK-5204 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-5203) union with different decimal type report error

2015-01-12 Thread guowei (JIRA)
guowei created SPARK-5203: - Summary: union with different decimal type report error Key: SPARK-5203 URL: https://issues.apache.org/jira/browse/SPARK-5203 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5203) union with different decimal type report error

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273404#comment-14273404 ] Apache Spark commented on SPARK-5203: - User 'guowei2' has created a pull request for

[jira] [Commented] (SPARK-5164) YARN | Spark job submits from windows machine to a linux YARN cluster fail

2015-01-12 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273631#comment-14273631 ] Kousuke Saruta commented on SPARK-5164: --- This ticket is a duplication of SPARK-1825

[jira] [Commented] (SPARK-5019) Update GMM API to use MultivariateGaussian

2015-01-12 Thread Travis Galoppo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273565#comment-14273565 ] Travis Galoppo commented on SPARK-5019: --- [~lewuathe] Are you still interested in

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273560#comment-14273560 ] Shixiong Zhu commented on SPARK-5124: - {quote} 1. Let's not rely on the property of

[jira] [Commented] (SPARK-5012) Python API for Gaussian Mixture Model

2015-01-12 Thread Meethu Mathew (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273561#comment-14273561 ] Meethu Mathew commented on SPARK-5012: -- I added a new class GaussianMixtureModel in

[jira] [Closed] (SPARK-5204) Column case need to be consistent with Hive

2015-01-12 Thread shengli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shengli closed SPARK-5204. -- Resolution: Not a Problem Column case need to be consistent with Hive

[jira] [Commented] (SPARK-4859) Improve StreamingListenerBus

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273535#comment-14273535 ] Apache Spark commented on SPARK-4859: - User 'zsxwing' has created a pull request for

[jira] [Commented] (SPARK-1405) parallel Latent Dirichlet Allocation (LDA) atop of spark in MLlib

2015-01-12 Thread Valeriy Avanesov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273551#comment-14273551 ] Valeriy Avanesov commented on SPARK-1405: - [~josephkb], I've read your proposal

[jira] [Created] (SPARK-5205) Inconsistent behaviour between Streaming job and others, when click kill link in WebUI

2015-01-12 Thread uncleGen (JIRA)
uncleGen created SPARK-5205: --- Summary: Inconsistent behaviour between Streaming job and others, when click kill link in WebUI Key: SPARK-5205 URL: https://issues.apache.org/jira/browse/SPARK-5205 Project:

[jira] [Commented] (SPARK-5205) Inconsistent behaviour between Streaming job and others, when click kill link in WebUI

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273759#comment-14273759 ] Apache Spark commented on SPARK-5205: - User 'uncleGen' has created a pull request for

[jira] [Created] (SPARK-5207) StandardScalerModel mean and variance re-use

2015-01-12 Thread Octavian Geagla (JIRA)
Octavian Geagla created SPARK-5207: -- Summary: StandardScalerModel mean and variance re-use Key: SPARK-5207 URL: https://issues.apache.org/jira/browse/SPARK-5207 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-5078) Allow setting Akka host name from env vars

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5078. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Allow setting Akka

[jira] [Comment Edited] (SPARK-2584) Do not mutate block storage level on the UI

2015-01-12 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273877#comment-14273877 ] Ilya Ganelin edited comment on SPARK-2584 at 1/12/15 7:08 PM: --

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274019#comment-14274019 ] Josh Rosen commented on SPARK-4879: --- I think that part of the reproduction issues that I

[jira] [Resolved] (SPARK-5102) CompressedMapStatus needs to be registered with Kryo

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5102. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Fixed by:

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

2015-01-12 Thread Mohit Jaggi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273994#comment-14273994 ] Mohit Jaggi commented on SPARK-5097: Hi, This is Mohit Jaggi, author of

[jira] [Updated] (SPARK-5063) Raise more helpful errors when RDD actions or transformations are called inside of transformations

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5063: -- Description: Spark does not support nested RDDs or performing Spark actions inside of transformations;

[jira] [Resolved] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5172. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Sean Owen

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274239#comment-14274239 ] Patrick Wendell commented on SPARK-4923: Hey All, Sorry this has caused a

[jira] [Comment Edited] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Chip Senkbeil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274253#comment-14274253 ] Chip Senkbeil edited comment on SPARK-4923 at 1/12/15 10:07 PM:

[jira] [Commented] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274227#comment-14274227 ] Apache Spark commented on SPARK-4296: - User 'yhuai' has created a pull request for

[jira] [Comment Edited] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274239#comment-14274239 ] Patrick Wendell edited comment on SPARK-4923 at 1/12/15 9:58 PM:

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Chip Senkbeil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274253#comment-14274253 ] Chip Senkbeil commented on SPARK-4923: -- [~pwendell], I can definitely do that. Would

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274263#comment-14274263 ] Patrick Wendell commented on SPARK-4923: [~senkwich] definitely prefer github.

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Chip Senkbeil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274268#comment-14274268 ] Chip Senkbeil commented on SPARK-4923: -- Okay, I'll do that and update this JIRA once

[jira] [Closed] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

2015-01-12 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu closed SPARK-4004. -- Resolution: Won't Fix I'd close the PR as I saw some discussions in https://github.com/apache/spark/pull/3825

[jira] [Comment Edited] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

2015-01-12 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274299#comment-14274299 ] Nan Zhu edited comment on SPARK-4004 at 1/12/15 10:30 PM: -- I'd

[jira] [Closed] (SPARK-4667) Spillable can request more than twice its current memory from pool

2015-01-12 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Williams closed SPARK-4667. Resolution: Not a Problem Spillable can request more than twice its current memory from pool

[jira] [Updated] (SPARK-5207) StandardScalerModel mean and variance re-use

2015-01-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5207: - Issue Type: Improvement (was: Wish) StandardScalerModel mean and variance re-use

[jira] [Updated] (SPARK-5207) StandardScalerModel mean and variance re-use

2015-01-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5207: - Target Version/s: 1.3.0 StandardScalerModel mean and variance re-use

[jira] [Commented] (SPARK-5207) StandardScalerModel mean and variance re-use

2015-01-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274335#comment-14274335 ] Xiangrui Meng commented on SPARK-5207: -- [~ogeagla] I've assigned this ticket to you.

[jira] [Commented] (SPARK-4821) pyspark.mllib.rand docs not generated correctly

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274334#comment-14274334 ] Apache Spark commented on SPARK-4821: - User 'JoshRosen' has created a pull request for

[jira] [Updated] (SPARK-4348) pyspark.mllib.random conflicts with random module

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4348: -- Fix Version/s: 1.1.0 I've also fixed this in 1.1.2 by backporting the 1.2 patch:

[jira] [Closed] (SPARK-5056) Implementing Clara k-medoids clustering algorithm for large datasets

2015-01-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-5056. Resolution: Won't Fix Implementing Clara k-medoids clustering algorithm for large datasets

[jira] [Commented] (SPARK-5053) Test maintenance branches on Jenkins using SBT

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274472#comment-14274472 ] Josh Rosen commented on SPARK-5053: --- I fixed the {{branch-1.1}} PySpark issue in

[jira] [Resolved] (SPARK-5164) YARN | Spark job submits from windows machine to a linux YARN cluster fail

2015-01-12 Thread Aniket Bhatnagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Bhatnagar resolved SPARK-5164. - Resolution: Duplicate Duplicates and has similar findings to SPARK-1825. YARN | Spark

[jira] [Resolved] (SPARK-5049) ParquetTableScan always prepends the values of partition columns in output rows irrespective of the order of the partition columns in the original SELECT query

2015-01-12 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5049. - Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by

[jira] [Updated] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

2015-01-12 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-5147: - Target Version/s: 1.3.0, 1.2.1 write ahead logs from streaming receiver are not purged because

[jira] [Resolved] (SPARK-3910) ./python/pyspark/mllib/classification.py doctests fails with module name pollution

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3910. --- Resolution: Fixed Target Version/s: 1.2.0, 1.1.2 (was: 1.2.0) I backported Davies' 1.2 fix

[jira] [Updated] (SPARK-3433) Mima false-positives with @DeveloperAPI and @Experimental annotations

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-3433: -- Affects Version/s: 1.1.0 Fix Version/s: 1.1.2 I've backported this to {{branch-1.1}} in order

[jira] [Updated] (SPARK-5208) Add more documentation to Netty-based configs

2015-01-12 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-5208: -- Issue Type: Improvement (was: Bug) Add more documentation to Netty-based configs

[jira] [Created] (SPARK-5208) Add more documentation to Netty-based configs

2015-01-12 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-5208: - Summary: Add more documentation to Netty-based configs Key: SPARK-5208 URL: https://issues.apache.org/jira/browse/SPARK-5208 Project: Spark Issue Type:

[jira] [Created] (SPARK-5210) Support log rolling in EventLogger

2015-01-12 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-5210: - Summary: Support log rolling in EventLogger Key: SPARK-5210 URL: https://issues.apache.org/jira/browse/SPARK-5210 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-5211) Restore HiveMetastoreTypes.toDataType

2015-01-12 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5211: Priority: Critical (was: Major) Restore HiveMetastoreTypes.toDataType

[jira] [Created] (SPARK-5211) Restore HiveMetastoreTypes.toDataType

2015-01-12 Thread Yin Huai (JIRA)
Yin Huai created SPARK-5211: --- Summary: Restore HiveMetastoreTypes.toDataType Key: SPARK-5211 URL: https://issues.apache.org/jira/browse/SPARK-5211 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4924) Factor out code to launch Spark applications into a separate library

2015-01-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4924: - Target Version/s: 1.3.0 Factor out code to launch Spark applications into a separate library

[jira] [Updated] (SPARK-4924) Factor out code to launch Spark applications into a separate library

2015-01-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4924: - Affects Version/s: 1.2.0 Factor out code to launch Spark applications into a separate library

[jira] [Updated] (SPARK-4924) Factor out code to launch Spark applications into a separate library

2015-01-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4924: - Affects Version/s: (was: 1.2.0) 1.0.0 Factor out code to launch Spark

[jira] [Commented] (SPARK-5208) Add more documentation to Netty-based configs

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274489#comment-14274489 ] Apache Spark commented on SPARK-5208: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-5056) Implementing Clara k-medoids clustering algorithm for large datasets

2015-01-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274568#comment-14274568 ] Xiangrui Meng commented on SPARK-5056: -- This is along the same direction with our

[jira] [Commented] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

2015-01-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274685#comment-14274685 ] Saisai Shao commented on SPARK-5147: I'm working on this, the major part of work is

[jira] [Updated] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-1239: -- Assignee: (was: Josh Rosen) Don't fetch all map output statuses at each reducer during shuffles

[jira] [Resolved] (SPARK-4999) No need to put WAL-backed block into block manager by default

2015-01-12 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-4999. -- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 No need to put

[jira] [Commented] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274524#comment-14274524 ] Apache Spark commented on SPARK-4959: - User 'chenghao-intel' has created a pull

[jira] [Created] (SPARK-5209) Jobs fail with unexpected value exception in certain environments

2015-01-12 Thread Sven Krasser (JIRA)
Sven Krasser created SPARK-5209: --- Summary: Jobs fail with unexpected value exception in certain environments Key: SPARK-5209 URL: https://issues.apache.org/jira/browse/SPARK-5209 Project: Spark

[jira] [Commented] (SPARK-3821) Develop an automated way of creating Spark images (AMI, Docker, and others)

2015-01-12 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274535#comment-14274535 ] Nicholas Chammas commented on SPARK-3821: - That's correct. All those paths are

[jira] [Updated] (SPARK-4859) Refactor LiveListenerBus and StreamingListenerBus

2015-01-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4859: Description: [#4006|https://github.com/apache/spark/pull/4006] refactors LiveListenerBus and

[jira] [Commented] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

2015-01-12 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274683#comment-14274683 ] Tathagata Das commented on SPARK-5147: -- I think this is a critical bug. This should

[jira] [Commented] (SPARK-5206) Accumulators are not re-registered during recovering from checkpoint

2015-01-12 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274681#comment-14274681 ] Tathagata Das commented on SPARK-5206: -- Interesting observation! Can this be solved

[jira] [Updated] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

2015-01-12 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-5147: - Priority: Blocker (was: Major) write ahead logs from streaming receiver are not purged because

[jira] [Commented] (SPARK-5207) StandardScalerModel mean and variance re-use

2015-01-12 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274802#comment-14274802 ] DB Tsai commented on SPARK-5207: [~mengxr]'s idea sounds great for me. Specifically, let's

[jira] [Created] (SPARK-5212) Add support of schema-less transformation

2015-01-12 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-5212: -- Summary: Add support of schema-less transformation Key: SPARK-5212 URL: https://issues.apache.org/jira/browse/SPARK-5212 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5212) Add support of schema-less transformation

2015-01-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274845#comment-14274845 ] Apache Spark commented on SPARK-5212: - User 'viirya' has created a pull request for

[jira] [Resolved] (SPARK-5138) pyspark unable to infer schema of namedtuple

2015-01-12 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5138. - Resolution: Fixed Fix Version/s: 1.3.0 Issue resolved by pull request 3978

[jira] [Updated] (SPARK-5212) Add support of schema-less transformation

2015-01-12 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-5212: --- Issue Type: Improvement (was: Bug) Add support of schema-less transformation

[jira] [Commented] (SPARK-2584) Do not mutate block storage level on the UI

2015-01-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273871#comment-14273871 ] Andrew Or commented on SPARK-2584: -- When the in-memory cache is full, the RDD will be

[jira] [Commented] (SPARK-2584) Do not mutate block storage level on the UI

2015-01-12 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273877#comment-14273877 ] Ilya Ganelin commented on SPARK-2584: - Understood, I was looking at the UI for Spark

[jira] [Commented] (SPARK-2909) Indexing for SparseVector in pyspark

2015-01-12 Thread Manoj Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273880#comment-14273880 ] Manoj Kumar commented on SPARK-2909: [~josephkb] Sorry for spamming your inbox, but

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273885#comment-14273885 ] Reynold Xin commented on SPARK-5124: 1. Let's put that outside of this PR (either

[jira] [Commented] (SPARK-2584) Do not mutate block storage level on the UI

2015-01-12 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273762#comment-14273762 ] Ilya Ganelin commented on SPARK-2584: - Hi Andrew, question about this. When you say we

[jira] [Comment Edited] (SPARK-2584) Do not mutate block storage level on the UI

2015-01-12 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273762#comment-14273762 ] Ilya Ganelin edited comment on SPARK-2584 at 1/12/15 4:47 PM: --

[jira] [Updated] (SPARK-4859) Improve StreamingListenerBus

2015-01-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4859: - Affects Version/s: 1.0.0 Improve StreamingListenerBus

[jira] [Updated] (SPARK-4859) Improve StreamingListenerBus

2015-01-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4859: - Target Version/s: 1.3.0 Improve StreamingListenerBus

[jira] [Updated] (SPARK-4859) Improve StreamingListenerBus

2015-01-12 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-4859: - Priority: Major (was: Minor) Improve StreamingListenerBus

[jira] [Commented] (SPARK-3450) Enable specifiying the --jars CLI option multiple times

2015-01-12 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273924#comment-14273924 ] Marcelo Vanzin commented on SPARK-3450: --- [~pwendell] if your only concern is

[jira] [Commented] (SPARK-5206) Accumulators are not re-registered during recovering from checkpoint

2015-01-12 Thread vincent ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273927#comment-14273927 ] vincent ye commented on SPARK-5206: --- I guess that an Accumulator is registered to a

[jira] [Updated] (SPARK-5206) Accumulators are not re-registered during recovering from checkpoint

2015-01-12 Thread vincent ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vincent ye updated SPARK-5206: -- Description: I got exception as following while my streaming application restarts from crash from

[jira] [Resolved] (SPARK-5200) Disable web UI in Hive Thriftserver tests

2015-01-12 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-5200. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 1.1.2 Issue

[jira] [Updated] (SPARK-5102) CompressedMapStatus needs to be registered with Kryo

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5102: --- Target Version/s: 1.2.1 Assignee: Lianhui Wang CompressedMapStatus needs to be