[jira] [Comment Edited] (SPARK-1199) Type mismatch in Spark shell when using case class defined in shell

2014-04-25 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980756#comment-13980756 ] Piotr Kołaczkowski edited comment on SPARK-1199 at 4/25/14 7:26 AM

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-04 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated SPARK-1712: -- Attachment: executor.jstack.txt repl.jstack.txt

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-04 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated SPARK-1712: -- Description: {noformat} scala val collection = (1 to 100).map(i = (foo + i, i

[jira] [Updated] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-04 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated SPARK-1712: -- Description: {noformat} scala val collection = (1 to 100).map(i = (foo + i, i

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989876#comment-13989876 ] Piotr Kołaczkowski commented on SPARK-1712: --- This is log from shell: {noformat

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989882#comment-13989882 ] Piotr Kołaczkowski commented on SPARK-1712: --- There are some logs from

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990369#comment-13990369 ] Piotr Kołaczkowski commented on SPARK-1712: --- Hmm, it is not listed

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990383#comment-13990383 ] Piotr Kołaczkowski commented on SPARK-1712: --- Setting {{spark.akka.frameSize

[jira] [Commented] (SPARK-1712) ParallelCollectionRDD operations hanging forever without any error messages

2014-05-06 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990711#comment-13990711 ] Piotr Kołaczkowski commented on SPARK-1712: --- Do you need more info / help

[jira] [Created] (SPARK-1799) Add init script to the debian packaging

2014-05-11 Thread JIRA
Nicolas Lalevée created SPARK-1799: -- Summary: Add init script to the debian packaging Key: SPARK-1799 URL: https://issues.apache.org/jira/browse/SPARK-1799 Project: Spark Issue Type: New

[jira] [Created] (SPARK-1927) Implicits declared in companion objects not found in Spark shell

2014-05-26 Thread JIRA
Piotr Kołaczkowski created SPARK-1927: - Summary: Implicits declared in companion objects not found in Spark shell Key: SPARK-1927 URL: https://issues.apache.org/jira/browse/SPARK-1927 Project

[jira] [Created] (SPARK-2256) pyspark: RDD.take doesn't work ... sometimes ...

2014-06-24 Thread JIRA
Ángel Álvarez created SPARK-2256: Summary: pyspark: RDD.take doesn't work ... sometimes ... Key: SPARK-2256 URL: https://issues.apache.org/jira/browse/SPARK-2256 Project: Spark Issue Type

[jira] [Created] (SPARK-2418) Custom checkpointing with an external function as parameter

2014-07-09 Thread JIRA
András Barják created SPARK-2418: Summary: Custom checkpointing with an external function as parameter Key: SPARK-2418 URL: https://issues.apache.org/jira/browse/SPARK-2418 Project: Spark

[jira] [Commented] (SPARK-2418) Custom checkpointing with an external function as parameter

2014-07-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057361#comment-14057361 ] András Barják commented on SPARK-2418: -- A pull request for a simple implementation

[jira] [Commented] (SPARK-1981) Add AWS Kinesis streaming support

2014-07-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057872#comment-14057872 ] Urban Škudnik commented on SPARK-1981: -- Excellent, looking forward already! :) I'm

[jira] [Updated] (SPARK-2256) pyspark: RDD.take doesn't work ... sometimes ...

2014-07-17 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ángel Álvarez updated SPARK-2256: - Attachment: A_test.zip I've tried with different files and sizes ... but I can't figure out

[jira] [Commented] (SPARK-2256) pyspark: RDD.take doesn't work ... sometimes ...

2014-07-17 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065230#comment-14065230 ] Ángel Álvarez commented on SPARK-2256: -- I've tried using local and master spark

[jira] [Created] (SPARK-2573) This file make-distribution.sh has an error, please fix it

2014-07-18 Thread JIRA
王金子 created SPARK-2573: -- Summary: This file make-distribution.sh has an error, please fix it Key: SPARK-2573 URL: https://issues.apache.org/jira/browse/SPARK-2573 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2573) This file make-distribution.sh has an error, please fix it

2014-07-18 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 王金子 updated SPARK-2573: --- Component/s: Build Description: line 61: echo Error: '--with-hive' is no longer supported, use

[jira] [Updated] (SPARK-2753) Is it supposed --archives option in yarn cluster mode to uncompress file?

2014-07-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Manuel Abuín Mosquera updated SPARK-2753: -- Description: Hi all, this is my first sent issue, I googled

[jira] [Closed] (SPARK-2753) Is it supposed --archives option in yarn cluster mode to uncompress file?

2014-08-19 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Manuel Abuín Mosquera closed SPARK-2753. - Resolution: Not a Problem Is it supposed --archives option in yarn

[jira] [Updated] (SPARK-3245) spark insert into hbase class not serialize

2014-08-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘勇 updated SPARK-3245: -- Description: val result: org.apache.spark.rdd.RDD[(String, Int)] result.foreach(res ={ var put = new Put

[jira] [Updated] (SPARK-3245) spark insert into hbase class not serialize

2014-08-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘勇 updated SPARK-3245: -- Description: val result: org.apache.spark.rdd.RDD[(String, Int)] result.foreach(res ={ var put = new Put

[jira] [Commented] (SPARK-3245) spark insert into hbase class not serialize

2014-08-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111948#comment-14111948 ] 刘勇 commented on SPARK-3245: --- before this NotSerializableException class

[jira] [Closed] (SPARK-3245) spark insert into hbase class not serialize

2014-08-29 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘勇 closed SPARK-3245. - spark insert into hbase class not serialize Key: SPARK

[jira] [Commented] (SPARK-3232) Backport SPARK-3006 into branch-1.0

2014-10-01 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154737#comment-14154737 ] François Garillot commented on SPARK-3232: -- This is closed by https://github.com

[jira] [Commented] (SPARK-2256) pyspark: RDD.take doesn't work ... sometimes ...

2014-10-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157765#comment-14157765 ] Ángel Álvarez commented on SPARK-2256: -- It seems the problem has been solved in Spark

[jira] [Commented] (SPARK-2418) Custom checkpointing with an external function as parameter

2014-10-06 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160072#comment-14160072 ] András Barják commented on SPARK-2418: -- Hi, I would be happy if someone could comment

[jira] [Created] (SPARK-3954) promote the speed of convert files to RDDS

2014-10-15 Thread JIRA
宿荣全 created SPARK-3954: -- Summary: promote the speed of convert files to RDDS Key: SPARK-3954 URL: https://issues.apache.org/jira/browse/SPARK-3954 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-16 Thread JIRA
URL: https://issues.apache.org/jira/browse/SPARK-3967 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.1.0 Reporter: Christophe PRÉAUD Spark applications fail from time to time in yarn-cluster mode (but not in yarn

[jira] [Updated] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-3967: - Attachment: spark-1.1.0-yarn_cluster_tmpdir.patch Ensure that the temporary file which

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173566#comment-14173566 ] Christophe PRÉAUD commented on SPARK-3967: -- After investigating, it turns out

[jira] [Created] (SPARK-3989) Added the possibility to install Python packages via pip for pyspark directly from the ./spark_ec2 command

2014-10-17 Thread JIRA
Jan Zikeš created SPARK-3989: Summary: Added the possibility to install Python packages via pip for pyspark directly from the ./spark_ec2 command Key: SPARK-3989 URL: https://issues.apache.org/jira/browse/SPARK-3989

[jira] [Commented] (SPARK-3989) Added the possibility to install Python packages via pip for pyspark directly from the ./spark_ec2 command

2014-10-17 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174876#comment-14174876 ] Jan Zikeš commented on SPARK-3989: -- Implemented and sent pull request here: https

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD commented on SPARK-3967: -- Hi Ryan, Thanks for your help. You

[jira] [Created] (SPARK-4007) EOF exception to load from HDFS an JavaRDD

2014-10-20 Thread JIRA
Cristian Galán created SPARK-4007: - Summary: EOF exception to load from HDFS an JavaRDD Key: SPARK-4007 URL: https://issues.apache.org/jira/browse/SPARK-4007 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4007) EOF exception to load an JavaRDD from HDFS

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cristian Galán updated SPARK-4007: -- Priority: Major (was: Critical) Environment: hadoop-client-2.30 hadoop-hdfs-2.30 spark

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:20 PM

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:19 PM

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:20 PM

[jira] [Comment Edited] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176782#comment-14176782 ] Christophe PRÉAUD edited comment on SPARK-3967 at 10/20/14 12:21 PM

[jira] [Closed] (SPARK-4007) EOF exception to load an JavaRDD from HDFS

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cristian Galán closed SPARK-4007. - Resolution: Invalid Don't need fix because it's not a problem of Spark, the objects are saved

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

2014-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177040#comment-14177040 ] Christophe PRÉAUD commented on SPARK-3967: -- That's fine, thanks! Spark

[jira] [Commented] (SPARK-3954) Optimization to FileInputDStream

2014-10-23 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14181190#comment-14181190 ] 宿荣全 commented on SPARK-3954: Does someone take notice of this PR? Optimization

[jira] [Commented] (SPARK-3720) support ORC in spark sql

2014-11-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195287#comment-14195287 ] René X Parra commented on SPARK-3720: - [~zhazhan] should this JIRA ticket be closed

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-11-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195301#comment-14195301 ] René X Parra commented on SPARK-2883: - [~marmbrus] I see this was changed from Version

[jira] [Updated] (SPARK-3954) Optimization to FileInputDStream

2014-11-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 宿荣全 updated SPARK-3954: --- Description: about convert files to RDDS there are 3 loops with files sequence in spark source. loops files sequence

[jira] [Updated] (SPARK-3954) source code optimization

2014-11-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 宿荣全 updated SPARK-3954: --- Summary: source code optimization (was: Optimization to FileInputDStream) source code optimization

[jira] [Created] (SPARK-4243) Spark SQL SELECT COUNT DISTINCT optimization

2014-11-05 Thread JIRA
Bojan Kostić created SPARK-4243: --- Summary: Spark SQL SELECT COUNT DISTINCT optimization Key: SPARK-4243 URL: https://issues.apache.org/jira/browse/SPARK-4243 Project: Spark Issue Type

[jira] [Updated] (SPARK-1825) Windows Spark fails to work with Linux YARN

2014-11-11 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ángel Álvarez updated SPARK-1825: - Attachment: SPARK-1825.patch Is it really necessary to change the file ExecutorRunnableUtil.scala

[jira] [Commented] (SPARK-1825) Windows Spark fails to work with Linux YARN

2014-11-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219490#comment-14219490 ] Ángel Álvarez commented on SPARK-1825: -- I've had the following problems to make

[jira] [Comment Edited] (SPARK-1825) Windows Spark fails to work with Linux YARN

2014-11-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219490#comment-14219490 ] Ángel Álvarez edited comment on SPARK-1825 at 11/20/14 3:38 PM

[jira] [Commented] (SPARK-1867) Spark Documentation Error causes java.lang.IllegalStateException: unread block data

2014-11-21 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220736#comment-14220736 ] Michał Kosmulski commented on SPARK-1867: - I ran into the same issue (with Spark

[jira] [Commented] (SPARK-4637) spark-1.1.0 does not compile any more

2014-11-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227441#comment-14227441 ] Benjamin Cabé commented on SPARK-4637: -- bq. I did not find a different repository

[jira] [Created] (SPARK-4660) JavaSerializer uses wrong classloader

2014-11-30 Thread JIRA
Piotr Kołaczkowski created SPARK-4660: - Summary: JavaSerializer uses wrong classloader Key: SPARK-4660 URL: https://issues.apache.org/jira/browse/SPARK-4660 Project: Spark Issue Type

[jira] [Commented] (SPARK-3553) Spark Streaming app streams files that have already been streamed in an endless loop

2014-12-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231697#comment-14231697 ] Micael Capitão commented on SPARK-3553: --- I'm having that same issue running Spark

[jira] [Comment Edited] (SPARK-3553) Spark Streaming app streams files that have already been streamed in an endless loop

2014-12-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231697#comment-14231697 ] Micael Capitão edited comment on SPARK-3553 at 12/2/14 4:28 PM

[jira] [Comment Edited] (SPARK-3553) Spark Streaming app streams files that have already been streamed in an endless loop

2014-12-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231697#comment-14231697 ] Micael Capitão edited comment on SPARK-3553 at 12/2/14 4:30 PM

[jira] [Commented] (SPARK-3553) Spark Streaming app streams files that have already been streamed in an endless loop

2014-12-02 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231815#comment-14231815 ] Micael Capitão commented on SPARK-3553: --- I've already seen that post. It didn't work

[jira] [Commented] (SPARK-3553) Spark Streaming app streams files that have already been streamed in an endless loop

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232802#comment-14232802 ] Micael Capitão commented on SPARK-3553: --- I confirm the weird behaviour running

[jira] [Issue Comment Deleted] (SPARK-3553) Spark Streaming app streams files that have already been streamed in an endless loop

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micael Capitão updated SPARK-3553: -- Comment: was deleted (was: I confirm the weird behaviour running in HDFS too. I have the Spark

[jira] [Commented] (SPARK-3553) Spark Streaming app streams files that have already been streamed in an endless loop

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232801#comment-14232801 ] Micael Capitão commented on SPARK-3553: --- I confirm the weird behaviour running

[jira] [Created] (SPARK-4724) JavaNetworkWordCount.java has a wrong import

2014-12-03 Thread JIRA
Emre Sevinç created SPARK-4724: -- Summary: JavaNetworkWordCount.java has a wrong import Key: SPARK-4724 URL: https://issues.apache.org/jira/browse/SPARK-4724 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4724) JavaNetworkWordCount.java has a wrong import

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233053#comment-14233053 ] Emre Sevinç commented on SPARK-4724: Then how do I import

[jira] [Commented] (SPARK-4724) JavaNetworkWordCount.java has a wrong import

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233100#comment-14233100 ] Emre Sevinç commented on SPARK-4724: OK, now I see. What confused me was the word

[jira] [Resolved] (SPARK-4724) JavaNetworkWordCount.java has a wrong import

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emre Sevinç resolved SPARK-4724. Resolution: Not a Problem Not a Problem. It was a misunderstanding on my side

[jira] [Closed] (SPARK-4724) JavaNetworkWordCount.java has a wrong import

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Emre Sevinç closed SPARK-4724. -- JavaNetworkWordCount.java has a wrong import

[jira] [Created] (SPARK-4734) limit the file Dstream size for each batch

2014-12-03 Thread JIRA
宿荣全 created SPARK-4734: -- Summary: limit the file Dstream size for each batch Key: SPARK-4734 URL: https://issues.apache.org/jira/browse/SPARK-4734 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-4734) limit the file Dstream size for each batch

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 宿荣全 updated SPARK-4734: --- Description: Streaming scan new files form the HDFS and process those files in each batch process.Current streaming

[jira] [Updated] (SPARK-4734) [Streaming]limit the file Dstream size for each batch

2014-12-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 宿荣全 updated SPARK-4734: --- Summary: [Streaming]limit the file Dstream size for each batch (was: limit the file Dstream size for each batch

[jira] [Created] (SPARK-4748) PySpark can't read data in HDFS in YARN mode

2014-12-04 Thread JIRA
Sebastián Ramírez created SPARK-4748: Summary: PySpark can't read data in HDFS in YARN mode Key: SPARK-4748 URL: https://issues.apache.org/jira/browse/SPARK-4748 Project: Spark Issue

[jira] [Commented] (SPARK-4734) [Streaming]limit the file Dstream size for each batch

2014-12-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14235304#comment-14235304 ] 宿荣全 commented on SPARK-4734: [~srowen] I am very sorry that I can't describe the suggestion

[jira] [Resolved] (SPARK-4748) PySpark can't read data in HDFS in YARN mode

2014-12-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastián Ramírez resolved SPARK-4748. -- Resolution: Invalid I don't know what was happening, but once I restarted the cluster

[jira] [Created] (SPARK-4764) Ensure that files are fetched atomically

2014-12-05 Thread JIRA
Christophe PRÉAUD created SPARK-4764: Summary: Ensure that files are fetched atomically Key: SPARK-4764 URL: https://issues.apache.org/jira/browse/SPARK-4764 Project: Spark Issue Type

[jira] [Commented] (SPARK-4734) [Streaming]limit the file Dstream size for each batch

2014-12-05 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14236519#comment-14236519 ] 宿荣全 commented on SPARK-4734: [~srowen] [~srowen] I think that I still do not describe

[jira] [Updated] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 宿荣全 updated SPARK-4817: --- Summary: [streaming]Print the specified number of data and handle all of the elements in RDD (was: Print

[jira] [Created] (SPARK-4817) Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
宿荣全 created SPARK-4817: -- Summary: Print the specified number of data and handle all of the elements in RDD Key: SPARK-4817 URL: https://issues.apache.org/jira/browse/SPARK-4817 Project: Spark Issue

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14240934#comment-14240934 ] 宿荣全 commented on SPARK-4817: [~srowen] Ithink that this modification is not the same as [SPARK

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14240954#comment-14240954 ] 宿荣全 commented on SPARK-4817: Yes, with the other streaming's operate can be replaced

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14240955#comment-14240955 ] 宿荣全 commented on SPARK-4817: [~srowen] Yes, with the other streaming's operate can be replaced

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242090#comment-14242090 ] 宿荣全 commented on SPARK-4817: [~srowen] ' Neither prints the top elements. Did you mean first

[jira] [Comment Edited] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242177#comment-14242177 ] 宿荣全 edited comment on SPARK-4817 at 12/11/14 5:35 AM: -- [~srowen

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242177#comment-14242177 ] 宿荣全 commented on SPARK-4817: Always call foreachRDD, and operate on all of the RDD

[jira] [Comment Edited] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242177#comment-14242177 ] 宿荣全 edited comment on SPARK-4817 at 12/11/14 5:57 AM: -- [~srowen

[jira] [Comment Edited] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242177#comment-14242177 ] 宿荣全 edited comment on SPARK-4817 at 12/11/14 6:19 AM: -- [~srowen

[jira] [Comment Edited] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242177#comment-14242177 ] 宿荣全 edited comment on SPARK-4817 at 12/11/14 6:59 AM: -- [~srowen

[jira] [Comment Edited] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242177#comment-14242177 ] 宿荣全 edited comment on SPARK-4817 at 12/11/14 7:03 AM: -- [~srowen

[jira] [Comment Edited] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242177#comment-14242177 ] 宿荣全 edited comment on SPARK-4817 at 12/11/14 7:15 AM: -- [~srowen

[jira] [Commented] (SPARK-4817) [streaming]Print the specified number of data and handle all of the elements in RDD

2014-12-11 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243683#comment-14243683 ] 宿荣全 commented on SPARK-4817: [~srowen] I‘m sorry that didn't describe the problem clearly

[jira] [Commented] (SPARK-4660) JavaSerializer uses wrong classloader

2014-12-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14261016#comment-14261016 ] Piotr Kołaczkowski commented on SPARK-4660: --- https://github.com/apache/spark

[jira] [Issue Comment Deleted] (SPARK-4660) JavaSerializer uses wrong classloader

2014-12-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated SPARK-4660: -- Comment: was deleted (was: https://github.com/apache/spark/pull/3840) JavaSerializer

[jira] [Created] (SPARK-5026) PySpark rdd.randomSpit() is not documented

2014-12-30 Thread JIRA
Sebastián Ramírez created SPARK-5026: Summary: PySpark rdd.randomSpit() is not documented Key: SPARK-5026 URL: https://issues.apache.org/jira/browse/SPARK-5026 Project: Spark Issue Type

[jira] [Closed] (SPARK-5407) No 1.2 AMI available for ec2

2015-01-26 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Håkan Jonsson closed SPARK-5407. Resolution: Invalid Error on my side. No 1.2 AMI available for ec2

[jira] [Commented] (SPARK-3924) Upgrade to Akka version 2.3.7

2015-01-19 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-3924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282408#comment-14282408 ] François Garillot commented on SPARK-3924: -- [~helena_e] Please bump to 2.3.8

[jira] [Commented] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

2015-01-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280034#comment-14280034 ] François Garillot commented on SPARK-5147: -- I see. Thanks for your answers

[jira] [Commented] (SPARK-4357) Modify release publishing to work with Scala 2.11

2015-01-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280037#comment-14280037 ] François Garillot commented on SPARK-4357: -- Scala 2.11.5 [has been released|http

[jira] [Commented] (SPARK-1812) Support cross-building with Scala 2.11

2015-01-17 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281361#comment-14281361 ] François Garillot commented on SPARK-1812: -- Hem. Both issues are now closed

[jira] [Commented] (SPARK-5147) write ahead logs from streaming receiver are not purged because cleanupOldBlocks in WriteAheadLogBasedBlockHandler is never called

2015-01-14 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14276988#comment-14276988 ] François Garillot commented on SPARK-5147: -- 1. Yes, you're right I had forgotten

[jira] [Comment Edited] (SPARK-5265) Submitting applications on Standalone cluster controlled by Zookeeper forces to know active master

2015-02-13 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319965#comment-14319965 ] Wojciech Pituła edited comment on SPARK-5265 at 2/13/15 11:24 AM

[jira] [Closed] (SPARK-5026) PySpark rdd.randomSpit() is not documented

2015-02-17 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-5026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastián Ramírez closed SPARK-5026. You are right. It's OK now. Thanks. PySpark rdd.randomSpit() is not documented

  1   2   3   4   5   6   7   8   9   10   >