[jira] [Updated] (SPARK-13482) `spark.storage.memoryMapThreshold` has two kind of the value.

2016-02-24 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-13482: - Description: `spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024

[jira] [Updated] (SPARK-13482) `spark.storage.memoryMapThreshold` has two kind of the value.

2016-02-24 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-13482: - Description: `spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024

[jira] [Updated] (SPARK-13482) `spark.storage.memoryMapThreshold` has two kind of the value.

2016-02-24 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-13482: - Description: `spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024

[jira] [Updated] (SPARK-13482) `spark.storage.memoryMapThreshold` has two kind of the value.

2016-02-24 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-13482: - Description: `spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024

[jira] [Updated] (SPARK-13482) `spark.storage.memoryMapThreshold` has two kind of the value.

2016-02-24 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-13482: - Attachment: 2016-02-25_10-41-37.jpg > `spark.storage.memoryMapThreshold` has two kind of the

[jira] [Created] (SPARK-13482) `spark.storage.memoryMapThreshold` has two kind of the value.

2016-02-24 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-13482: Summary: `spark.storage.memoryMapThreshold` has two kind of the value. Key: SPARK-13482 URL: https://issues.apache.org/jira/browse/SPARK-13482 Project: Spark

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-02-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149810#comment-15149810 ] SaintBacchus commented on SPARK-12316: -- [~tgraves] The function of listFilesSorted will not throw

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-02-15 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148023#comment-15148023 ] SaintBacchus commented on SPARK-12316: -- [~tgraves] The application would not hit the same condition

[jira] [Updated] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-01-07 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-12316: - Description: When application end, AM will clean the staging dir. But if the driver trigger to

[jira] [Created] (SPARK-12672) Streaming batch ui can't be opened in jobs page in yarn mode.

2016-01-06 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-12672: Summary: Streaming batch ui can't be opened in jobs page in yarn mode. Key: SPARK-12672 URL: https://issues.apache.org/jira/browse/SPARK-12672 Project: Spark

[jira] [Updated] (SPARK-12672) Streaming batch ui can't be opened in jobs page in yarn mode.

2016-01-06 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-12672: - Attachment: 2016-1-6 15-57-05.jpg 2016-1-6 15-56-26.jpg > Streaming batch ui

[jira] [Created] (SPARK-12523) Support long-running of the Spark On HBase and hive meta store.

2015-12-25 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-12523: Summary: Support long-running of the Spark On HBase and hive meta store. Key: SPARK-12523 URL: https://issues.apache.org/jira/browse/SPARK-12523 Project: Spark

[jira] [Updated] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2015-12-24 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-12316: - Attachment: 20151210045533.jpg 20151210045149.jpg > Stack overflow with endless

[jira] [Created] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2015-12-13 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-12316: Summary: Stack overflow with endless call of `Delegation token thread` when application end. Key: SPARK-12316 URL: https://issues.apache.org/jira/browse/SPARK-12316

[jira] [Updated] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2015-12-13 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-12316: - Description: When application end, AM will clean the staging dir. But if the driver trigger to

[jira] [Created] (SPARK-11043) Hive Thrift Server will log warn "Couldn't find log associated with operation handle"

2015-10-09 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-11043: Summary: Hive Thrift Server will log warn "Couldn't find log associated with operation handle" Key: SPARK-11043 URL: https://issues.apache.org/jira/browse/SPARK-11043

[jira] [Updated] (SPARK-11043) Hive Thrift Server will log warn "Couldn't find log associated with operation handle"

2015-10-09 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-11043: - Description: The warnning log is below: {code:title=Warnning Log|borderStyle=solid} 15/10/09

[jira] [Created] (SPARK-11000) Derby have booted the database twice in yarn security mode.

2015-10-08 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-11000: Summary: Derby have booted the database twice in yarn security mode. Key: SPARK-11000 URL: https://issues.apache.org/jira/browse/SPARK-11000 Project: Spark

[jira] [Commented] (SPARK-11000) Derby have booted the database twice in yarn security mode.

2015-10-08 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948138#comment-14948138 ] SaintBacchus commented on SPARK-11000: -- Very similar with it but this is in yarn security mode. >

[jira] [Issue Comment Deleted] (SPARK-11000) Derby have booted the database twice in yarn security mode.

2015-10-08 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-11000: - Comment: was deleted (was: Very similar with it but this is in yarn security mode.) > Derby

[jira] [Updated] (SPARK-10473) EventLog will loss message in the long-running security application

2015-09-29 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-10473: - Affects Version/s: 1.6.0 > EventLog will loss message in the long-running security application >

[jira] [Closed] (SPARK-10755) Set the driver also update the token for long-running application

2015-09-25 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus closed SPARK-10755. Resolution: Not A Problem It's not a good idea to do so, since the hadoop rpc can automatically

[jira] [Created] (SPARK-10786) SparkSQLCLIDriver should take the whole statement to generate the CommandProcessor

2015-09-23 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-10786: Summary: SparkSQLCLIDriver should take the whole statement to generate the CommandProcessor Key: SPARK-10786 URL: https://issues.apache.org/jira/browse/SPARK-10786

[jira] [Created] (SPARK-10766) Add some configurations for the client process in yarn-cluster mode.

2015-09-22 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-10766: Summary: Add some configurations for the client process in yarn-cluster mode. Key: SPARK-10766 URL: https://issues.apache.org/jira/browse/SPARK-10766 Project: Spark

[jira] [Created] (SPARK-10755) Set the driver also update the token for long-running application

2015-09-22 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-10755: Summary: Set the driver also update the token for long-running application Key: SPARK-10755 URL: https://issues.apache.org/jira/browse/SPARK-10755 Project: Spark

[jira] [Created] (SPARK-10473) EventLog will loss message in the long-running security application

2015-09-07 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-10473: Summary: EventLog will loss message in the long-running security application Key: SPARK-10473 URL: https://issues.apache.org/jira/browse/SPARK-10473 Project: Spark

[jira] [Commented] (SPARK-9091) Add the codec interface to Text DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629483#comment-14629483 ] SaintBacchus commented on SPARK-9091: - [~srowen] Sorry for forgetting to change the

[jira] [Updated] (SPARK-9091) Add the codec interface to Text DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Description: Since the RDD has the function *saveAsTextFile* which can use *CompressionCodec* to

[jira] [Updated] (SPARK-9091) Add the codec interface to Text DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Description: Since the RDD has the function *saveAsTextFile* which can use *CompressionCodec* to

[jira] [Updated] (SPARK-9091) Add the codec interface to Text DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Summary: Add the codec interface to Text DStream. (was: Add the codec interface to DStream.) Add

[jira] [Updated] (SPARK-9091) Add the codec interface to DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Priority: Minor (was: Major) Add the codec interface to DStream.

[jira] [Updated] (SPARK-9091) Add the codec interface to DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Issue Type: Improvement (was: Bug) Add the codec interface to DStream.

[jira] [Updated] (SPARK-9091) Add the codec interface to Text DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Description: Since the RDD has the function *saveAsTextFile* which can use *CompressionCodec* to

[jira] [Updated] (SPARK-9091) Add the codec interface to Text DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Description: Since the RDD has the function *saveAsTextFile* which can use *CompressionCodec* to

[jira] [Commented] (SPARK-9091) Add the codec interface to Text DStream.

2015-07-16 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629639#comment-14629639 ] SaintBacchus commented on SPARK-9091: - [~sowen] I agree user can design the output by

[jira] [Updated] (SPARK-9091) Add the codec interface to DStream.

2015-07-15 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-9091: Description: Add description later. Add the codec interface to DStream.

[jira] [Created] (SPARK-9091) Add the codec interface to DStream.

2015-07-15 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-9091: --- Summary: Add the codec interface to DStream. Key: SPARK-9091 URL: https://issues.apache.org/jira/browse/SPARK-9091 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-8839) Thrift Sever will throw `java.util.NoSuchElementException: key not found` exception when many clients connect it

2015-07-06 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8839: --- Summary: Thrift Sever will throw `java.util.NoSuchElementException: key not found` exception when many clients connect it Key: SPARK-8839 URL:

[jira] [Created] (SPARK-8820) Add a configuration to set the checkpoint directory for convenience.

2015-07-03 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8820: --- Summary: Add a configuration to set the checkpoint directory for convenience. Key: SPARK-8820 URL: https://issues.apache.org/jira/browse/SPARK-8820 Project: Spark

[jira] [Updated] (SPARK-8755) Streaming application from checkpoint will fail to load in security mode.

2015-07-01 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8755: Description: If the user set *spark.yarn.principal* and *spark.yarn.keytab* , he does not need

[jira] [Created] (SPARK-8755) Streaming application from checkpoint will fail to load in security mode.

2015-07-01 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8755: --- Summary: Streaming application from checkpoint will fail to load in security mode. Key: SPARK-8755 URL: https://issues.apache.org/jira/browse/SPARK-8755 Project: Spark

[jira] [Updated] (SPARK-8687) Spark on yarn-client mode can't send `spark.yarn.credentials.file` to executor.

2015-06-29 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8687: Summary: Spark on yarn-client mode can't send `spark.yarn.credentials.file` to executor. (was:

[jira] [Created] (SPARK-8688) Hadoop Configuration has to disable client cache when writing or reading delegation tokens.

2015-06-28 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8688: --- Summary: Hadoop Configuration has to disable client cache when writing or reading delegation tokens. Key: SPARK-8688 URL: https://issues.apache.org/jira/browse/SPARK-8688

[jira] [Created] (SPARK-8687) Spark on yarn can't send `spark.yarn.credentials.file` to executor.

2015-06-28 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8687: --- Summary: Spark on yarn can't send `spark.yarn.credentials.file` to executor. Key: SPARK-8687 URL: https://issues.apache.org/jira/browse/SPARK-8687 Project: Spark

[jira] [Closed] (SPARK-8163) CheckPoint mechanism did not work well when error happened in big streaming

2015-06-25 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus closed SPARK-8163. --- Resolution: Not A Problem Mistake the prolem. CheckPoint mechanism did not work well when error

[jira] [Created] (SPARK-8619) Can't find the keytab file when recovering the streaming application.

2015-06-24 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8619: --- Summary: Can't find the keytab file when recovering the streaming application. Key: SPARK-8619 URL: https://issues.apache.org/jira/browse/SPARK-8619 Project: Spark

[jira] [Updated] (SPARK-8367) ReliableKafka will loss data when `spark.streaming.blockInterval` was 0

2015-06-14 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8367: Description: {code:title=BlockGenerator.scala|borderStyle=solid} /** Change the buffer to which

[jira] [Updated] (SPARK-8367) ReliableKafka will loss data when `spark.streaming.blockInterval` was 0

2015-06-14 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8367: Description: {code:title=BlockGenerator.scala|borderStyle=solid} /** Change the buffer to which

[jira] [Updated] (SPARK-8367) ReliableKafka will loss data when `spark.streaming.blockInterval` was 0

2015-06-14 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8367: Description: {code:title=BlockGenerator.scala|borderStyle=solid} /** Change the buffer to which

[jira] [Updated] (SPARK-8367) ReliableKafka will loss data when `spark.streaming.blockInterval` was 0

2015-06-14 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8367: Description: {code:title=BlockGenerator.scala|borderStyle=solid} /** Change the buffer to which

[jira] [Reopened] (SPARK-8163) CheckPoint mechanism did not work well when error happened in big streaming

2015-06-08 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus reopened SPARK-8163: - CheckPoint mechanism did not work well when error happened in big streaming

[jira] [Commented] (SPARK-8163) CheckPoint mechanism did not work well when error happened in big streaming

2015-06-08 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578161#comment-14578161 ] SaintBacchus commented on SPARK-8163: - Hi [~sowen] all the description was the problem

[jira] [Created] (SPARK-8163) CheckPoint mechanism did not work well when error happened in big streaming

2015-06-08 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8163: --- Summary: CheckPoint mechanism did not work well when error happened in big streaming Key: SPARK-8163 URL: https://issues.apache.org/jira/browse/SPARK-8163 Project:

[jira] [Updated] (SPARK-8119) Spark will set total executor when some executors fail.

2015-06-04 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-8119: Description: DynamicAllocation will set the total executor to a little number when it wants to

[jira] [Created] (SPARK-8119) Spark will set total executor when some executors fail.

2015-06-04 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-8119: --- Summary: Spark will set total executor when some executors fail. Key: SPARK-8119 URL: https://issues.apache.org/jira/browse/SPARK-8119 Project: Spark Issue

[jira] [Commented] (SPARK-7942) Receiver's life cycle is inconsistent with streaming job.

2015-06-03 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572212#comment-14572212 ] SaintBacchus commented on SPARK-7942: - [~tdas] Now I'm not clear about how to deal

[jira] [Created] (SPARK-7942) Receiver's life cycle is inconsistent with streaming job.

2015-05-29 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-7942: --- Summary: Receiver's life cycle is inconsistent with streaming job. Key: SPARK-7942 URL: https://issues.apache.org/jira/browse/SPARK-7942 Project: Spark Issue

[jira] [Commented] (SPARK-7942) Receiver's life cycle is inconsistent with streaming job.

2015-05-29 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564411#comment-14564411 ] SaintBacchus commented on SPARK-7942: - [~tdas] Should we add a logic to shut down the

[jira] [Updated] (SPARK-7942) Receiver's life cycle is inconsistent with streaming job.

2015-05-29 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-7942: Description: Streaming consider the receiver as a common spark job, thus if an error occurs in the

[jira] [Created] (SPARK-6955) Do not let Yarn Shuffle Server retry its server port.

2015-04-15 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-6955: --- Summary: Do not let Yarn Shuffle Server retry its server port. Key: SPARK-6955 URL: https://issues.apache.org/jira/browse/SPARK-6955 Project: Spark Issue

[jira] [Commented] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386324#comment-14386324 ] SaintBacchus commented on SPARK-6605: - Hi, [~tdas] can you have a look at this

[jira] [Created] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-6605: --- Summary: Same transformation in DStream leads to different result Key: SPARK-6605 URL: https://issues.apache.org/jira/browse/SPARK-6605 Project: Spark Issue

[jira] [Commented] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386610#comment-14386610 ] SaintBacchus commented on SPARK-6605: - Yeah, [~srowen] it's not a wrong answer but

[jira] [Commented] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386540#comment-14386540 ] SaintBacchus commented on SPARK-6605: - Hi [~srowen], my test code is this :

[jira] [Comment Edited] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387811#comment-14387811 ] SaintBacchus edited comment on SPARK-6605 at 3/31/15 1:54 AM: --

[jira] [Resolved] (SPARK-6605) Same transformation in DStream leads to different result

2015-03-30 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus resolved SPARK-6605. - Resolution: Won't Fix {{reduceByKeyAndWindow }} has two implementations and leads to two

[jira] [Created] (SPARK-6582) Support ssl for this AvroSink in Spark Streaming External

2015-03-28 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-6582: --- Summary: Support ssl for this AvroSink in Spark Streaming External Key: SPARK-6582 URL: https://issues.apache.org/jira/browse/SPARK-6582 Project: Spark Issue

[jira] [Created] (SPARK-6584) Provide ExecutorPrefixTaskLocation to support the rdd which can be aware of partition's executor location.

2015-03-28 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-6584: --- Summary: Provide ExecutorPrefixTaskLocation to support the rdd which can be aware of partition's executor location. Key: SPARK-6584 URL:

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

2015-03-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Affects Version/s: (was: 1.3.0) 1.4.0 Add a new transformation of rdd

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

2015-03-23 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

2015-03-23 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the

[jira] [Created] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

2015-03-23 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-6464: --- Summary: Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd Key: SPARK-6464 URL:

[jira] [Updated] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

2015-03-23 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6464: Description: Nowadays, the transformation *coalesce* was always used to expand or reduce the

[jira] [Updated] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6056: Description: No matter set the `preferDirectBufs` or limit the number of thread or not ,spark can

[jira] [Updated] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6056: Description: No matter set the `preferDirectBufs` or limit the number of thread or not ,spark can

[jira] [Updated] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6056: Description: No matter set the `preferDirectBufs` or limit the number of thread or not ,spark can

[jira] [Updated] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6056: Description: No matter set the `preferDirectBufs` or limit the number of thread or not ,spark can

[jira] [Comment Edited] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341957#comment-14341957 ] SaintBacchus edited comment on SPARK-6056 at 3/1/15 6:01 AM: -

[jira] [Commented] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341957#comment-14341957 ] SaintBacchus commented on SPARK-6056: - [~adav] Thx for comment. I can't understand

[jira] [Comment Edited] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341957#comment-14341957 ] SaintBacchus edited comment on SPARK-6056 at 3/1/15 6:03 AM: -

[jira] [Comment Edited] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341957#comment-14341957 ] SaintBacchus edited comment on SPARK-6056 at 3/1/15 6:04 AM: -

[jira] [Updated] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6056: Description: No matter set the `preferDirectBufs` or limit the number of thread or not ,spark can

[jira] [Updated] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-28 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6056: Description: No matter set the `preferDirectBufs` or limit the number of thread or not ,spark can

[jira] [Comment Edited] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-27 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339861#comment-14339861 ] SaintBacchus edited comment on SPARK-6056 at 2/27/15 8:13 AM: --

[jira] [Updated] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-27 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-6056: Description: No matter set the `preferDirectBufs` or limit the number of thread or not ,spark can

[jira] [Commented] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-27 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339861#comment-14339861 ] SaintBacchus commented on SPARK-6056: - Hi,[~adav] [~lianhuiwang][~zzcclp] I had read

[jira] [Created] (SPARK-6056) Unlimit offHeap memory use cause RM killing the container

2015-02-26 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-6056: --- Summary: Unlimit offHeap memory use cause RM killing the container Key: SPARK-6056 URL: https://issues.apache.org/jira/browse/SPARK-6056 Project: Spark Issue

[jira] [Created] (SPARK-5444) 'spark.blockManager.port' conflict in netty service

2015-01-27 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-5444: --- Summary: 'spark.blockManager.port' conflict in netty service Key: SPARK-5444 URL: https://issues.apache.org/jira/browse/SPARK-5444 Project: Spark Issue Type:

[jira] [Created] (SPARK-5065) BroadCast can still work after sc had been stopped.

2015-01-03 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-5065: --- Summary: BroadCast can still work after sc had been stopped. Key: SPARK-5065 URL: https://issues.apache.org/jira/browse/SPARK-5065 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5065) BroadCast can still work after sc had been stopped.

2015-01-03 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-5065: Description: Code as follow: {code:borderStyle=solid} val sc1 = new SparkContext val sc2 = new

[jira] [Created] (SPARK-4929) Yarn Client mode can not support the HA after the exitcode change

2014-12-22 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-4929: --- Summary: Yarn Client mode can not support the HA after the exitcode change Key: SPARK-4929 URL: https://issues.apache.org/jira/browse/SPARK-4929 Project: Spark

[jira] [Commented] (SPARK-4694) Long-run user thread(such as HiveThriftServer2) causes the 'process leak' in yarn-client mode

2014-12-03 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232724#comment-14232724 ] SaintBacchus commented on SPARK-4694: - Thanks for reply. [~vanzin] the problem is very

[jira] [Created] (SPARK-4694) Long-run user thread(such as HiveThriftServer2) causes the 'process leak' in yarn-client mode

2014-12-02 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-4694: --- Summary: Long-run user thread(such as HiveThriftServer2) causes the 'process leak' in yarn-client mode Key: SPARK-4694 URL: https://issues.apache.org/jira/browse/SPARK-4694

[jira] [Updated] (SPARK-4694) Long-run user thread(such as HiveThriftServer2) causes the 'process leak' in yarn-client mode

2014-12-02 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SaintBacchus updated SPARK-4694: Component/s: YARN Long-run user thread(such as HiveThriftServer2) causes the 'process leak' in

[jira] [Commented] (SPARK-4694) Long-run user thread(such as HiveThriftServer2) causes the 'process leak' in yarn-client mode

2014-12-02 Thread SaintBacchus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231440#comment-14231440 ] SaintBacchus commented on SPARK-4694: - The reason was that Yarn had reported the