[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865229#comment-15865229
]
Deenbandhu Agarwal commented on SPARK-15716:
What happened to this issue ?
> Memory usage
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330526#comment-15330526
]
Yan Chen commented on SPARK-15716:
--
The checkpoint path is on HDFS. The application is already shutdown,
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330497#comment-15330497
]
Shixiong Zhu commented on SPARK-15716:
--
I saw there were a lot of
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329990#comment-15329990
]
Yan Chen commented on SPARK-15716:
--
!http://i.imgur.com/gcm4Y6p.png!
I have difficulties uploading
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323019#comment-15323019
]
Sean Owen commented on SPARK-15716:
---
I agree you obviously have some issue in your system. The question
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323004#comment-15323004
]
Yan Chen commented on SPARK-15716:
--
So you agree that it is a problem. I also agree that it is still in
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322989#comment-15322989
]
Yan Chen commented on SPARK-15716:
--
Will do it in a few days when I get a chance.
> Memory usage of
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322990#comment-15322990
]
Sean Owen commented on SPARK-15716:
---
I wouldn't go that far. Without a pretty clear reproduction (or
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322982#comment-15322982
]
Shixiong Zhu commented on SPARK-15716:
--
If possible, could you provide your heap dump based on the
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322959#comment-15322959
]
Yan Chen commented on SPARK-15716:
--
I've marked it as invalid since it is a problem, but you think it is
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322950#comment-15322950
]
Yan Chen commented on SPARK-15716:
--
Does it mean if we can not identify what the problem is, we should
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322938#comment-15322938
]
Sean Owen commented on SPARK-15716:
---
[~yani.chen] the problem is you've described several different
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322942#comment-15322942
]
Sean Owen commented on SPARK-15716:
---
[~yani.chen] the problem is you've described several different
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322930#comment-15322930
]
Yan Chen commented on SPARK-15716:
--
Why it is "not a problem" even if it crashes the streaming process?
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322920#comment-15322920
]
Yan Chen commented on SPARK-15716:
--
[~srowen] Could I know why this issue is closed?
> Memory usage of
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322912#comment-15322912
]
Yan Chen commented on SPARK-15716:
--
Original problem comes from Hortonworks. We also tried to use
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317340#comment-15317340
]
Shixiong Zhu commented on SPARK-15716:
--
[~yani.chen] Are you using the Apache Spark or other
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316539#comment-15316539
]
Yan Chen commented on SPARK-15716:
--
I'm not sure what is causing that behavior, but that is what we saw.
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316325#comment-15316325
]
Sean Owen commented on SPARK-15716:
---
Hm, but this is unrelated to memory. An application that runs out
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316050#comment-15316050
]
Yan Chen commented on SPARK-15716:
--
We think the behavior of using local directory is due to this piece
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316047#comment-15316047
]
Yan Chen commented on SPARK-15716:
--
The memory usage goes up, and finally will make the whole streaming
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316045#comment-15316045
]
Yan Chen commented on SPARK-15716:
--
The same thing happens on 1.6.0, 1.6.1, 2.0.0 preview. However, if I
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315972#comment-15315972
]
Yan Chen commented on SPARK-15716:
--
Actually I tried in another environment with version 1.6.1, the same
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315895#comment-15315895
]
Sean Owen commented on SPARK-15716:
---
[~HalfVim] are you experiencing out of memory errors? it's normal
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315890#comment-15315890
]
Yan Chen commented on SPARK-15716:
--
We already tried 1.6.0, it does not happen. Could I know what
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314908#comment-15314908
]
Shixiong Zhu commented on SPARK-15716:
--
[~yani.chen] could you try 1.6.1 and see if this still
[
https://issues.apache.org/jira/browse/SPARK-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314190#comment-15314190
]
Yan Chen commented on SPARK-15716:
--
I also tried to run scala code adapted from the example code in
27 matches
Mail list logo
