[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390771#comment-14390771
]
Antony Mayi commented on SPARK-6334:
bq. btw. I see based on the sourcecode
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371122#comment-14371122
]
Antony Mayi commented on SPARK-6334:
bq. 2. Use less number of blocks, even you have
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364764#comment-14364764
]
Antony Mayi commented on SPARK-6334:
users: 12.5 millions
ratings: 3.3 billions
rank:
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366021#comment-14366021
]
Xiangrui Meng commented on SPARK-6334:
--
Couple suggestions before SPARK-5955 is
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363629#comment-14363629
]
Xiangrui Meng commented on SPARK-6334:
--
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362085#comment-14362085
]
Joseph K. Bradley commented on SPARK-6334:
--
I'm not sure about that. [~mengxr] ?
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361706#comment-14361706
]
Sean Owen commented on SPARK-6334:
--
Do you have 12TB of disk available to the YARN local
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361715#comment-14361715
]
Antony Mayi commented on SPARK-6334:
bq. What are the files that are filling up the
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361712#comment-14361712
]
Antony Mayi commented on SPARK-6334:
it is 12TB combined across all nodes available to
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361912#comment-14361912
]
Sean Owen commented on SPARK-6334:
--
Hm, is this a case where it's necessary to cut the
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361914#comment-14361914
]
Joseph K. Bradley commented on SPARK-6334:
--
I don't think persist() will
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361947#comment-14361947
]
Antony Mayi commented on SPARK-6334:
I had to increase the partitioning up to this
[
https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361951#comment-14361951
]
Antony Mayi commented on SPARK-6334:
btw. I see based on the sourcecode checkpointing
13 matches
Mail list logo
