[
https://issues.apache.org/jira/browse/SPARK-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784831#comment-16784831
]
Parth Gandhi commented on SPARK-26947:
--
[~srowen] Yes your suggestion to limit the vocab size
[
https://issues.apache.org/jira/browse/SPARK-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783858#comment-16783858
]
Sean Owen commented on SPARK-26947:
---
That doesn't sound "very big" but how big are the vectors you
[
https://issues.apache.org/jira/browse/SPARK-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783849#comment-16783849
]
Parth Gandhi commented on SPARK-26947:
--
[~srowen] for this particular case, k is set to 1.
[
https://issues.apache.org/jira/browse/SPARK-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782076#comment-16782076
]
Sean Owen commented on SPARK-26947:
---
How big is k? yes, you're going to run out of memory eventually
[
https://issues.apache.org/jira/browse/SPARK-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1665#comment-1665
]
Marco Gaido commented on SPARK-26947:
-
Cloud you also please provide the heap dump of the JVM? You
[
https://issues.apache.org/jira/browse/SPARK-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773457#comment-16773457
]
Parth Gandhi commented on SPARK-26947:
--
I am unable to attach the dummy dataset as the size of the
