Hi Vasia, It seems to work OK up to about 50MB of input, and dies after that point. If i disable just this connected components step the rest of my program is happy with the full 1.5GB test dataset. It seems to be specifically limited to GraphAlgorithms in my case.
Do you know what the units are when it is saying Partition memory: 8388608? If it is bytes then it sounds like its using around 256MB per hash table of 32 partitions (which is then multiplied by number of task slots i guess). Can this number be configured do you know? Perhaps the windows version of the JVM is defaulting it to a lower value than on Linux? Thanks, Rob -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Memory-ran-out-error-when-running-connected-components-tp6888p6899.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
