Hello! Because of deprecation of GraphX in Spark 4.x I have a question. Working on performance improvements in GraphFrames that is using GraphX under the hood, I found a way to improve the performance of the LabelPropagation algorithm in GraphX.
On my tests (LDBC graph "wiki-Talk", 2.3M vertices, 5M edges) it improves the performance from ~3500 seconds to ~50 seconds. The new solution is slightly increasing the average memory usage per iteration but also it is decreasing the peak memory usage overall (the 1st iteration of the current implementation). I'm ready to provide all the details and explanations, fill the Jira ticket, etc. But my main question is does GraphX accept patches or because of deprecation it is not considered anymore? Thanks in advance! Best regards, Sem --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
