Hi, I have implemented a custom partitioning algorithm to partition graphs in GraphX. Saving the partitioning graph (the edges) to HDFS creates separate files in the output folder with the number of files equal to the number of Partitions.
However, reading back the edges creates number of partitions that are equal to the number of blocks in the HDFS folder. Is there a way to instead create the same number of partitions as the number of files written to HDFS while preserving the original partitioning? I would like to avoid repartitioning. Thanks. - Bilal