sanha opened a new pull request #85: [NEMO-172] Implement one partition per one element partitioner URL: https://github.com/apache/incubator-nemo/pull/85 JIRA: [NEMO-##: TITLE](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-##) **Major changes:** - Implement `IncrementPartitioner` which assigns a dedicated key for each element - Flush partitions which have dedicated key to disk in large shuffle optimization - Remove redundant (de)compression before and after the vertex having `RelayTransform` in large shuffle optimization **Minor changes to note:** - `None` value for (de)compression is added - `Block#commitPartitions` method is added to commit all un-committed partitions **Tests for the changes:** - `BlockTest` is implemented - Existing IT cases running large shuffle optimization also cover this change **Other comments:** - N/A resolves [NEMO-##](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-##)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
