[ https://issues.apache.org/jira/browse/SYSTEMML-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm closed SYSTEMML-1029. ------------------------------------ > Perftest: Data generator for descriptive statistics (slowdown due to update > in-place) > ------------------------------------------------------------------------------------- > > Key: SYSTEMML-1029 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1029 > Project: SystemML > Issue Type: Bug > Reporter: Matthias Boehm > Assignee: Matthias Boehm > Priority: Blocker > Fix For: SystemML 0.11 > > > The data generator for descriptive statistics shows severe performance > issues, due to update-in-place. In order to avoid buffer pool serialization > and evictions via shallow serialize, we represent sparse matrices which are > subject to update in-place in CSR. This works very well for row major updates > but has severe problems on column-wise updates (due to shifting of values). > There are three potential options: > * (1) Workaround: Change the data generator script from column-wise indexing > to row-wise indexing and transpose after the parfor loop. > * (2) Parfor rewrite: Disable update in-place, if sparse input, no parfor > update in-place, and column-wise access pattern. > * (3) Improved left indexing over CSR: Reallocate and shift the output once. > This is feasible as left indexing (specifically sparse-sparse and > dense-sparse) are the only operations which update the CSR representation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)