Matthias, thank you for all the hard work preparing the 0.10.0 release notes. I've updated the website with the release notes at: http://systemml.apache.org/0.10.0-incubating/release_notes.html
Deron On Tue, Jun 14, 2016 at 12:39 AM, Matthias Boehm <[email protected]> wrote: > the sub-bullet "c - fixes backwards compatibility MRv1" also referred to > (1) guava dependency conflicts (for MRv1, the classpath has a very old > version), and (2) a couple of runtime changes such as SYSTEMML-513 (task > handling for multiple output committer which overwrites the default file > output committer). > > Regards, > Matthias > > [image: Inactive hide details for Berthold Reinwald---06/14/2016 12:18:09 > AM---this color Regards,]Berthold Reinwald---06/14/2016 12:18:09 > AM---this color Regards, > > From: Berthold Reinwald/Almaden/IBM@IBMUS > To: [email protected] > Date: 06/14/2016 12:18 AM > Subject: Re: Release notes for 0.10.0 > ------------------------------ > > > > this color > > > Regards, > Berthold Reinwald > IBM Almaden Research Center > office: (408) 927 2208; T/L: 457 2208 > e-mail: [email protected] > > > > From: Matthias Boehm/Almaden/IBM@IBMUS > To: [email protected] > Date: 06/13/2016 11:25 PM > Subject: Re: Release notes for 0.10.0 > > > > ok, here is a first draft of the release notes - please feel free to > extend or prune this: > > a) [SYSTEMML-377] Different internal implementation types of spark matrix > blocks > * Supported internal formats: MCSR (default), CSR, COO > * Automatic MCSR->CSR on Spark read/caching (for memory efficiency) > * Automatic MCSR->CSR on sparse update-in-place (avoid serialization) > > b) [SYSTEMML-554] Enhanced SystemML Frame support for JMLC API / CP > * SystemML frame data type, deeply integrated into compiler and runtime > * New builtin functions: transform, transformapply, transformencode, > transformdecode, transformmeta > * Supported operations: read/write, left/right indexing, casting, append, > transform/transformapply > > c) Framework compatibility / configuration > * [SYSTEMML-418] Version-specific Spark memory budgets (>=1.6, legacy) > * [SYSTEMML-158] Updated deprecated Hadoop properties > * [SYSTEMML-476] Version-specific MR configuration handling (MRv2, MRv1) > * Fixes for backwards compatibility to MRv1 (?? shouldn't this be in -158 > ??) > * New pass-through mapred/mapreduce configurations through SystemML-config > * [SYSTEMML-584/585] New thread-local configuration handling (compiler/dml > config) > > d) Deep learning support (experimental) > * [SYSTMEML-618] New DML-bodied function library for deep learning > * [SYSTEMML-540] New builtin singlenode operations: conv2d, maxpooling, > im2col, col2im, rotate > * New lenet-train DML script (staging) > > e) API/script usability > * [SYSTEMML-607/604/611] Parser error handling > * [SYSTEMML-506/508/544/577/649/651] Extended MLContext/JMLC APIs > * [SYSTEMML-625/626/632] Improved source statement handling (e.g., > imports, absolute paths) > * [SYSTEMML-617/631/654] Improved name space handling > * [SYSTEMML-240] Extended stats outputs for Spark > collect/broadcast/parallelize > * [SYSTEMML-495] SystemML configuration handling > * [SYSTEMML-209] Include algorithms in systemml jar > * [SYSTEMML-647/648] Deprecated castAsScalar, ppred > * [SYSTEMML-477] JSON meta data handling > * [SYSTEMML-294] Print matrix builtin function > * [SYSTEMML-296/676/670] Improved PyDML syntax: slicing, rand, cdf, elif > * [SYSTEMML-675] Support for negative for/parfor loop increments > > f) New fused physical operators > * [SYSTEMML-488] Fused wdivmm w/ 4 operands > * [SYSTEMML-510] Fused wdivmm/wcemm w/ eps term > > g) Various performance features > * [SYSTEMML-427/512] Extended IPA (propagate scalar variables) > * [SYSTEMML-282] Extended update-in-place support for parfor intermediates > > * [SYSTEMML-552/399] Performance parallel binary/text readers (sort > sparse/nnz handling) > * [SYSTEMML-552/641] Cache-conscious operations: sparse-dense > wdivmm/wsloss, sparse-dense/sparse-sparse mm, dense-dense skinny rhs mm > * [SYSTEMML-641] Tuned special cases for block matrix multiplication: > e.g., mm w/ skinny rhs, colwise parallelization wide rhs > * [SYSTEMML-396/400] New/extended multithreaded operations: > cumsum/cummin/cummax/cumprod, transpose, and rand > * [SYSTEMML-510/694] New simplification rewrites: "pushdown > unaryagg-transpose", "simplify transpose-aggbin-binary chains", "reorder > minus-mmult", "canonicalize matmult-add-scalar", improved constant folding > (all unary) > * [SYSTEMML-653] Asynchronous bufferpool cleanup of evicted files / nio > file eviction > - MR iqm/quantile/median (qsort num reducers, qpick buffer size) > > h) DML script updates > * [SYSTEMML-536] New KNN algorithm (still staging) > * [SYSTEMML-534] Optional console output univariate statistics > * [SYSTMEML-494] GLM compiler warnings > * Robustness input/output handling L2SVM, MSVM, and Naive Bayes > * Random data generator for ALS > > i) Various fixes > - Dozens of fixes for diverse issues, fix pack for 0.9 release > > j) Build, documentation, examples > * [SYSTEMML-551] Enhanced jmlc javadoc > * [SYSTEMML-484] Build javadoc jar > * [SYSTEMML-468] Contributing to SystemML doc > * [SYSTEMML-517/524] DML language ref updates > * [SYSTEMML-498] Trouble shooting guide > * SystemML Jupyter/Zeppelin notebook examples > > > Regards, > Matthias > > Deron Eriksson ---06/13/2016 01:29:16 PM---Hi, We need to put together a > page of release notes for the 0.10.0 release > > From: Deron Eriksson <[email protected]> > To: [email protected] > Date: 06/13/2016 01:29 PM > Subject: Release notes for 0.10.0 > > > > Hi, > > We need to put together a page of release notes for the 0.10.0 release > (similar to http://systemml.apache.org/0.9.0-incubating/release_notes.html > ). > Could SystemML committers and contributors please respond to this thread > with what they feel are the major updates/improvements in this release? > > Thanks! > Deron > > > > > > > >
