The Apache SystemML team is pleased to announce the release of Apache SystemML version 0.9.0-incubating. This is the first release as an Apache project.
Apache SystemML provides declarative large-scale machine learning (ML) that aims at flexible specification of ML algorithms and automatic generation of hybrid runtime plans ranging from single-node, in-memory computations, to distributed computations on Apache Hadoop MapReduce and Apache Spark. Extensive updates have been made to the release in several areas. These include APIs, data ingestion, optimizations, language and runtime operators, new algorithms, testing, and online documentation. *APIs* Improvements to MLContext and to MLPipeline wrappers *Data Ingestion* Data conversion utilities (from RDDs and DataFrames) Data transformations on raw data sets *Optimizations* Extensions to compilation chain, including IPA Improvements to parfor Improved execution of concurrent Spark jobs New rewrites, including eager RDD caching and repartitioning Improvements to buffer pool caching Partitioning-preserving operations On-demand creation of SparkContext Efficient use of RDD checkpointing *Language and Runtime Operators* New matrix multiplication operators (e.g., ZipMM) New multi-threaded readers and operators Extended aggregation-outer operations for different relational operators Sample capability *New Algorithms* Alternating Least Squares (Conjugate Gradient) Cubic Splines (Conjugate Gradient and Direct Solve) *Testing* PyDML algorithm tests Test suite refactoring Improvements to performance tests *Online Documentation* GitHub README Quick Start Guide DML and PyDML Programming Guide MLContext Programming Guide Algorithms Reference DML Language Reference Debugger Guide To download the distribution, please go to : http://systemml.apache.org/ The Apache SystemML Team --- Apache SystemML is an effort undergoing Incubation <https://incubator.apache.org/index.html> at The Apache Software Foundation (ASF), sponsored by the Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
