Luciano, There is a copy/paste error pointing to http://wiki.apache.org/incubator/Nuvem.
Sent from my iPhone 6 Plus > On Oct 27, 2015, at 10:03 PM, Luciano Resende <luckbr1...@gmail.com> wrote: > > On Tue, Oct 27, 2015 at 9:52 PM, Luciano Resende <luckbr1...@gmail.com> > wrote: > >> >> After initial discussion, please vote on the acceptance of SystemML >> Project for incubation at the Apache Incubator. The full proposal is >> available at the end of this message and on the wiki at : >> >> https://wiki.apache.org/incubator/SystemML >> <http://wiki.apache.org/incubator/Nuvem> >> >> Please cast your votes: >> >> [ ] +1, bring SystemML into Incubator >> [ ] +0, I don't care either way >> [ ] -1, do not bring SystemML into Incubator, because... >> >> The vote is open for the next 72 hours and only votes from the >> Incubator PMC are binding. >> >> >> = SystemML = >> >> == Abstract == >> >> SystemML provides declarative large-scale machine learning (ML) that aims >> at flexible specification of ML algorithms and automatic generation of >> hybrid runtime plans ranging from single node, in-memory computations, to >> distributed computations on Apache Hadoop MapReduce and Apache Spark. ML >> algorithms are expressed in an R-like syntax, that includes linear algebra >> primitives, statistical functions, and ML-specific constructs. This >> high-level language significantly increases the productivity of data >> scientists as it provides (1) full flexibility in expressing custom >> analytics, and (2) data independence from the underlying input formats and >> physical data representations. Automatic optimization according to data >> characteristics such as distribution on the disk file system, and sparsity >> as well as processing characteristics in the distributed environment like >> number of nodes, CPU, memory per node, ensures both efficiency and >> scalability. >> >> == Proposal == >> >> The goal of SystemML is to create a commercial friendly, scalable and >> extensible machine learning framework for data scientists to create or >> extend machine learning algorithms using a declarative syntax. The machine >> learning framework enables data scientists to develop algorithms locally >> without the need of a distributed cluster, and scale up and scale out the >> execution of these algorithms to distributed Apache Hadoop MapReduce or >> Apache Spark clusters. >> >> == Background == >> >> SystemML started as a research project in the IBM Almaden Research Center >> around 2007 aiming to enable data scientists to develop machine learning >> algorithms independent of data and cluster characteristics. >> >> == Rationale == >> >> SystemML enables the specification of machine learning algorithms using a >> declarative machine learning (DML) language. DML includes linear algebra >> primitives, statistical functions, and additional constructs. This >> high-level language significantly increases the productivity of data >> scientists as it provides (1) full flexibility in expressing custom >> analytics and (2) data independence from the underlying input formats and >> physical data representations. >> >> SystemML computations can be executed in a variety of different modes. It >> supports single node in-memory computations and large-scale distributed >> cluster computations. This allows the user to quickly prototype new >> algorithms in local environments but automatically scale to large data >> sizes as well without changing the algorithm implementation. >> >> Algorithms specified in DML are dynamically compiled and optimized based >> on data and cluster characteristics using rule-based and cost-based >> optimization techniques. The optimizer automatically generates hybrid >> runtime execution plans ranging from in-memory single-node execution to >> distributed computations on Apache Spark or Apache Hadoop MapReduce. This >> ensures both efficiency and scalability. Automatic optimization reduces or >> eliminates the need to hand-tune distributed runtime execution plans and >> system configurations. >> >> == Initial Goals == >> >> The initial goals to move SystemML to the Apache Incubator is to broaden >> the community foster the contributions from data scientists to develop new >> machine learning algorithms and enhance the existing ones. Ultimately, this >> may lead to the creation of an industry standard in specifying machine >> learning algorithms. >> >> == Current Status == >> >> The initial code has been developed at the IBM Almaden Research Center in >> California and has recently been made available in GitHub under the Apache >> Software License 2.0. The project currently supports a single node (in >> memory computation) as well as distributed computations utilizing Apache >> Hadoop MapReduce or Apache Spark clusters. >> >> === Meritocracy === >> >> We plan to invest in supporting a meritocracy. We will discuss the >> requirements in an open forum. Several companies have already expressed >> interest in this project, and we intend to invite additional developers to >> participate. We will encourage and monitor community participation so that >> privileges can be extended to those that contribute operating to the >> standard of meritocracy that Apache emphasizes. >> >> === Community === >> >> The need for a generic scalable and declarative machine learning approach >> in the open source is tremendous, so there is a potential for a very large >> community. We believe that SystemML’s extensible architecture, declarative >> syntax, cost based optimizer and its alignment with Spark will further >> encourage community participation not only in enhancing the infrastructure >> but also speed up the creation of algorithms for a wide range of use >> cases. We expect that over time SystemML will attract a large community. >> >> === Alignment === >> >> The initial committers strongly believe that a generic scalable and >> declarative machine learning approach for machine learning will gain >> broader adoption as an open source, community driven project, where the >> community can contribute not only to the core components, but also to a >> growing collection of algorithms which will leverage the optimizations and >> ease of scaling in SystemML. Our hope is that the Apache Spark, Apache >> Hadoop and other communities will find tremendous value in SystemML and >> this will foster further collaboration between these projects furthering >> the already existing integration points. >> >> == Known Risks == >> >> To-date, development has been sponsored by IBM and coordinated mostly by >> the core team of researchers at the IBM Almaden Research Center. >> >> For SystemML to fully transition to an "Apache Way" governance model, it >> needs to start embracing the meritocracy-centric way of growing the >> community of contributors. >> >> === Orphaned Products === >> >> The SystemML developers and previous sponsor have a long-term interest in >> use and maintenance of the code and there is also hope that growing a >> diverse community around the project will become a guarantee against the >> project becoming orphaned. We feel that it is also important to put formal >> governance in place both for the project and the contributors as the >> project expands. We feel ASF is the best location for this. >> >> === Inexperience with Open Source === >> >> The current SystemML set of contributors are very diverse regarding >> participation in Open Source. While some initial members are experiencing >> an open source project for the first time, others have been contributing >> and mentoring various Apache and non-Apache open source projects. >> >> === Reliance on Salaried Developers === >> >> SystemML currently receives substantial support from salaried developers. >> However, they are all passionate about the project, and we are confident >> that the project will continue even if no salaried developers contribute to >> the project. We are committed to recruiting additional committers including >> non-salaried developers. >> >> >> === Relationships with Other Apache Products === >> >> Currently, SystemML integrates with Apache Hadoop MapReduce and Apache >> Spark as underlying computational distributed runtimes. >> >> === An Excessive Fascination with the Apache Brand === >> >> SystemML solves a real need for generic scalable and declarative machine >> learning approach for machine learning in the Apache Hadoop and Spark >> ecosystems, something that has been addressed in a very ad hoc manner so >> far by multiple Apache projects. Our rationale for developing SystemML as >> an Apache project is detailed in the Rationale section. We believe that the >> Apache brand and community process will help us attract more contributors >> to this project, and help establish ubiquitous APIs. >> >> >> == Documentation == >> >> Documentation regarding SystemML is available in the current GitHub >> repository https://github.com/SparkTC/systemml/tree/master/system-ml/docs. >> >> >> == Initial Source == >> >> Initial source is available on GitHub under the Apache License 2.0 >> >> https://github.com/SparkTC/systemml >> >> == Source and Intellectual Property Submission Plan == >> >> We know of no legal encumbrances in the transfer of source code and rights >> to Apache. In fact, given the internal IBM due diligence performed on the >> source code during open sourcing, we expect the code base to be free from >> any IP issues. >> >> == External Dependencies == >> >> SystemML is written in Java and currently supports Apache Hadoop MapReduce >> and Apache Spark runtimes. >> >> To the best of our knowledge, all dependencies of SystemML are distributed >> under Apache compatible licenses. Upon acceptance to the incubator, we >> would begin a thorough analysis of all transitive dependencies to verify >> this fact and introduce license checking into the build and release process >> (for instance integrating Apache Rat). >> >> Cryptography >> N/A >> >> == Required Resources == >> >> === Mailing lists === >> * priv...@sysml.incubator.apache.org (moderated subscriptions) >> * comm...@sysml.incubator.apache.org >> * d...@sysml.incubator.apache.org >> >> === Git Repository === >> * https://git-wip-us.apache.org/repos/asf/incubator-sysml.git >> >> === Issue Tracking === >> * JIRA (SYSML) >> >> == Initial Committers == >> >> * Luciano Resende (lresende AT apache DOT org) >> * Berthold Reinwald (reinwald AT us DOT ibm DOT com) >> * Matthias Boehm (mboehm AT us DOT ibm DOT com) >> * Shirish Tatikonda (statiko AT us DOT ibm DOT com) >> * Niketan Pansare (npansar AT us DOT ibm DOT com) >> * Prithviraj Sen (senp AT us DOT ibm DOT com) >> * Alexandre V Evfimievski (evfimi AT us DOT ibm DOT com) >> * Fred Reiss (frreiss AT us DOT ibm DOT com) >> * Deron Eriksson (deron AT us DOT ibm DOT com) >> * Arvind Surve (asurve AT us DOT ibm DOT com) >> * Mike Dusenberry (mwdusenb AT us DOT ibm DOT com) >> * Reynold Xin (rxin AT apache DOT org) >> * Xiangrui Meng (meng AT apache DOT org) >> * Joseph Bradley (jkbradley AT apache DOT org) >> * Patrick Wendell (pwendell AT apache DOT org) >> * Holden Karau (holden AT apache DOT org) >> * DB Tsai (dbtsai AT apache DOT org) >> >> == Affiliations == >> >> * DataBricks: Reynold Xin, Xiangrui Meng, Joseph Bradley, Patrick Wendell >> * Netflix: DB Tsai >> * IBM: Luciano Resende, Berthold Reinwald, Matthias Boehm, Shirish >> Tatikonda, Niketan Pansare, Prithviraj Sen, Alexandre V Evfimievski, Fred >> Reiss, Deron Eriksson, Arvind Surve, Mike Dusenberry and Holden Karau. >> >> == Sponsors == >> >> === Champion === >> * Luciano Resende >> >> === Nominated Mentors === >> * Luciano Resende >> * Reynold Xin >> * Patrick Wendell >> * Rich Bowen >> >> === Sponsoring Entity === >> We would like to propose the Apache Incubator to sponsor this project. > Off course, my +1 > > -- > Luciano Resende > http://people.apache.org/~lresende > http://twitter.com/lresende1975 > http://lresende.blogspot.com/
