Thanks Till, Will try to solicit more mentors to help. Especially with initial committers mostly have not been exposed to contributing the Apache way.
- Henry On Mon, Jan 19, 2015 at 5:28 PM, Till Westmann <t...@westmann.org> wrote: > Hi Henry, > > thanks! It’s great that you’ve seen (and liked) AsterixDB before. > > Even if your time is very limited we would be very happy to have you on board > as a mentor. > I’ll add you to the proposal. > > Cheers, > Till > >> On Jan 19, 2015, at 10:26 AM, Henry Saputra <henry.sapu...@gmail.com> wrote: >> >> +1 This is GREAT News! >> >> Was watching and trying AsterixDB last year and looked in awesome shape. >> >> I have my plate full but would love to help mentor this project to get >> it going to ASF if needed! >> >> - Henry >> >> On Wed, Jan 14, 2015 at 6:21 PM, Mattmann, Chris A (3980) >> <chris.a.mattm...@jpl.nasa.gov> wrote: >>> Hi Folks, >>> >>> I am pleased to bring forth the Apache AsterixDB proposal to the >>> Apache Incubator as Champion, working in collaboration with the >>> team. Please find the wiki proposal here: >>> >>> https://wiki.apache.org/incubator/AsterixDBProposal >>> >>> >>> Full text of the proposal is below. Please discuss and enjoy. I’ll >>> leave the discussion open for a week, and then look to call a VOTE >>> hopefully end of next week if all is well. >>> >>> Cheers! >>> Chris Mattmann >>> >>> ============================================================= >>> Apache AsterixDB Proposal >>> >>> Abstract >>> >>> Apache AsterixDB is a scalable big data management system (BDMS) that >>> provides storage, management, and query capabilities for large >>> collections of semi-structured data. >>> >>> Proposal >>> >>> AsterixDB is a big data management system (BDMS) that makes it >>> well-suited to needs such as web data warehousing and social data >>> storage and analysis. Feature-wise, AsterixDB has: >>> >>> * A NoSQL style data model (ADM) based on extending JSON with object >>> database concepts. >>> * An expressive and declarative query language (AQL) for querying >>> semi-structured data. >>> * A runtime query execution engine, Hyracks, for partitioned-parallel >>> execution of query plans. >>> * Partitioned LSM-based data storage and indexing for efficient >>> ingestion of newly arriving data. >>> * Support for querying and indexing external data (e.g., in HDFS) as >>> well as data stored within AsterixDB. >>> * A rich set of primitive data types, including support for spatial, >>> temporal, and textual data. >>> * Indexing options that include B+ trees, R trees, and inverted >>> keyword index support. >>> * Basic transactional (concurrency and recovery) capabilities akin to >>> those of a NoSQL store. >>> >>> >>> Background and Rationale >>> >>> In the world of relational databases, the need to tackle data volumes >>> that exceed the capabilities of a single server led to the >>> development of “shared-nothing” parallel database systems several >>> decades ago. These systems spread data over a cluster based on a >>> partitioning strategy, such as hash partitioning, and queries are >>> processed by employing partitioned-parallel divide-and-conquer >>> techniques. Since these systems are fronted by a high-level, >>> declarative language (SQL), their users are shielded from the >>> complexities of parallel programming. Parallel database systems have >>> been an extremely successful application of parallel computing, and >>> quite a number of commercial products exist today. >>> >>> In the distributed systems world, the Web brought a need to index and >>> query its huge content. SQL and relational databases were not the >>> answer, though shared-nothing clusters again emerged as the hardware >>> platform of choice. Google developed the Google File System (GFS) and >>> MapReduce programming model to allow programmers to store and process >>> Big Data by writing a few user-defined functions. The MapReduce >>> framework applies these functions in parallel to data instances in >>> distributed files (map) and to sorted groups of instances sharing a >>> common key (reduce) -- not unlike the partitioned parallelism in >>> parallel database systems. Apache's Hadoop MapReduce platform is the >>> most prominent implementation of this paradigm for the rest of the >>> Big Data community. On top of Hadoop and HDFS sit declarative >>> languages like Pig and Hive that each compile down to Hadoop >>> MapReduce jobs. >>> >>> The big Web companies were also challenged by extreme user bases >>> (100s of millions of users) and needed fast simple lookups and >>> updates to very large keyed data sets like user profiles. SQL >>> databases were deemed either too expensive or not scalable, so the >>> “NoSQL movement” was born. The ASF now has HBase and Cassandra, two >>> popular key-value stores, in this space. MongoDB and Couchbase are >>> other open source alternatives (document stores). >>> >>> It is evident from the rapidly growing popularity of "NoSQL" stores, >>> as well as the strong demand for Big Data analytics engines today, >>> that there is a strong (and growing!) need to store, process, *and* >>> query large volumes of semi-structured data in many application >>> areas. Until very recently, developers have had to ``choose'' between >>> using big data analytics engines like Apache Hive or Apache Spark, >>> which can do complex query processing and analysis over HDFS-resident >>> files, and flexible but low-function data stores like MongoDB or >>> Apache HBase. (The Apache Phoenix project, >>> http://phoenix.apache.org/, is a recent SQL-over-HBase effort that >>> aims to bridge between these choices.) >>> >>> AsterixDB is a highly scalable data management system that can store, >>> index, and manage semi-structured data, e.g., much like MongoDB, but >>> it also supports a full-power query language with the expressiveness >>> of SQL (and more). Unlike analytics engines like Hive or Spark, it >>> stores and manages data, so AsterixDB can exploit its knowledge of >>> data partitioning and the availability of indexes to avoid always >>> scanning data set(s) to process queries. Somewhat surprisingly, there >>> is no open source parallel database system (relational or otherwise) >>> available to developers today -- AsterixDB aims to fill this need. >>> Since Apache is where the majority of the today's most important Big >>> Data technologies live, the ASF seems like the obvious home for a >>> system like AsterixDB. >>> >>> Current Status >>> >>> The current version of AsterixDB was co-developed by a team of >>> faculty, staff, and students at UC Irvine and UC Riverside. The >>> project was initiated as a large NSF-sponsored project in 2009, the >>> goal of which was to combine the best ideas from the parallel >>> database world, the then new Hadoop world, and the semi-structured >>> (e.g., XML/JSON) data world in order to create a next-generation >>> BDMS. A first informal open source release was made four years later, >>> in June of 2013, under the Apache Software License 2.0. >>> >>> >>> Meritocracy >>> >>> The current developers are familiar with meritocratic open source >>> development at Apache. Apache was chosen specifically because we want >>> to encourage this style of development for the project. >>> >>> >>> Community >>> >>> While AsterixDB started as a university project it has developed into >>> a community. A number of the initial committers started contributing >>> in academia and continue to actively participate and contribute after >>> graduation. And we seek to further develop developer and user >>> communities. One way to broaden the community that is ongoing is >>> through academic collaborations (currently with IIT Mumbai in India >>> and TU Berlin in Germany). During incubation we will also explicitly >>> seek increased industrial participation. >>> >>> Some indicators of the effort's development community and history can >>> be >>> found at: >>> https://www.openhub.net/p/asterixdb/contributors?query=&sort=commits_12_mo, >>> https://www.openhub.net/p/hyracks/contributors?query=&sort=commits_12_mo >>> >>> >>> Core Developers >>> >>> The core developers of the project are diverse, although initially UC >>> Irvine heavy (roughly 50) due to the project's origins at UCI. The >>> other 50 are from other academic institutions (UC Riverside and the >>> Hebrew University in Jerusalem) and companies (Couchbase, Facebook, >>> IBM, KACST Saudi Arabia, Oracle, Saudi Aramco, X15 Software). >>> >>> >>> Alignment >>> >>> Apache is, by far, the most natural home for taking the AsterixDB >>> project forward. A large fraction of today's top Big Data >>> technologies have their homes in Apache, including Hadoop, YARN, Pig, >>> Hive, Spark, Flink, HBase, Cassandra and others. AsterixDB fills a >>> significant gap -- the parallel data management system gap -- that >>> exists in the Big Data open source world. It is well-aligned with a >>> number of the Apache projects, e.g., it has strong support for >>> accessing and indexing external data in HDFS, and it uses YARN as an >>> answer to basic cluster resource management. AsterixDB also seeks to >>> achieve an Apache-style development model; it is seeking a broader >>> community of contributors and users in order to achieve its full >>> potential and value to the Big Data community. >>> >>> There are also a number of related Apache projects and dependencies >>> that will be mentioned below in the Relationships with Other Apache >>> products section. >>> >>> >>> Known Risks >>> >>> Orphaned products >>> >>> Given the current level of intellectual investment in AsterixDB, the >>> risk of the project being abandoned is very small. The UCI/UCR >>> faculty team leads are highly incentivized to continue development >>> since the database groups at UC Irvine and UC Riverside are both >>> reliant on AsterixDB as a platform for long-term graduate research >>> projects. UC San Diego is also beginning to contribute to the code >>> base, and a collaboration involving public health applications is >>> forming with UCLA. The work on AsterixDB is managed via a mix of >>> mailing list discussions supplemented by weekly project status >>> meetings which are summarized on the mailing list. Typical (local >>> plus Skype-in) attendance to the weekly status meetings runs at about >>> 20 active contributors. >>> >>> >>> Inexperience with Open Source >>> >>> AsterixDB and Hyracks were completely developed in Open Source under >>> the ASL 2.0. The source code repositories, issue tracker, and mailing >>> lists are available on Google Code and discussions and decisions >>> happen on the mailing lists (which is necessary due to the geographic >>> distribution of the current developers). >>> >>> Also a few of the initial committers have contributed to Apache >>> projects. Vinayak Borkar is a committer on the Apache Helix and >>> Apache VXQuery projects. Till Westmann is the VP VXQuery at the ASF >>> and an IPMC member. Preston Carman and Steven Jacobs are committers >>> on the Apache VXQuery project. >>> >>> >>> Relationships with Other Apache Products >>> >>> Apache VXQuery is based on the Hyracks data-parallel runtime, which >>> is also included in the AsterixDB code base. >>> >>> AsterixDB is closely related to Apache Hadoop. Included in AsterixDB >>> is support for accessing external data in HDFS (and Hive formats), >>> and resource management and system administration features are in the >>> process of being migrated to YARN. >>> >>> AsterixDB's AQL query facilities offer comparable query power to >>> Apache's Pig and Hive systems for big data analytics. AsterixDB >>> differs in storing and indexing data and thus being able to quickly >>> answer small and medium queries without large HDFS data scans - >>> thereby targeting a different class of use cases. >>> >>> AsterixDB's data storage and indexing facilities are similar to those >>> of HBase, but AsterixDB differs in being a much more complete and >>> queryable BDMS (not just a key-value style store). >>> >>> AsterixDB's target use cases are not in-memory processing or >>> iterative algorithm support, making AsterixDB complementary to the >>> Apache Spark platform. (Spark interoperability is on our longer-term >>> to-do wishlist.) >>> >>> >>> Homogeneous Developers >>> >>> As mentioned before the current community is already organizationally >>> and geographically distributed - and we would like to increase the >>> heterogeneity. >>> >>> >>> Reliance on Salaried Developers >>> >>> Of the initial committers only 3 are full-time UCI staff. The other >>> committers are a mix of students, alumni who continue to contribute >>> to the effort, and individuals working with permission part-time (or >>> in spare time) on this project. >>> >>> >>> A Excessive Fascination with the Apache Brand >>> >>> We believe in the processes, systems, and framework Apache has put in >>> place. Apache is also known to foster a great community around their >>> projects and provide exposure. While brand is important, our >>> fascination with it is not excessive. We believe that the ASF is the >>> right home for AsterixDB and that having AsterixDB inside of the ASF >>> will lead to a better long-term outcome for the Big Data community. >>> >>> >>> Documentation >>> >>> Documentation and publications related to AsterixDB can be found at >>> http://asterixdb.ics.uci.edu/. >>> >>> >>> Initial Source >>> >>> Current source resides in Google code: >>> https://code.google.com/p/asterixdb/ (query language and upper system >>> layers) and https://code.google.com/p/hyracks/ (dataflow runtime >>> system and storage management libraries). >>> >>> >>> External Dependencies >>> >>> AsterixDB depends on a number of Apache projects: >>> >>> - Ant >>> - Avro >>> - ApacheDB JDO >>> - Commons >>> - Derby >>> - Hadoop >>> - Hive >>> - HTTPComponents >>> - Jakarta ORO >>> - Maven >>> - Tomcat >>> - Thrift >>> - Velocity >>> - Wicket >>> - Xerces >>> >>> and other open source projects (organized by license): >>> >>> -- ASL 2.0: >>> - Jackson >>> - Google Guava >>> - Google Guice >>> - JSON-simple >>> - BoneCP >>> - Microsoft Azure SDK >>> - Netty >>> - Rome >>> - JetS3t >>> - Groovy >>> - Jettison >>> - Plexus >>> - Datanucleus (JDO) >>> - Jetty >>> - Twitter4J >>> - Snappy-java >>> >>> -- BSD: >>> - Antlr >>> - ObjectWeb ASM >>> - Protobuf >>> - JSCH >>> - JavaCC >>> - Paranamer >>> - JLine >>> - Stax >>> - StringTemplate >>> - xmlEnc >>> >>> -- MIT >>> - AppAssembler >>> - SimpleLog4J >>> >>> -- CDDL 1.0 >>> - Java Activation Framework >>> - Java Transactions >>> - Java Servlet API >>> - Grizzly >>> - gmbal >>> - Glassfish >>> >>> -- CDDL 1.1 >>> - Jersey >>> - JAXB Reference Implementation >>> >>> -- JSON License >>> - JSON >>> >>> -- EPL 1.0 >>> - JUnit >>> >>> -- JDOM License >>> - JDOM >>> >>> -- Public Domain >>> - xz >>> - AOPAlliance >>> >>> As all dependencies are managed using Apache Maven, none of the >>> external libraries need to be packaged in a source distribution. >>> >>> >>> Required Resources >>> >>> Developer and user mailing lists >>> >>> priv...@asterixdb.incubator.apache.org (with moderated subscriptions) >>> comm...@asterixdb.incubator.apache.org >>> d...@asterixdb.incubator.apache.org >>> us...@asterixdb.incubator.apache.org >>> >>> >>> A git repository >>> >>> https://git-wip-us.apache.org/repos/asf/incubator-asterixdb.git >>> >>> >>> A JIRA issue tracker >>> >>> https://issues.apache.org/jira/browse/ASTERIXDB >>> >>> >>> Initial Committers >>> >>> The following is a list of the planned initial Apache committers (the >>> active subset of the committers for the current repository at Google >>> code). >>> >>> Abdullah Alamoudi (bamou...@gmail.com) >>> Cameron Samak (euf...@gmail.com) >>> Chen Li (che...@gmail.com) >>> Ian Maxon (ima...@uci.edu) >>> Ildar Absalyamov (ildar.absalya...@gmail.com) >>> Jianfeng Jia (jianfeng....@gmail.com) >>> Karen Ouaknine (ker...@gmail.com) >>> Markus Dreseler (apa...@dreseler.de) >>> Mike Carey (dtab...@apache.org) >>> Murtadha Hubail (hubail...@gmail.com) >>> Pouria Pirzadeh (pouria.pirza...@gmail.com) >>> Preston Carman (prest...@apache.org) >>> Raman Grover (ramangrove...@gmail.com) >>> Sattam Alsubaiee (salsuba...@gmail.com) >>> Steven Jacobs (sjaco...@apache.org) >>> Taewoo Kim (wangs...@gmail.com) >>> Till Westmann (ti...@apache.org) >>> Vinayak Borkar (vinay...@apache.org) >>> Yingyi Bu (buyin...@gmail.com) >>> Young-Seok Kim (kiss...@gmail.com) >>> Zach Heilbron (zheilb...@gmail.com) >>> >>> >>> Affiliations >>> >>> UC Irvine >>> - Mike Carey >>> - Chen Li >>> - Ian Maxon >>> - Yingyi Bu >>> - Raman Grover >>> - Pouria Pirzadeh >>> - Young-Seok Kim >>> - Cameron Samak >>> - Taewoo Kim >>> - Jianfeng Jia >>> - Murtadha Hubail >>> - Markus Dreseler >>> >>> UC Riverside >>> - Ildar Absalyamov >>> - Preston Carman >>> - Steven Jacobs >>> >>> Hebrew University >>> - Keren Ouaknine >>> >>> Oracle >>> - Till Westmann >>> >>> X15 Software >>> - Vinayak Borkar >>> - Zach Heilbron >>> >>> KACST Saudi Arabia >>> - Sattam Alsubaiee >>> >>> Saudi Aramco >>> - Abdullah Alamoudi >>> >>> Carey, Li, and Maxon are full-time UCI staff, with the remaining UCI >>> (UC Irvine) and UCR (UC Riverside) affiliates being students. The >>> non-UC committers are a mix of alumni who continue to contribute to >>> the effort and individuals working with permission part-time (or in >>> spare time) on this project. >>> >>> >>> Sponsors >>> >>> Champion >>> >>> Chris Mattmann (NASA/JPL) >>> >>> Nominated Mentors >>> >>> TBD >>> >>> Sponsoring Entity >>> >>> The Apache Incubator >>> >>> >>> >>> >>> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Chris Mattmann, Ph.D. >>> Chief Architect >>> Instrument Software and Science Data Systems Section (398) >>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>> Office: 168-519, Mailstop: 168-527 >>> Email: chris.a.mattm...@nasa.gov >>> WWW: http://sunset.usc.edu/~mattmann/ >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Adjunct Associate Professor, Computer Science Department >>> University of Southern California, Los Angeles, CA 90089 USA >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>> >>> >>> > --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
