Andrew, as far as I know Ignite has a component that works with Hadoop (HDFS) as a storage layer and that's pretty much all I could say about the declared dependency. Ignite doesn't require Hadoop to deliver any functionality, it doesn't provide any additions to Hadoop - HDFS is just yet another file system. Now, because Ignite is an In-Memory Data Fabric I am sure it does decent amount of caching to reduce IO overhead.
It seems like an implementation detail to me, but do you think this needs to be added into the proposal? Thanks! Cos On Mon, Sep 22, 2014 at 08:34AM, Andrew Purtell wrote: > On Sun, Sep 21, 2014 at 6:59 PM, Konstantin Boudnik <c...@apache.org> wrote: > > On Thu, Sep 18, 2014 at 10:21PM, Henry Saputra wrote: > >> Hi Cos, > >> > >> Looks like a good start of the proposal. > >> > >> How would this project relate to compare to existing ones like Apache > >> Spark, Storm, or Samza? > > > > The proposal will be updated shortly with these details. > > I browsed GridGain's website briefly, perhaps there could also be a > bit of detail in the proposal how it relates to the Apache Hadoop > project? > > > >> Would love to have comparisons to existing ASF projects section to the > >> proposal. > >> > >> Also, would you guys mind adding or soliciting more mentors? > >> Seemed like most of initial committers have not been involved in ASF > >> yet so may need some help to adjust to Apache way. > > > > Good point, Henry! Would you consider investing a few cycles on your own and > > join the mentors for this proposal? Thanks in advance! > > If you need more mentors, I'd be happy to volunteer. > > > > > > Cos > > > >> > >> - Henry > >> > >> On Thu, Sep 18, 2014 at 9:40 PM, Konstantin Boudnik <c...@apache.org> > >> wrote: > >> > I would like to propose Silk as an Apache Incubator project. The new > >> > proposal is added to https://wiki.apache.org/incubator/SilkProposal and > >> > is duplicated below. > >> > > >> > -- > >> > Regards, > >> > Cos > >> > > >> > > >> > = Silk Apache Incubator Proposal = > >> > > >> > == Abstract == > >> > > >> > Apache Silk will be a unified In-Memory Data Fabric providing > >> > high-performance, > >> > distributed in-memory data management software layer between various data > >> > sources and user applications. > >> > > >> > == Proposal == > >> > > >> > Apache Silk is written mostly in Java and Scala with small amount of C++ > >> > code > >> > and will initially combine the following technologies under one unified > >> > umbrella: > >> > * In-Memory Data Grid > >> > * In-Memory Compute Grid > >> > * In-Memory Streaming Processing > >> > This unified in-memory fabric will provide high-performance, distributed > >> > in-memory software layer that sits in between various data sources and > >> > user > >> > applica tions. Data sources can include SQL RDBMS, NoSQL, or HDFS. > >> > Applications > >> > APIs will be available for Java (and Java-based scripting languages), > >> > Scala, > >> > C++ and .NET (C#). > >> > > >> > GridGain Systems, Inc. submits this proposal to donate its Apache > >> > 2.0-licensed > >> > open source project generally known as “GridGain In-Memory Computing > >> > Platform”, > >> > its source code, documentation, and websites to the Apache Software > >> > Foundation > >> > (“ASF”) with the goal of extending the vibrant open source community > >> > around > >> > this technology ultimately governed by “Apache Way”. Proposed Naming > >> > > >> > We have been advised by the ASF mentors that the name “Silk” may not be > >> > ideal > >> > because the name may be too generic and may not pass ASF legal check. > >> > Here are > >> > the alternatives that we have come up with and any of those will be > >> > acceptable > >> > for the project pending the ASF legal green light: > >> > * Apache Silk (preferable name) > >> > * Apache Sylk > >> > * Apache Memstor > >> > * Apache Ignite > >> > > >> > == Background & Rationale == > >> > > >> > In-Memory Data Fabric is a natural and evolutionary consolidation of > >> > various > >> > “in-memory technologies” from the last decade. From simple local caching > >> > (JSR-107), to distributed caching, to data grids and databases, to > >> > streaming > >> > and plug-n-play acceleration - the in-memory space has grown quite > >> > dramatically. > >> > > >> > With rapid advances in NVRAM and significant price reduction of > >> > traditional > >> > DRAM on one hand, and growing sophistication and demand for faster data > >> > processing on another - many users of these silo-ed technologies and > >> > products > >> > started to look for a “strategic approach” to in-memory - an in-memory > >> > data > >> > fabric - that would provide suitable APIs for different types of > >> > payloads: from > >> > data caching, to data grids, to in-memory SQL data stores, to HPC, to > >> > streaming > >> > processing. > >> > > >> > With expensive and proprietary in-memory computing products from > >> > companies like > >> > Oracle, SAP, Microsoft, and IBM - the developers worldwide need an > >> > unhindered > >> > access to advanced open source in-memory software technology, the > >> > technology > >> > they can trust to develop with and deploy for critical applications. > >> > Current > >> > Status > >> > > >> > Apache Silk will be based on the technology that is currently developed > >> > by > >> > GridGain Systems and available under Apache 2.0 license > >> > (http://www.gridgain.org). The software has been in development since > >> > 2007 and > >> > in production since 2009. It is currently used by over 500 production > >> > deployments with over 1,000,000 downloads to date, and with over > >> > 20,000,000 > >> > GridGain nodes started in the last 5 years. > >> > > >> > == Initial Goals == > >> > > >> > The number one goal during ASF incubation will coalesce around building > >> > a true > >> > active and vibrant community governed by the “Apache Way”. The initial > >> > development goals for Silk primarily revolve around migrating the > >> > existing code > >> > base, documentation, and refactoring of the existing internal build, > >> > test & > >> > release processes. We believe these initial goals are sufficiently > >> > difficult to > >> > be considered early milestones. > >> > > >> > Some of the specific initial goals include: > >> > * Migrate the existing Silk code base to the ASF. > >> > * Refactor development, testing, build and release processes to work in > >> > ASF. > >> > * Attract developer and user interest in the new Apache Silk project. > >> > * Road map the integration efforts with “sister” projects in ASF > >> > eco-system like Storm and Spark. > >> > * Incorporate externally developed features into the core Apache Silk > >> > project. > >> > > >> > == Known Risks == > >> > > >> > This proposal is not without its risks, some of which are outlined below. > >> > > >> > The current list of committers are primarily from GridGain Systems. One > >> > of the > >> > key purposes of proposing Silk for incubation is to attract new > >> > committers and > >> > spur the adoption of Silk. The ASF has a well-deserved reputation of > >> > fostering > >> > and building open source communities, which makes it the ideal location > >> > to > >> > attempt this community bootstrap. > >> > > >> > Most of the initial committers are supported by their employers to work > >> > on > >> > Silk, and may be assigned to work on other priorities. However, the > >> > employers > >> > of these salaried individuals - GridGain Systems or current customers > >> > and users > >> > - have a vested interest in seeing Silk thrive as a long-term, growing > >> > project. > >> > > >> > GridGain Systems understands that their employees are acting as > >> > individuals > >> > when contributing to Apache projects. As a major initial contributor > >> > GridGain > >> > Systems is prepared to bring additional staff on board to assist with > >> > Silk > >> > development to ensure its active growth. > >> > > >> > One of the key motivators in creating the Silk project as part of the > >> > Incubator > >> > is to leverage the vendor-neutral nature of the ASF. The ASF has a > >> > strong and > >> > recognized brand as being a leader in open source, and by hosting Silk > >> > at the > >> > ASF, we hope to attract developers to build a viable community for the > >> > project. > >> > > >> > == Meritocracy == > >> > > >> > Apache Silk plans to adopt the policy that encourages an environment that > >> > supports a meritocracy. We intend to actively ask the community for > >> > help, > >> > listing/specifying the work that needs to be done, and keeping track of > >> > and > >> > encouraging members of the community who make any contributions. > >> > Community & > >> > Core Developers > >> > > >> > GridGain project has been actively building community of users in the > >> > last > >> > couple of years with an active StackOverflow group, support groups, and > >> > Meetups > >> > (http://www.meetup.com/Bay-Area-In-Memory-Computing). This group includes > >> > active members of Apache community as well. We strongly believe that this > >> > community will grow and develop substantially as part of Apache family > >> > and > >> > that’s our commitment. > >> > > >> > == Existing Documentation == > >> > > >> > Current documentation for GridGain project can be found here: > >> > http://www.gridgain.org/documentation/ We intend to migrate it into ASF > >> > podling. > >> > > >> > == Initial Source == > >> > > >> > Initial Apache 2.0 licensed source code can be found here: > >> > http://www.gridgain.org/download/ > >> > > >> > == External Dependencies == > >> > > >> > Here’s the list of 3rd party JAR-only dependencies: > >> > * Apache Hadoop > >> > * Apache Commons > >> > * H2 > >> > * JTS > >> > * Apache Lucene > >> > * Spring > >> > > >> > Here’s the list of the all licenses for 3rd party libraries currently > >> > used: > >> > * Apache 2.0 > >> > > >> > == Required Resources == > >> > > >> > === Mailing lists === > >> > * silk-private (with moderated subscriptions) > >> > * silk-dev > >> > * silk-user > >> > > >> > === Git & JIRA === > >> > * Git: https://git-wip-us.apache.org/repos/asf/incubator-silk.git > >> > * JIRA: JIRA Silk (SILK) > >> > > >> > == Initial Committers & Affiliation == > >> > * Dmitriy Setrakyan (GridGain Systems, dsetrakyan at gridgain dot com) > >> > * Yakov Zhdanov (GridGain Systems, yzhdanov at gridgain dot com) > >> > * Alexey Goncharuk (GridGain Systems, agoncharuk at gridgain dot com) > >> > * Sergey Vladykin (GridGain Systems, svladykin at gridgain dot com) > >> > * Valentin Kulichenko (GridGain Systems, vkulichenko at gridgain dot > >> > com) > >> > * Semen Boikov (GridGain Systems, sboikov at gridgain dot com) > >> > * Vladimir Ozerov (GridGain Systems, vozerov at gridgain dot com) > >> > * Nikita Ivanov (GridGain Systems, nivanov30 at gmail dot com) > >> > * Sergey Khisamov (FitechSource, skh at gmail dot com) > >> > * Ilya Sterin (ChronoTrack, isterin at gmail dot com) > >> > * Ryan Rawson (WANdisco, rawson at apache dot org) > >> > * Konstantin Boudnik (cos at apache dot org) > >> > * Roman Shaposhnik (rvs at apache dot org) > >> > > >> > == Sponsors == > >> > > >> > === Apache Champion === > >> > * Konstantin Boudnik (cos at apache dot org) > >> > > >> > === Nominated Mentors === > >> > * Michael Stack (stack at apache dot org) > >> > * Roman Shaposhnik (rvs at apache dot org) > >> > * Konstantin Boudnik (cos at apache dot org) > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > >> For additional commands, e-mail: general-h...@incubator.apache.org > >> > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein (via Tom White) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org >
signature.asc
Description: Digital signature
