Re: [RESULT] [VOTE] Apache Ignite 1.2.0 release (RC2)
On Sun, Jun 28, 2015 at 10:56 PM, Yakov Zhdanov yzhda...@apache.org wrote: Guys, It's nice to be informal, but we're not just guys. :) Voting for 1.2 has passed with 3 votes for (+1) and 1 +0 vote. Congrats! Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [RESULT] [VOTE] Apache Ignite 1.2.0 release (RC2)
On 29.06.2015 15:36, Marvin Humphrey wrote: On Sun, Jun 28, 2015 at 10:56 PM, Yakov Zhdanov yzhda...@apache.org wrote: Guys, It's nice to be informal, but we're not just guys. :) Aye; next time, please address mails to laddies and gentlemen. :) -- Brane - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Last chance to tell your story at ApacheCON EU 2015
Thanks for reminding us podlings of opportunity to grow the community through the conference! I already submitted a talk on Sentry, would also like to do a speed date. I am not able to update the wiki here. Can some one update it for me please or give me permission to do so? (id:sravya) https://wiki.apache.org/apachecon/ACEU15Incubator Thanks! On Mon, Jun 29, 2015 at 12:42 AM, jan i j...@apache.org wrote: On 29 June 2015 at 10:33, Don Bosco Durai bo...@apache.org wrote: Can I have write permission (wiki id=bosco) or someone update the wiki for me? Don Bosco Durai - Apache Ranger I am still not sure who will talk, but can we put mine as a place holder for now? please remember that for a full talk you need to submit a proposal http://events.linuxfoundation.org/events/apachecon-core-europe/ including a speaker name. hint: 36 hours to go. thanks in advance rgds jan i. Thanks Bosco On 6/29/15, 10:09 AM, jan i j...@apache.org wrote: On 29 June 2015 at 01:28, Anatole Tresch atsti...@gmail.com wrote: Hi all I proposed to talk on Tamaya. I would also do a speed date on Tamaya, but unfortunately I cannot update: https://wiki.apache.org/apachecon/ACEU15Incubator Can somebody do this for me, or give the rights todo so (UID: AnatoleTresch)? updated have fun jan i. Thanks, Anatole 2015-06-27 23:10 GMT+02:00 Roman Shaposhnik ro...@shaposhnik.org: On Sat, Jun 27, 2015 at 1:44 PM, Pierre Smits pierre.sm...@gmail.com wrote: Roman, There is a talk in preparation regarding Trafodion. Great! Once it it submitted, could you please update the wiki? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- *Anatole Tresch* Java Engineer Architect, JSR Spec Lead Glärnischweg 10 CH - 8620 Wetzikon *Switzerland, Europe Zurich, GMT+1* *Twitter: @atsticks* *Blogs: **http://javaremarkables.blogspot.ch/ http://javaremarkables.blogspot.ch/* *Google: atsticksMobile +41-76 344 62 79* - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Sravya Tirukkovalur
Re: [PROPOSAL]Pistachio
Hi Roman, I've taken a brief look at Geode. Pistachio and Geode both can support similar in-memory consistent HA distributed storage, and the data aware function routing idea is also similar. Here are what I think are different: 1. Pistachio actually uses a very different approach to resolve the problem. Pistachio leverages Apache Helix to do partition management and Apache Kafka as a WAL implementation to leverage the rich features and great performances in those systems. 2. Pistachio supports simple plugin of different storage engines like pure memory or different disk based local storage engines like kyoto cabinet or Rocks DB. On our production we mainly used kyoto cabinet based SSD storage system to optimize for read to resolve the low latency high volume read problem in RTB applications. Thanks, Gavin Li On Mon, Jun 22, 2015 at 8:25 PM, Gavin Li lyo.ga...@gmail.com wrote: Roman, I think Pistachio is similar to Ignite in the sense that they both try to distribute the computation to storage to co-locate the data and computation. One difference might be Pistachio also supports other storage options like disk based storage to support longer term durability. Actually Pistachio was originally developed as a storage system of SSD disk and has been used on our large scale production serving system with SSD disk. We're not that familiar with Geode, I'll look into it and provide some detailed comparisons. Thanks, Gavin Li On Mon, Jun 22, 2015 at 8:00 PM, Roman Shaposhnik ro...@shaposhnik.org wrote: On Mon, Jun 22, 2015 at 7:54 PM, Gavin Li lyo.ga...@gmail.com wrote: The other difference is in Pistachio we can do computation based on in-memory storage with data replication. Different from the in-memory computation in Spark, the storage can be in-memory here. Have you guys looked at in-memory computation layers offered by Ignite and Geode? I would love to know what you think about those. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [PROPOSAL]Pistachio
Hi Andrew, I agree with you. I've updated the proposal to include a little bit more explanations about the difference with Hadoop. Purely pursuing novelty is never our interest. Instead I believe even for the same problem different design and implementation ideas can make big difference. I think that's why there are many internal competitions in ASF. Having looked at other systems like Ignite and Geode I believe Pistachio is still quite different in design and implementation when solving some common problems like in-memory distributed storage and co-locating computation and data. Thanks, Gavin Li On Fri, Jun 26, 2015 at 12:07 PM, Andrew Purtell apurt...@apache.org wrote: Thanks Gavin. Please let me suggest that novelty is not a requirement for incubation, and a proposal doesn't need to make claims of novelty to be accepted. Should the proposal be accepted for incubation, you may find your new neighbors at Apache can do X where you weren't aware of it. It will be totally up to the new podling if you want to survey the landscape when figuring out how to differentiate, but I do recommend it, it may help you crystallize a community around a real difference and advantage provided by Pistachio. On Mon, Jun 22, 2015 at 7:54 PM, Gavin Li lyo.ga...@gmail.com wrote: Hi Andrew, As we described more in http://yahooeng.tumblr.com/post/116291838351/pistachio-co-locate-the-data-and-compute-for , a very common problem we saw in Hadoop use cases is we often need to persist the previous result of one map reduce job onto HDFS, then the next day we process the new data together with the previous result. Usually the most expensive part is the shuffling part where we need to join the previous data and the new data together. It's so expensive because HDFS doesn't store the data in a partitioned way. So data have to be transferred again and again in the shuffling phase. Instead, in Pistachio we do the computation right on top of the partitioned storage layer, so that the previous result is always stored in a partitioned way, so shuffling can be avoided. Expensive IO and roundtrips can thus be avoided so that much better performance can be achieved. The other difference is in Pistachio we can do computation based on in-memory storage with data replication. Different from the in-memory computation in Spark, the storage can be in-memory here. Please let me know if I'm not clear enough. Thanks, Gavin Li On Mon, Jun 22, 2015 at 7:53 PM, Andrew Purtell apurt...@apache.org wrote: It was a simple question, and not meant to suggest anything one way or other regarding my opinion of this proposal. On Monday, June 22, 2015, John D. Ament johndam...@apache.org wrote: On Mon, Jun 22, 2015 at 10:26 PM Andrew Purtell apurt...@apache.org javascript:; wrote: Pistachio can easily embed computation to the storage layer to achieve the best data locality to improve the computation performance significantly which is an innovative model comparing with the normal ways where the storage and compute are independent to each other. Have you heard of something called Hadoop? Regardless of whether he has or not - what's your point? The ASF has historically not denied the entry of new projects just because their domain intersects with another project's. On Thu, Jun 18, 2015 at 10:17 AM, Gavin Li lyo.ga...@gmail.com javascript:; wrote: Hi, I want to propose project Pistachio to enter Apache Incubator. Below please find the proposal. Thanks, Gavin Li = Pistachio = == Abstract == Pistachio is a fault-tolerant low latency distributed storage system which enables simple embedding the computation to the storage layer to achieve best data locality. It evolves from Yahoo’s global user profile storage system. == Proposal == Pistachio is a distributed key value store system with fault tolerance and consistency guarantee. It supports multiple local storage engine including in-memory, kyoto cabinet, rocks DB etc. Pistachio is being used as the user profile storage for massive scale global ads products in Yahoo storing 10+ billion user profiles. The performance and reliability has been well proven on production. Pistachio can easily embed computation to the storage layer to achieve the best data locality to improve the computation performance significantly which is an innovative model comparing with the normal ways where the storage and compute are independent to each other. == Background == Pistachio is originally designed and optimized for Yahoo’s large scale global open
Re: [VOTE] Release Apache Usergrid 1.0.2 (incubating) RC3
Shouldn't this close Thursday at 2200 UTC ? Either way +1 binding. On Jun 29, 2015 16:13, Dave snoopd...@gmail.com wrote: The Usergrid PPMC has voted to release Usergrid 1.0.2. Incubator PMC members please review and vote. Vote will close Wednesday July 1 at 10:11 UTC. Here is the vote thread: http://mail-archives.apache.org/mod_mbox/incubator-usergrid-dev/201506.mbox/%3ccaf1aazbhq7jrvbpnv9wgui_f-hla7pxoeaetgsp6spuy43b...@mail.gmail.com%3e And here is the release information: All, I have removed the ARSMaquette font files and made the LICENSE and NOTICE file fixes that Justin Mclean pointed out. Thanks Justin! I tested the portal and it seems to work fine without those files. I propose that we accept the following release candidate as the official Apache Usergrid 1.0.2 release. Usergrid 1.0.2-rc3 includes the following: --- The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.gitf=CHANGELOGhb=1.0.2-rc3 The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.githb=1.0.2-rc3 The current Git commit ID is 5456bfc57329d4a18998c511d0559e0a6cfe5461 The release candidate is available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz The MD5 checksum of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.md5 The signature of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.asc The GPG key used to sign the release are available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/KEYS Please download, verify, and test. On June 26, 2015 at 1:01:02 PM, Dave (snoopd...@gmail.com) wrote: All, I have removed the ARSMaquette font files and made the LICENSE and NOTICE file fixes that Justin Mclean pointed out. Thanks Justin! I tested the portal and it seems to work fine without those files. I propose that we accept the following release candidate as the official Apache Usergrid 1.0.2 release. Usergrid 1.0.2-rc3 includes the following: --- The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.gitf=CHANGELOGhb=1.0.2-rc3 The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.githb=1.0.2-rc3 The current Git commit ID is 5456bfc57329d4a18998c511d0559e0a6cfe5461 The release candidate is available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz The MD5 checksum of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.md5 The signature of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.asc The GPG key used to sign the release are available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/KEYS Please download, verify, and test. The vote will close on Mon Jun 29 14:36:55 EDT 2015 [ ] +1 Release this as Apache Usergrid 1.0.2 [ ] +0 [ ] -1 Do not release this as Apache Usergrid 1.0.2 because...
[VOTE] Release Apache Usergrid 1.0.2 (incubating) RC3
The Usergrid PPMC has voted to release Usergrid 1.0.2. Incubator PMC members please review and vote. Vote will close Wednesday July 1 at 10:11 UTC. Here is the vote thread: http://mail-archives.apache.org/mod_mbox/incubator-usergrid-dev/201506.mbox/%3ccaf1aazbhq7jrvbpnv9wgui_f-hla7pxoeaetgsp6spuy43b...@mail.gmail.com%3e And here is the release information: All, I have removed the ARSMaquette font files and made the LICENSE and NOTICE file fixes that Justin Mclean pointed out. Thanks Justin! I tested the portal and it seems to work fine without those files. I propose that we accept the following release candidate as the official Apache Usergrid 1.0.2 release. Usergrid 1.0.2-rc3 includes the following: --- The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.gitf=CHANGELOGhb=1.0.2-rc3 The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.githb=1.0.2-rc3 The current Git commit ID is 5456bfc57329d4a18998c511d0559e0a6cfe5461 The release candidate is available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz The MD5 checksum of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.md5 The signature of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.asc The GPG key used to sign the release are available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/KEYS Please download, verify, and test. On June 26, 2015 at 1:01:02 PM, Dave (snoopd...@gmail.com) wrote: All, I have removed the ARSMaquette font files and made the LICENSE and NOTICE file fixes that Justin Mclean pointed out. Thanks Justin! I tested the portal and it seems to work fine without those files. I propose that we accept the following release candidate as the official Apache Usergrid 1.0.2 release. Usergrid 1.0.2-rc3 includes the following: --- The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.gitf=CHANGELOGhb=1.0.2-rc3 The branch used to create the release candidate is: https://git-wip-us.apache.org/repos/asf?p=incubator-usergrid.githb=1.0.2-rc3 The current Git commit ID is 5456bfc57329d4a18998c511d0559e0a6cfe5461 The release candidate is available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz The MD5 checksum of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.md5 The signature of the release candidate can be found at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/1.0.2-rc3/apache-usergrid-1.0.2-rc3-incubating.tar.gz.asc The GPG key used to sign the release are available at: https://dist.apache.org/repos/dist/dev/incubator/usergrid/KEYS Please download, verify, and test. The vote will close on Mon Jun 29 14:36:55 EDT 2015 [ ] +1 Release this as Apache Usergrid 1.0.2 [ ] +0 [ ] -1 Do not release this as Apache Usergrid 1.0.2 because...
Re: [RESULT] [VOTE] Apache Ignite 1.2.0 release (RC2)
Here are the votes received: +1: - Branko Cibej (binding) - Konstantin Boudnik (binding) - Roman Shaposhnik (binding) - Gianfranco Murador - Sergi Vladykin - Alexey Goncharuk - Valentin Kulichenko - Semyon Boikov +0: - Justin Mclean (binding) --Yakov 2015-06-28 22:56 GMT-07:00 Yakov Zhdanov yzhda...@apache.org: Guys, Voting for 1.2 has passed with 3 votes for (+1) and 1 +0 vote. The pointed issue will be fixed and new release will be submitted shortly, Thanks! --Yakov
Re: Last chance to tell your story at ApacheCON EU 2015
On 29 June 2015 at 10:33, Don Bosco Durai bo...@apache.org wrote: Can I have write permission (wiki id=bosco) or someone update the wiki for me? Don Bosco Durai - Apache Ranger I am still not sure who will talk, but can we put mine as a place holder for now? please remember that for a full talk you need to submit a proposal http://events.linuxfoundation.org/events/apachecon-core-europe/ including a speaker name. hint: 36 hours to go. thanks in advance rgds jan i. Thanks Bosco On 6/29/15, 10:09 AM, jan i j...@apache.org wrote: On 29 June 2015 at 01:28, Anatole Tresch atsti...@gmail.com wrote: Hi all I proposed to talk on Tamaya. I would also do a speed date on Tamaya, but unfortunately I cannot update: https://wiki.apache.org/apachecon/ACEU15Incubator Can somebody do this for me, or give the rights todo so (UID: AnatoleTresch)? updated have fun jan i. Thanks, Anatole 2015-06-27 23:10 GMT+02:00 Roman Shaposhnik ro...@shaposhnik.org: On Sat, Jun 27, 2015 at 1:44 PM, Pierre Smits pierre.sm...@gmail.com wrote: Roman, There is a talk in preparation regarding Trafodion. Great! Once it it submitted, could you please update the wiki? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- *Anatole Tresch* Java Engineer Architect, JSR Spec Lead Glärnischweg 10 CH - 8620 Wetzikon *Switzerland, Europe Zurich, GMT+1* *Twitter: @atsticks* *Blogs: **http://javaremarkables.blogspot.ch/ http://javaremarkables.blogspot.ch/* *Google: atsticksMobile +41-76 344 62 79* - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
[RESULT][VOTE] Release Apache Atlas version 0.5-incubating
Hi folks, Thanks for taking time to review and vote on this release. The vote passes with five binding +1's, six non-binding +1s and no 0/-1's. Binding: Amareshwari Sriramdasu Arun C Murthy Jakob Homan Justin Mclean Chris Douglas Non-binding: Jon Maron Arpit Gupta Harish Butani Suma Shivaprasad Venkat Ranganathan Seetharam Venkatesh I'll start working on the release artifacts and then announce the release to this list. Thanks! Regards, Venkatesh
Re: Last chance to tell your story at ApacheCON EU 2015
On 29 June 2015 at 01:28, Anatole Tresch atsti...@gmail.com wrote: Hi all I proposed to talk on Tamaya. I would also do a speed date on Tamaya, but unfortunately I cannot update: https://wiki.apache.org/apachecon/ACEU15Incubator Can somebody do this for me, or give the rights todo so (UID: AnatoleTresch)? updated have fun jan i. Thanks, Anatole 2015-06-27 23:10 GMT+02:00 Roman Shaposhnik ro...@shaposhnik.org: On Sat, Jun 27, 2015 at 1:44 PM, Pierre Smits pierre.sm...@gmail.com wrote: Roman, There is a talk in preparation regarding Trafodion. Great! Once it it submitted, could you please update the wiki? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- *Anatole Tresch* Java Engineer Architect, JSR Spec Lead Glärnischweg 10 CH - 8620 Wetzikon *Switzerland, Europe Zurich, GMT+1* *Twitter: @atsticks* *Blogs: **http://javaremarkables.blogspot.ch/ http://javaremarkables.blogspot.ch/* *Google: atsticksMobile +41-76 344 62 79*
Re: Last chance to tell your story at ApacheCON EU 2015
Can I have write permission (wiki id=bosco) or someone update the wiki for me? Don Bosco Durai - Apache Ranger I am still not sure who will talk, but can we put mine as a place holder for now? Thanks Bosco On 6/29/15, 10:09 AM, jan i j...@apache.org wrote: On 29 June 2015 at 01:28, Anatole Tresch atsti...@gmail.com wrote: Hi all I proposed to talk on Tamaya. I would also do a speed date on Tamaya, but unfortunately I cannot update: https://wiki.apache.org/apachecon/ACEU15Incubator Can somebody do this for me, or give the rights todo so (UID: AnatoleTresch)? updated have fun jan i. Thanks, Anatole 2015-06-27 23:10 GMT+02:00 Roman Shaposhnik ro...@shaposhnik.org: On Sat, Jun 27, 2015 at 1:44 PM, Pierre Smits pierre.sm...@gmail.com wrote: Roman, There is a talk in preparation regarding Trafodion. Great! Once it it submitted, could you please update the wiki? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- *Anatole Tresch* Java Engineer Architect, JSR Spec Lead Glärnischweg 10 CH - 8620 Wetzikon *Switzerland, Europe Zurich, GMT+1* *Twitter: @atsticks* *Blogs: **http://javaremarkables.blogspot.ch/ http://javaremarkables.blogspot.ch/* *Google: atsticksMobile +41-76 344 62 79* - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [PROPOSAL]Pistachio
Hi Gavin! On Mon, Jun 29, 2015 at 11:27 AM, Gavin Li lyo.ga...@gmail.com wrote: Hi Roman, I've taken a brief look at Geode. Pistachio and Geode both can support similar in-memory consistent HA distributed storage, and the data aware function routing idea is also similar. Here are what I think are different: 1. Pistachio actually uses a very different approach to resolve the problem. Pistachio leverages Apache Helix to do partition management and Apache Kafka as a WAL implementation to leverage the rich features and great performances in those systems. 2. Pistachio supports simple plugin of different storage engines like pure memory or different disk based local storage engines like kyoto cabinet or Rocks DB. On our production we mainly used kyoto cabinet based SSD storage system to optimize for read to resolve the low latency high volume read problem in RTB applications. This is *really* useful to know. Thank you so much for taking time and providing this compare-n-contrast. Helps me a great deal to know where Pistachio is coming from. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [PROPOSAL]Pistachio
Hi I can for sure follow the argument that different design ideas around a problem complex leads to different implementations. My concern is a little bit different. I assume that the developers are in general more interested in the problem complex than the design. If I am correct such projects will be competing for the same developer, and might find it hard to grow. I respect internal competition it can be very fruitful, we just need to make sure that we don´t split a good community into smaller communities that are too small to survive. just my little concern after having read the last couple of emails. rgds jan i. On 29 June 2015 at 20:53, Gavin Li lyo.ga...@gmail.com wrote: Hi Andrew, I agree with you. I've updated the proposal to include a little bit more explanations about the difference with Hadoop. Purely pursuing novelty is never our interest. Instead I believe even for the same problem different design and implementation ideas can make big difference. I think that's why there are many internal competitions in ASF. Having looked at other systems like Ignite and Geode I believe Pistachio is still quite different in design and implementation when solving some common problems like in-memory distributed storage and co-locating computation and data. Thanks, Gavin Li On Fri, Jun 26, 2015 at 12:07 PM, Andrew Purtell apurt...@apache.org wrote: Thanks Gavin. Please let me suggest that novelty is not a requirement for incubation, and a proposal doesn't need to make claims of novelty to be accepted. Should the proposal be accepted for incubation, you may find your new neighbors at Apache can do X where you weren't aware of it. It will be totally up to the new podling if you want to survey the landscape when figuring out how to differentiate, but I do recommend it, it may help you crystallize a community around a real difference and advantage provided by Pistachio. On Mon, Jun 22, 2015 at 7:54 PM, Gavin Li lyo.ga...@gmail.com wrote: Hi Andrew, As we described more in http://yahooeng.tumblr.com/post/116291838351/pistachio-co-locate-the-data-and-compute-for , a very common problem we saw in Hadoop use cases is we often need to persist the previous result of one map reduce job onto HDFS, then the next day we process the new data together with the previous result. Usually the most expensive part is the shuffling part where we need to join the previous data and the new data together. It's so expensive because HDFS doesn't store the data in a partitioned way. So data have to be transferred again and again in the shuffling phase. Instead, in Pistachio we do the computation right on top of the partitioned storage layer, so that the previous result is always stored in a partitioned way, so shuffling can be avoided. Expensive IO and roundtrips can thus be avoided so that much better performance can be achieved. The other difference is in Pistachio we can do computation based on in-memory storage with data replication. Different from the in-memory computation in Spark, the storage can be in-memory here. Please let me know if I'm not clear enough. Thanks, Gavin Li On Mon, Jun 22, 2015 at 7:53 PM, Andrew Purtell apurt...@apache.org wrote: It was a simple question, and not meant to suggest anything one way or other regarding my opinion of this proposal. On Monday, June 22, 2015, John D. Ament johndam...@apache.org wrote: On Mon, Jun 22, 2015 at 10:26 PM Andrew Purtell apurt...@apache.org javascript:; wrote: Pistachio can easily embed computation to the storage layer to achieve the best data locality to improve the computation performance significantly which is an innovative model comparing with the normal ways where the storage and compute are independent to each other. Have you heard of something called Hadoop? Regardless of whether he has or not - what's your point? The ASF has historically not denied the entry of new projects just because their domain intersects with another project's. On Thu, Jun 18, 2015 at 10:17 AM, Gavin Li lyo.ga...@gmail.com javascript:; wrote: Hi, I want to propose project Pistachio to enter Apache Incubator. Below please find the proposal. Thanks, Gavin Li = Pistachio = == Abstract == Pistachio is a fault-tolerant low latency distributed storage system which enables simple embedding the computation to the storage layer to achieve best data locality. It evolves from Yahoo’s global user profile storage system. == Proposal == Pistachio is a distributed key value store system with fault tolerance
Re: [RESULT] [VOTE] Apache Ignite 1.2.0 release (RC2)
Ladies would be even better. :-) Op maandag 29 juni 2015 heeft Branko Čibej br...@apache.org het volgende geschreven: On 29.06.2015 15:36, Marvin Humphrey wrote: On Sun, Jun 28, 2015 at 10:56 PM, Yakov Zhdanov yzhda...@apache.org javascript:; wrote: Guys, It's nice to be informal, but we're not just guys. :) Aye; next time, please address mails to laddies and gentlemen. :) -- Brane - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org javascript:; For additional commands, e-mail: general-h...@incubator.apache.org javascript:; -- Pierre Smits *ORRTIZ.COM http://www.orrtiz.com* Services Solutions for Cloud- Based Manufacturing, Professional Services and Retail Trade http://www.orrtiz.com
