Hi moon, Thanks for your idea. You talked about single online storage and then sharing with others like(Google docs). Then handling fault tolerance as multiple instances would change the same storage repo. I was thinking till now that P2P implementation would be generating the torrent file in case of Bittorrent or dat ://......dat link in case of dat protocol and that file would be changed/versioned as changes are made new hashes will be generated. So the question now is how to share the torrent file or dat link to other peers?
Is it that there would be a Zeronet site for having a list of peers online and each user sharing his notebooks which he has chosen to share. On Thu, Mar 24, 2016 at 7:59 AM, moon soo Lee <m...@apache.org> wrote: > Hi, > > Scope of ZEPPELIN-683 is implementing a Zeppelin NotebookRepo [1] based on > one of P2P technology. I think ZEPPELIN-683 leads to very interesting > challenge (as a future work). > > I can see characteristics of P2P technology based NotebookRepo as, > > * Massively (globally) scalable. > * Very Elastic. Any peer can join and leave at any time. > > Therefore i can see following possibilities and challenges. > > * Make every zeppelin instance connect to the single storage network. > * Then it is possible to provide user unlimited online notebook storage. > * And there will be nicer way to share notebook to the other people. > * Zeppelin currently does handle the case multiple zeppelin instance share > single storage. To leverage advantage of P2P technology based NotebookRepo, > Zeppelin need to aware that storage can be updated by other Zeppelin > instances. This could be challenging job. > * I think it's very much related to support of fault tolerance. > > I think ZEPPELIN-683 is very wide open to be evolved. Please feel free to > add your idea. > > Thanks, > moon > > [1] > > https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepo.java > > > On Tue, Mar 22, 2016 at 10:34 PM onkar shedge <shedge31on...@gmail.com> > wrote: > > > Just wondering isn't this project important as the other two Apache Beam > > interpreter and Sample Notebooks ? > > > > On Tue, Mar 22, 2016 at 10:59 AM, onkar shedge <shedge31on...@gmail.com> > > wrote: > > > > > Hello, > > > Apologies for late reply. > > > I have spend time understanding the protocol.I installed all the three > > > techs and tried them. > > > Also I read about the docs, whitepapers[1].I read the bittorent > protocol > > > and wrote a blog[2] about it using jekyll. Please do watch the video. > > > > > > I have written the proposal[3].Your feedback are welcomed. I am > confused > > , > > > exactly what to write in implementation part.right now I have written > > about > > > the current implementation. > > > > > > The available clients are > > > IPFS - [4] Java > > > dat : browserify [5], desktop[6], [7] python . If not available, > should I > > > consider writing own Java client? is it doable? > > > Zeronet : I didn't understand how notebooks can be shared with zeronet > > > which serves sites?. > > > > > > Please comment. I guess there are many mistakes. Thank You. > > > > > > 1] > > > > > > https://github.com/ipfs/papers/raw/master/ipfs-cap2pfs/ipfs-p2p-file-system.pdf > > > 2] https://onkarshedge.github.io/2016/03/16/peeking-in-p2p.html > > > https://www.youtube.com/watch?v=WxX0AjqQ28g > > > 3] > > > > > > https://docs.google.com/document/d/1GVu_LEi8o6wnnoj9vrt07j8ByiDCLemYh9F9ERLyni8/edit?usp=sharing > > > 4] https://github.com/ipfs/java-ipfs-api > > > 5] https://github.com/karissa/dat-browserify > > > 6] https://github.com/karissa/dat-desk > > > 7]https://github.com/karissa/datpy > > > > > > > > > > > > On Thu, Mar 10, 2016 at 6:28 PM, Alexander Bezzubov <b...@apache.org> > > > wrote: > > > > > >> Hi Onkar, > > >> > > >> great to hear the you are interested and thank you for sharing the > > example > > >> notebook that you'v built, preview [0] looks great. > > >> > > >> I encourage you review this mailing list archives very carefully, > > looking > > >> for the advices to other students on how to get started with zeppelin > > and > > >> proceed with proposal draft [1] [2] [3]. > > >> > > >> Research, as well as publishing the results of such in wiki\blogs > should > > >> be > > >> substantial part of this project. The expectations are though that you > > >> will > > >> be able to familiarize yourself with the p2p protocols at least a bit > > >> before starting actual gsoc project. Engaging and bridging multiple > > >> project > > >> communities is very welcome as well. Next steps could build building > > >> low-fi > > >> PoC using JVM tools, and then a hi-fi one, using pluggable Repository > > >> abstraction [4] > > >> > > >> Hope this helps and looking forward your proposal draft: plaintext in > > wiki > > >> [5] or a link to a google doc will work nicely to gather the feedback > > and > > >> engage with potential mentors. > > >> > > >> -- > > >> Alex > > >> > > >> 0. https://www.zeppelinhub > > >> > > >> > > > .com/viewer/notebooks/aHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL29ua2Fyc2hlZGdlL3NhbXBsZS1ub3RlYm9va3MvbWFzdGVyLzJCRllGVVpDUC9ub3RlLmpzb24 > > >> 1. http://markmail.org/thread/abw6hoayuvi54ghk > > >> 2. http://markmail.org/thread/j53j7d4rsiisewfb > > >> 3. http://markmail.org/message/naocktanol5iuot3 > > >> 4. http://zeppelin.incubator.apache > > >> .org/docs/0.6.0-incubating-SNAPSHOT/storage/storage.html > > >> 5. https://cwiki.apache > > >> .org/confluence/display/ZEPPELIN/Google+Summer+Of+Code+2016 > > >> > > >> > > >> On Wed, Mar 9, 2016 at 11:56 PM, onkar shedge < > shedge31on...@gmail.com> > > >> wrote: > > >> > > >> > Hello Alexander, > > >> > I am Onkar from PICT, Pune India. I am interested in the project > idea > > >> > regarding Notebook distributed Storage using P2P protocols. > > >> > In order to contribute and aid in this project, I have been working > > with > > >> > Zeppelin Notebooks.This is a link to one of my sample notebook which > > >> uses a > > >> > dataset about Indian school data from data.gov.in: github-repo > > >> > < > > >> > > > >> > > > https://github.com/onkarshedge/sample-notebooks/blob/master/2BFYFUZCP/note.json > > >> > > > > >> > . > > >> > > > >> > I am familiar with IPython it also uses similar json(.ipynb) way to > > >> > represent notebook. So as per my understanding we have to divide the > > >> json > > >> > file into chunks and store in a distributed manner according to > > >> protocol. > > >> > While I am familiar with the basics of the product and have a clear > > >> idea of > > >> > what is required by the problem statement, I am not quite sure how > to > > >> > proceed about it. I would appreciate your guidance regarding the > > same. I > > >> > was thinking about starting with a brief comparative study of the > > >> protocols > > >> > suggested( dat, ipfs, zeronet). I hope to hear your views about > this. > > >> > > > >> > Thanking you, > > >> > Onkar Shedge > > >> > > > >> > > > > > > > > >
