Sorry forgot to mention this in previous email. I wanted to ask one thing. How to dive into a large codebase ?. I read some answers suggesting see old commits how the project was developed and debug to see control flow.
On Mon, Apr 25, 2016 at 3:26 PM, onkar shedge <shedge31on...@gmail.com> wrote: > Hello Alex, > > Here is a link to my blog[1]. > I have added google calendar with three weeks events. I haven't written > any posts about my progress, Will do soon. > > 1] http://gsoc2016onkar.blogspot.in/ > > On Sun, Apr 24, 2016 at 4:34 PM, Alexander Bezzubov <b...@apache.org> > wrote: > >> Thank you Onkar, >> >> Its great to have you on board and looking forward experiments with P2P >> notebook storage! >> >> It's great idea to keep list posted on your progress as well as having >> deeper writeups on a personal blog, please feel free to share links, etc. >> >> It would be good if in emails you could not only list things that you did >> last week, but also to include a brief plan for the next week. This way it >> should be easier for me as a mentor to align efforts in the same >> direction. >> >> It is going to be an exciting project! >> >> -- >> Alex >> >> On Sat, Apr 23, 2016, 22:55 onkar shedge <shedge31on...@gmail.com> wrote: >> >> > Hello zeppelin community, >> > >> > Thanks for giving me the opportunity. I will get myself more familiar >> with >> > the codebase and ask questions on mailing list about my doubts . Also I >> > will post updates weekly/(4 day interval) what I understood,what I >> > worked/read, on my blog. >> > >> > >> > Regards, >> > Onkar Shedge. >> > >> > On Fri, Mar 25, 2016 at 10:59 AM, Alexander Bezzubov <b...@apache.org> >> > wrote: >> > >> > > Hi Onkar, >> > > >> > > that sounds great, thank you. >> > > >> > > Looking forward helping with this project though the summer! >> > > >> > > -- >> > > Alex >> > > >> > > On Fri, Mar 25, 2016 at 2:27 PM, onkar shedge < >> shedge31on...@gmail.com> >> > > wrote: >> > > >> > > > Thanks Sir, >> > > > I have made the changes: diagram, Deliverables removed research. >> And >> > > > support for one more p2p storage. >> > > > Also I have uploaded the pdf. >> > > > >> > > > Regards, >> > > > Onkar Shedge >> > > > >> > > > On Thu, Mar 24, 2016 at 3:49 PM, Alexander Bezzubov <b...@apache.org >> > >> > > > wrote: >> > > > >> > > > > Hi Onkar, >> > > > > >> > > > > thank you for sharing a blog and even a video of your >> explorations in >> > > > > preparation for the project. >> > > > > Your timeline and proposal looks very strong and it seems that you >> > > > relevant >> > > > > experience for this project. >> > > > > >> > > > > On the GSoC scope - it would be a good start with a storage >> > > > > implementation(s) that result in dat://, magnet:// or ipfs:// >> links >> > > for a >> > > > > notebooks, as a first step. From there sharing the link can be >> done >> > > > through >> > > > > any communication medium (IM, email, etc) and importing such link >> in >> > > > > Zeppelin instance is a matter of changing "Import" dialog\backend >> > > inside >> > > > > Zeppelin to support it. And then proper note >> versioning\modification >> > > > > support, as well as make sure that it plays nicely with multiple >> > > > > NotebookRepo plugged in though NotebookRepoSync [1]. >> > > > > I would expect at least those use-cases to be implemented as a >> part >> > of >> > > > the >> > > > > GSoC project. >> > > > > >> > > > > On the deliverables: >> > > > > - "research" is not a deliverable item, may be would be better to >> > put >> > > a >> > > > > "Report on results of the research, covering suitability of each >> p2p >> > > > > network\stack for the Zeppelin case" >> > > > > >> > > > > Would you be willing to, just as an extra bonus material, take >> care >> > of >> > > > more >> > > > > the one p2p NotebookRepo implementation, of course in case if time >> > > > permits? >> > > > > >> > > > > I have also added few comments to the doc itself. >> > > > > >> > > > > Please feel free to incorporate feedback do not forget to submit >> the >> > > > final >> > > > > pdf to google before the deadline tomorrow! >> > > > > >> > > > > 1. >> > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java#L40 >> > > > > >> > > > > -- >> > > > > Alex >> > > > > >> > > > > >> > > > > >> > > > > On Thu, Mar 24, 2016 at 12:11 PM, onkar shedge < >> > > shedge31on...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Hi moon, >> > > > > > Thanks for your idea. You talked about single online storage and >> > then >> > > > > > sharing with others like(Google docs). Then handling fault >> > tolerance >> > > as >> > > > > > multiple instances would change the same storage repo. >> > > > > > I was thinking till now that P2P implementation would be >> generating >> > > the >> > > > > > torrent file in case of Bittorrent or dat ://......dat link in >> case >> > > of >> > > > > dat >> > > > > > protocol and that file would be changed/versioned as changes are >> > made >> > > > new >> > > > > > hashes will be generated. So the question now is how to share >> the >> > > > torrent >> > > > > > file or dat link to other peers? >> > > > > > >> > > > > > Is it that there would be a Zeronet site for having a list of >> peers >> > > > > online >> > > > > > and each user sharing his notebooks which he has chosen to >> share. >> > > > > > >> > > > > > On Thu, Mar 24, 2016 at 7:59 AM, moon soo Lee <m...@apache.org> >> > > wrote: >> > > > > > >> > > > > > > Hi, >> > > > > > > >> > > > > > > Scope of ZEPPELIN-683 is implementing a Zeppelin NotebookRepo >> [1] >> > > > based >> > > > > > on >> > > > > > > one of P2P technology. I think ZEPPELIN-683 leads to very >> > > interesting >> > > > > > > challenge (as a future work). >> > > > > > > >> > > > > > > I can see characteristics of P2P technology based NotebookRepo >> > as, >> > > > > > > >> > > > > > > * Massively (globally) scalable. >> > > > > > > * Very Elastic. Any peer can join and leave at any time. >> > > > > > > >> > > > > > > Therefore i can see following possibilities and challenges. >> > > > > > > >> > > > > > > * Make every zeppelin instance connect to the single storage >> > > network. >> > > > > > > * Then it is possible to provide user unlimited online >> notebook >> > > > > storage. >> > > > > > > * And there will be nicer way to share notebook to the other >> > > people. >> > > > > > > * Zeppelin currently does handle the case multiple zeppelin >> > > instance >> > > > > > share >> > > > > > > single storage. To leverage advantage of P2P technology based >> > > > > > NotebookRepo, >> > > > > > > Zeppelin need to aware that storage can be updated by other >> > > Zeppelin >> > > > > > > instances. This could be challenging job. >> > > > > > > * I think it's very much related to support of fault >> tolerance. >> > > > > > > >> > > > > > > I think ZEPPELIN-683 is very wide open to be evolved. Please >> feel >> > > > free >> > > > > to >> > > > > > > add your idea. >> > > > > > > >> > > > > > > Thanks, >> > > > > > > moon >> > > > > > > >> > > > > > > [1] >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepo.java >> > > > > > > >> > > > > > > >> > > > > > > On Tue, Mar 22, 2016 at 10:34 PM onkar shedge < >> > > > shedge31on...@gmail.com >> > > > > > >> > > > > > > wrote: >> > > > > > > >> > > > > > > > Just wondering isn't this project important as the other two >> > > Apache >> > > > > > Beam >> > > > > > > > interpreter and Sample Notebooks ? >> > > > > > > > >> > > > > > > > On Tue, Mar 22, 2016 at 10:59 AM, onkar shedge < >> > > > > > shedge31on...@gmail.com> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > Hello, >> > > > > > > > > Apologies for late reply. >> > > > > > > > > I have spend time understanding the protocol.I installed >> all >> > > the >> > > > > > three >> > > > > > > > > techs and tried them. >> > > > > > > > > Also I read about the docs, whitepapers[1].I read the >> > > bittorent >> > > > > > > protocol >> > > > > > > > > and wrote a blog[2] about it using jekyll. Please do watch >> > the >> > > > > video. >> > > > > > > > > >> > > > > > > > > I have written the proposal[3].Your feedback are >> welcomed. I >> > am >> > > > > > > confused >> > > > > > > > , >> > > > > > > > > exactly what to write in implementation part.right now I >> have >> > > > > written >> > > > > > > > about >> > > > > > > > > the current implementation. >> > > > > > > > > >> > > > > > > > > The available clients are >> > > > > > > > > IPFS - [4] Java >> > > > > > > > > dat : browserify [5], desktop[6], [7] python . If not >> > > available, >> > > > > > > should I >> > > > > > > > > consider writing own Java client? is it doable? >> > > > > > > > > Zeronet : I didn't understand how notebooks can be shared >> > with >> > > > > > zeronet >> > > > > > > > > which serves sites?. >> > > > > > > > > >> > > > > > > > > Please comment. I guess there are many mistakes. Thank >> You. >> > > > > > > > > >> > > > > > > > > 1] >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/ipfs/papers/raw/master/ipfs-cap2pfs/ipfs-p2p-file-system.pdf >> > > > > > > > > 2] >> > > https://onkarshedge.github.io/2016/03/16/peeking-in-p2p.html >> > > > > > > > > https://www.youtube.com/watch?v=WxX0AjqQ28g >> > > > > > > > > 3] >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://docs.google.com/document/d/1GVu_LEi8o6wnnoj9vrt07j8ByiDCLemYh9F9ERLyni8/edit?usp=sharing >> > > > > > > > > 4] https://github.com/ipfs/java-ipfs-api >> > > > > > > > > 5] https://github.com/karissa/dat-browserify >> > > > > > > > > 6] https://github.com/karissa/dat-desk >> > > > > > > > > 7]https://github.com/karissa/datpy >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Thu, Mar 10, 2016 at 6:28 PM, Alexander Bezzubov < >> > > > > b...@apache.org> >> > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > >> Hi Onkar, >> > > > > > > > >> >> > > > > > > > >> great to hear the you are interested and thank you for >> > sharing >> > > > the >> > > > > > > > example >> > > > > > > > >> notebook that you'v built, preview [0] looks great. >> > > > > > > > >> >> > > > > > > > >> I encourage you review this mailing list archives very >> > > > carefully, >> > > > > > > > looking >> > > > > > > > >> for the advices to other students on how to get started >> with >> > > > > > zeppelin >> > > > > > > > and >> > > > > > > > >> proceed with proposal draft [1] [2] [3]. >> > > > > > > > >> >> > > > > > > > >> Research, as well as publishing the results of such in >> > > > wiki\blogs >> > > > > > > should >> > > > > > > > >> be >> > > > > > > > >> substantial part of this project. The expectations are >> > though >> > > > that >> > > > > > you >> > > > > > > > >> will >> > > > > > > > >> be able to familiarize yourself with the p2p protocols at >> > > least >> > > > a >> > > > > > bit >> > > > > > > > >> before starting actual gsoc project. Engaging and >> bridging >> > > > > multiple >> > > > > > > > >> project >> > > > > > > > >> communities is very welcome as well. Next steps could >> build >> > > > > building >> > > > > > > > >> low-fi >> > > > > > > > >> PoC using JVM tools, and then a hi-fi one, using >> pluggable >> > > > > > Repository >> > > > > > > > >> abstraction [4] >> > > > > > > > >> >> > > > > > > > >> Hope this helps and looking forward your proposal draft: >> > > > plaintext >> > > > > > in >> > > > > > > > wiki >> > > > > > > > >> [5] or a link to a google doc will work nicely to gather >> the >> > > > > > feedback >> > > > > > > > and >> > > > > > > > >> engage with potential mentors. >> > > > > > > > >> >> > > > > > > > >> -- >> > > > > > > > >> Alex >> > > > > > > > >> >> > > > > > > > >> 0. https://www.zeppelinhub >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> .com/viewer/notebooks/aHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL29ua2Fyc2hlZGdlL3NhbXBsZS1ub3RlYm9va3MvbWFzdGVyLzJCRllGVVpDUC9ub3RlLmpzb24 >> > > > > > > > >> 1. http://markmail.org/thread/abw6hoayuvi54ghk >> > > > > > > > >> 2. http://markmail.org/thread/j53j7d4rsiisewfb >> > > > > > > > >> 3. http://markmail.org/message/naocktanol5iuot3 >> > > > > > > > >> 4. http://zeppelin.incubator.apache >> > > > > > > > >> .org/docs/0.6.0-incubating-SNAPSHOT/storage/storage.html >> > > > > > > > >> 5. https://cwiki.apache >> > > > > > > > >> >> .org/confluence/display/ZEPPELIN/Google+Summer+Of+Code+2016 >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> On Wed, Mar 9, 2016 at 11:56 PM, onkar shedge < >> > > > > > > shedge31on...@gmail.com> >> > > > > > > > >> wrote: >> > > > > > > > >> >> > > > > > > > >> > Hello Alexander, >> > > > > > > > >> > I am Onkar from PICT, Pune India. I am interested in >> the >> > > > project >> > > > > > > idea >> > > > > > > > >> > regarding Notebook distributed Storage using P2P >> > protocols. >> > > > > > > > >> > In order to contribute and aid in this project, I have >> > been >> > > > > > working >> > > > > > > > with >> > > > > > > > >> > Zeppelin Notebooks.This is a link to one of my sample >> > > notebook >> > > > > > which >> > > > > > > > >> uses a >> > > > > > > > >> > dataset about Indian school data from data.gov.in: >> > > > github-repo >> > > > > > > > >> > < >> > > > > > > > >> > >> > > > > > > > >> >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/onkarshedge/sample-notebooks/blob/master/2BFYFUZCP/note.json >> > > > > > > > >> > > >> > > > > > > > >> > . >> > > > > > > > >> > >> > > > > > > > >> > I am familiar with IPython it also uses similar >> > json(.ipynb) >> > > > way >> > > > > > to >> > > > > > > > >> > represent notebook. So as per my understanding we have >> to >> > > > divide >> > > > > > the >> > > > > > > > >> json >> > > > > > > > >> > file into chunks and store in a distributed manner >> > according >> > > > to >> > > > > > > > >> protocol. >> > > > > > > > >> > While I am familiar with the basics of the product and >> > have >> > > a >> > > > > > clear >> > > > > > > > >> idea of >> > > > > > > > >> > what is required by the problem statement, I am not >> quite >> > > sure >> > > > > how >> > > > > > > to >> > > > > > > > >> > proceed about it. I would appreciate your guidance >> > regarding >> > > > the >> > > > > > > > same. I >> > > > > > > > >> > was thinking about starting with a brief comparative >> study >> > > of >> > > > > the >> > > > > > > > >> protocols >> > > > > > > > >> > suggested( dat, ipfs, zeronet). I hope to hear your >> views >> > > > about >> > > > > > > this. >> > > > > > > > >> > >> > > > > > > > >> > Thanking you, >> > > > > > > > >> > Onkar Shedge >> > > > > > > > >> > >> > > > > > > > >> >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
