On Tue, Jun 6, 2017 at 6:54 PM, Jean Bigboute <[email protected]> wrote: > Matthias and Team: > > On Tuesday, June 6, 2017 at 5:04:29 PM UTC-7, Matthias Bussonnier wrote: >> >> On Tue, Jun 6, 2017 at 4:12 PM, Jean Bigboute <[email protected]> wrote: >> > Thank you for posting this and I hope you are successful. As an >> > occasional Jupyter user, I have struggled to understand its use for >> > collaborative work when so much depends on future developments and >> > additional infrastructure. In my work environment we can't deploy IT >> > services such as servers, containers etc. on our own. We don't always have >> > access to help sites either. It is hard to know when to push and when to >> > punt. I don't feel so alone any more. >> >> The usage for collaborative work should become easier soon. We had our >> dev meeting last week, and a version that allow some[1] syncing via >> google drive is almost ready (Ian did and does a fantastic job). You >> will probably see that in JupyterLab in one of the next releases (at >> least as an extension). ... > > > > Thanks for your prompt response. With due respect, I would like to point > out that Google Drive doesn't solve the problem. In corporate environments > where proprietary data and algorithms are at stake, it is highly unlikely > that users will be allowed access to Google Drive, Github, NBViewer, > StackOverflow, etc. That is certainly true with my employer so I can only > post completely generic questions and then again from my own account. In > the present IT security environment (which is only going to get more > restrictive) any type of added service requires multiple levels of scrutiny > and review all tied into business cases. Bypassing the system or attempting > to do so will be detected quickly and the people responsible will get fired. > Unless the product or service is incredibly compelling (see below) no one > will go through the effort to get a product approved because the odds of > success are practically zero.
But, we have to start somewhere. We definitively do not have the bandwidth, and likely miss experience in designing realtime collaboration protocols and implementing a server that does it. At least by having GoogleDrive as a backend we can show that the frontend does support that, and hopefully someone will implement a backend that can be hosted on site by those that desire it. We are aware that not everybody can use GoogleDrive, or any other services because of network restriction or simply because they are on Air-Gapped network. Though we are funded by non-profit and the majority of our users can, so that's the first step we took. We did leave room for changing the implementation though. It may not be one of our priority. Services like CoCalc (ex SageMathCloud) do provide on premise deployment of Jupyter with RT collaboration that are completely custom. CoCalc have some restriction on what you can do in notebooks – for security reasons – and is one of the answer we give to users that have custom needs. WE understand that you may not have your word to say in your institution, but if a lot of users really want something like this, you can likely contract with William (CoCalc creator), or even Continuum (Anaconda), I'm pretty sure they will be happy for you to consult with one of their employee under NDA. In the end the success of these companies and that they can sell support is in part what allow the project to be successful, survive financially and be open source. Being on the Open Source and Academic Side, even if I care about you as a user, and I will do my best to help you, this is (I think) the right place to tell "This one place where commercial support is available". > > Additionally, I looked up JupyterLab just now and read that > "JupyterLab computational environment. This is a very early preview, and is > not suitable for general usage yet. " > This has been the case for quite some time. Well we should probably update this, we don't want to say it's ready for prime time yet (but soon, and I'll let the pleasure to announce that to the team behind it). Though it is quite stable , already deploy in a couple of commercial offer (Last one we learned of was r-brain). It's just that the standard of "Ready" for JupyterLab will be way higher than the current notebook, and the team is spending a large chunk of time to make sure the transition is as seamless a possible. We want people that try jupyterLab one to not go back to the classical notebook because we break their workflow as otherwise they will go back and not bother to re-try another time. And if there is one thing we don't want it is the Python 2 vs Python 3 debacle once more. >> >> Finding the right balance between consumers of Jupyter that want to >> deploy that on large number of nodes with hundreds of users – ... > > [deletia] >> >> >> The documentation can definitively get improvement – and as you might >> have seen in an earlier mail Jessica is starting to work with us on >> that and we are thrilled – but we can't cover all the ground. We also >> tend to assume more and more that people have internet to get help if >> you have an interest if having our docs offline and we don't provide a >> link please open an issue and we'll try to find time to make that >> happen. If you have experience in deploying in a difficult environment >> and have manage to go around some pain point, please let us know – or >> even better make a PR against our documentation. We'd love to have a >> clearer way of expressing the advantages drawback of each deployment >> but so far didn't had the time to sit down and write these as things >> are moving really fast even for us. > > > And this is the key point. I have been following IPython then Jupyter now > Jupyter +Lab and +Hub since about 2014. The pace of developments is > incredibly fast and I admire the team for doing so much. These days most of the new things happening in Jupyter are actually not from the core team. > But, I think the > mindset is very much from academia. I love academia, its ethos, and the > exuberance but Jupyter is now fully promoting into other areas which have > very different cultures and needs. Documentation, user testing, stability, > self-containment,... these are all important and I don't think are being > given sufficient consideration. That's weird to hear as most of the development in JupyterLab these days is done by corporate sponsors and get a lot of feedback from internal needs. You can quickly grep through the main commiters of Lab and you will see that mostly they work at Bloomberg and Continuum.io. I'm pretty sure they have internal deployment with customization and that some of their network policies are likely as constrained as your. So yes their IT does deploy this – which is not the case for you. But they do run it, which should be a large selling point. Microsoft, IBM,... are also deploying notebooks often with commercial offers, and while maybe not as much involve in day-to day commits to the core components on the GitHub side they do share feedback and contribute a lot of extensions. > There's a fascinating blogpost on making > Jupyter Notebooks more reproducible: > > https://jakevdp.github.io/blog/2017/03/03/reproducible-data-analysis-in-jupyter/ > > The author uses Github as a repository and a standalone text editor among > other tools to accomplish this. In one of the comments, he remarks "I agree > that collaborating on notebooks is tough. I mostly avoid it, and instead > collaborate on plain-text documents like markdown and code files." Then, he > recommends the nbdime project as another workaround. I bring him into this > discussion because I think he is incredibly smart, an advocate for the > project, highly experienced with the tool, and yet still has to find ways > to do things which, on the surface, are asserted as built in. Yes, and jake did a lot of efforts, complained and ask a lot of questions as well, even wrote books using notebooks, which was not an easy thing. We can't perfectly accommodate everyone workflow (we do our best), things are getting way easier than they were, and the Jupyter is flexible enough to accommodate custom format if you wish – [1][2][3] store notebooks as "flat" formats editable with any text editor ; with the constraint of partially losing some guaranties like metadata. And so far a lot of people are using Git for full notebooks and it works. There is some frustration – yes – but so far every-time someone was frustrated enough they came and fixed it ; in the core or as an extension. That's how most of us started this project to scratch our own itch and if someone take the time to scratch their itch; write a flat format that have a good enough experience and is widespread enough we'll gradely move that into the core. > Please excuse the frustration that is coming though. No worries, we are happy that user like you come and express these. It's the kind of feedback that we can cite when going through agencies/lab/companies and show them that their users are in needs of change and functionalities but that we don't have the bandwidth/funding. And hopefully convince them to either fund us or contribute back. > I think I am a > well-wisher to the project but after nearly three years of observation, I > suggest, again respectfully, that you won't always be reaching the elite. > If Jupyter is to succeed, it will have to be used by rank-and-file slobs > like myself in environments where we have pressures to produce. We aren't > dumb but neither can we 't be diving into the gizzards of servers, source > code, and containers. We need stuff to work so we can get on with our jobs. > 'Real Soon Now' only goes so far. Like you we have limited time, and can only do a best-effort to provide things that work out of the box. The simplicity with how you can get things today is so much better than a couple of years ago. We are not yet at the install simplicity of something like the AppStore or similar but it is not _that_ far, and hopefully with your help we can get to it faster. Thanks again for reaching out, and keep in mind that if you can't ask things publicly you can alway try to reach our via private channels. Cheers, -- Matthias P.S: Apologies for the too long response but it's already written... [1] https://github.com/rossant/ipymd [2] https://github.com/aaren/notedown [3] https://github.com/takluyver/nbexplode -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CANJQusWqJD9sth4DiRbOd4349eS3VbbipjV0kgZa%3DsTPW%3DtNDw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
