On Mon, Oct 23, 2017 at 3:19 PM, Emmanuel Charpentier <emanuel.charpent...@gmail.com> wrote: > > > Le lundi 23 octobre 2017 14:32:03 UTC+2, Erik Bray a écrit : >> >> On Mon, Oct 23, 2017 at 12:27 PM, Emmanuel Charpentier >> <emanuel.c...@gmail.com> wrote: >> > >> > >> > Le lundi 23 octobre 2017 12:17:06 UTC+2, Erik Bray a écrit : >> >> >> >> On Mon, Oct 23, 2017 at 11:57 AM, Emmanuel Charpentier >> >> <emanuel.c...@gmail.com> wrote: >> >> > Dear Erik, >> >> > >> >> > Le lundi 23 octobre 2017 11:16:05 UTC+2, Erik Bray a écrit : >> >> >> >> >> >> On Thu, Oct 19, 2017 at 5:19 PM, Emmanuel Charpentier >> >> >> <emanuel.c...@gmail.com> wrote: >> >> >> > Again : R is not only a software package but also an ecosystem. >> >> >> > The >> >> >> > 11638 >> >> >> > (as of today) packages available to R users are a large part of R >> >> >> > usefulness >> >> >> > to its users. So, "disabling downloads from CRAN" is *NOT* fine >> >> >> > (to >> >> >> > them, at >> >> >> > least...). >> >> >> >> >> >> I'm not saying it shouldn't be possible to install R packages at >> >> >> all, >> >> >> any less than it should be possible to install Python packages. My >> >> >> point here was that with Python, for example, one can manually >> >> >> download a package tarball or wheel from PyPI using, say, curl for >> >> >> example (maybe if they running on an air-gapped network this was >> >> >> done >> >> >> on a separate machine and sneaker-netted over, etc.). pip can then >> >> >> install from the manually downloaded package file. >> >> >> >> >> >> I don't know if the same is possible with R but you'd think it >> >> >> should >> >> >> be. However R installs packages the sequence still has to be >> >> >> something like >> >> >> >> >> >> 1) Download package from CRAN >> >> >> 2) Verify that package downloaded successfully (maybe it does this >> >> >> maybe it doesn't) >> >> >> 3) Install the package >> >> >> >> >> >> So it should be possible to do steps 1 and 2 manually, and then skip >> >> >> straight to 3. Admittedly running R on an air-gapped network is >> >> >> probably not a situation the developers have in mind but I have very >> >> >> little doubt that the use case exists. >> >> > >> >> > >> >> > Indeed, it *is* possible to install a manually downloaded package >> >> > (not >> >> > as >> >> > straightforward as the automatic download-and-install default >> >> > method). >> >> > But >> >> > the problem isn't here : >> >> > >> >> > There are a large number of CRAN packages (11660 as of this morning). >> >> > More >> >> > and more of these packages have mutual dependencies, which are easily >> >> > accounted for bu install.packages() but are a pain to deal with >> >> > "manually". >> >> > >> >> > In fact, the problem (roughly) has same magnitude as the maintainance >> >> > of >> >> > your operating system : it *is* indeed possible to maintain a >> >> > Unix/Linux >> >> > installation using only tarballs conveyed to the system via >> >> > sneakersnet, >> >> > but >> >> > it' certainly a) not fun and b) chronophage to the extreme... >> >> > >> >> > As it happened with Linux distributions, these intermutual >> >> > dependencies, >> >> > which started scarce and lightweight, are getting more and more >> >> > important. >> >> > My prediction (or prognostication, or oracle, if you wish...;-) is >> >> > that >> >> > they >> >> > will reach the level of complexity of operating system distributions >> >> > in >> >> > an >> >> > amount of time short enough for this problem to be of interest to all >> >> > but >> >> > the oldest (i. e. closest to retirement or death) of R users. >> >> >> >> I understand all that, and the same is true of some Python packages as >> >> well (which is why sometimes Sage winds up having to add quite a few >> >> spkgs for Python packages--see for example the dependencies of Flask). >> >> But that's simply a matter that has to be dealt with, and people do >> >> (*I* do, even, in some cases). >> >> >> >> My point here is not that that that's a nice thing to foist on average >> >> users (it isn't). Just that it *should* be possible regardless, >> >> without patching or anything else. My concern is just why are we >> >> maintaining a patch just to be able to build R without SSL... >> > >> > >> > ...which is *exactly* my point ! >> > >> > Presently, the only reason ever given to this patch (which lifts >> > upstream >> > R's requirement on an https-enabled library) is to be able to build Sage >> > without OpenSSL because it's not, in some eyes, GPL-compatible. >> > >> > I balk at the idea of maintaining this patch again and again for this >> > issue, >> > which can be solved either by depending on a systemwide openssl (but >> > there >> > goes our "batteries included" philosophy) or including OpenSSL >> > (currently >> > not possible for licensing reasons, but that should change). >> >> I agree. >> >> > I also balk at the idea of shipping a crippled pip. >> >> It's not crippled if you don't need it to install from HTTPS which not >> everyone does. > > > Unless I'm mistaken, pip won't instamm from Pipy if https isn't enabled. > Isn't that an important source of Python modules ?
*sigh* I feel like I've been over this but pip works without PyPI and this is *by design*. >> Okay. I think I see the problem here. We're talking about multiple >> issues simultaneously and some things are getting confused--some >> streams crossed. > > > Indeed : SSL is a *general* tool, and its absence affects *many* things. > >> >> First: There is the *general* issue of whether or not these packages >> need any kind of SSL support in order to function. This is an issue >> *completely* independent from Sage (except insofar as we may or may >> not need to maintain some patches to ensure that they don't require >> SSL, but that is a separate issue that I will get to next). From the >> general standpoint of pip's design and functionality, it does not in >> any way require SSL to work. There was a time when it did, but only >> due to a bug where some module assumed the `ssl` module would always >> be importable which is not necessarily the case. There are completely >> legitimate reasons to either *explicitly* run pip without SSL support, >> and there are completely legitimate reasons to simply not need it at >> all or not care. > > > Okay : we have another confusion here : you want to *run* pip (or R) > without SSL support. The problem (at least with R) is that we shouldn't be > able to *build* R without SSL support. Yes, we should be able to. We can't currently (without patch), but we should be able to. >> Therefore pip should be able to work (and does) >> without SSL support, without patching. The same should be true for R, >> and if this is not the case (and I'm not convinced it isn't), > > > It isn't : I've spent CPU *days* (two or three upgrades ago, can't exactly > rember when) to prove that neither GnuTLS nor Netscape's library allow to > compile an unmodified upstream R. You are, of course, welcome to check. I don't mean a different SSL library. I mean without *any*. >> then it >> should be for the same or similar reasons as pip. Again, this is >> purely from an abstract standpoint in how that software should, >> ideally, be designed. >> >> Second: The question is, should it be possible to build/install Sage >> (specifically the distribution) without an SSL library? There are two >> sub-problems that arise from this question: >> >> a) If we agree that that should at least be *possible* (I think it >> should be, even if not the default) to build Sage without SSL, then >> all of Sage's dependencies should be able to build without SSL. > > > I think that this is where we disagree : I tend to follow upstream's authors > : see below. I don't know. Do we really disagree? Or are you just agreeing with the upstream authors? >> This >> includes pip and R. If these packages satisfy the first issue then >> there is no problem; nothing to discuss. But your assertion is that R >> can't even be built without SSL support unless it is patched. > > > See above for the source of this assertion. The R Core Team (roughly the > author of upstream R) has decided that R *must* support https-enabled > downloads. It has taken steps to ensure that such support *is* available at > compile time. Therefore, we have to patch "our" R to lift this enforcement. This is wrong. R works without *any* kind of download capability whatsoever. This includes installing R packages (which I agree is important functionality). But that doesn't mean R has to be able to download them. It's not like starting up R is going to fail if it can't download anything (if that were the case you could never use it offline). And indeed it isn't the case. They're simply refusing to modularize their functionality for some reason. >> I would >> consider that an upstream deficiency that should be addressed >> aggressively just in principle if nothing else. > > > Again, you are welcome to try to change R Core Team's mind. I doubt you can > manage it : they seem to think that http mirrors are too insecure to be > trusted, and decided to enforce https *ability* in recent versions of R. > They probably won't be impressed by a couple of mathematicians with > pseudo-legal licensing issues. They are welcome to require HTTPS for CRAN and any other public mirrors. I would insist on it too. But that doesn't mean there's no good reason for plain HTTP networks (e.g. on a closed network). Or, as I feel like I've said repeatedly and am perhaps still unclear about somehow: It's not necessary for it to have *any* kind of download capability much less with/without SSL. > And, BTW, R is GPL-licensed... Stop brandishing that strawman. Thank you. I'm not sure what strawman you're referring to here but it's probably not one I've used. I don't personally care about licensing issues at all even if I probably should :) >> b) If we agree that it should not be possible to build Sage >> without SSL support (which I think is a poor decision, due to the >> first point) then there is no issue about needing to maintain a patch >> to remove SSL requirement from any package. > > > Indeed... > >> >> But there is the separate >> issue of having this added dependency that may be inconvenient for >> some people. > > > The same problem applies the the same people wanting to use upstream's R. It's not a problem if they're installing from binaries, which most are? Same for Sage. >> Third: There's a question of who the audience is. For the "casual" >> user who does not care about building things or compiling things or >> licenses or patches and just wants to run Sage, and be able to install >> packages from pip and CRAN (should the need even arise) this should >> all Just Work. From that point of view, they should be installing >> Sage from pre-built binaries, and those binaries should include the >> necessary SSL support, period, end of story. > > > Agreed. Fully. > > I'd like to add that the "average user" of R seems to be be much more prone > to install external packages than the "average user" of Sage : CRAN is a > very large part of the "R ecosystem". Agreed, but this still goes back to my first point which is the "average user" is also not everyone, and there are legitimate reasons to not care about the R library itself handling package downloads (regardless of how that affects Sage). > From a larger point of view : for an applied statistician, as judged by > their respective presences in scholarly publications, R is the dominant > platform for code supporting the papers, with Matlab being a (distant) > second ; Python has yet to make a marked dent ; SAS, which was dominant > about 20 years ago, seems to slide in oblivion. > > So, maintaining compatibility with the R-and--CRAN ecosystem is important. This is all well understood as to the need for this functionality in a normal install of R, but it's still irrelevant to the general point that it's not essential functionality merely for building and installing R in all cases. > Being a (very) casual user of Python, I am not aware of the nature and > importance of the Python ecosystem. But I'd be surprised if it pip-oriented > repositories didn't become as important to "the Python users community" as > CRAN is to the "R users community... They are and were before "pip" even came along. I used to manage an internal PyPI instance, for example, for proprietary Python packages. This is irrelevant though. >> This requirement is >> independent of how SSL support is achieved, what that means for >> developers or maintainers of binary packages, etc. > > > Indeed. You're welcome to try and find an unencubered SSL library that is > swallowed by upstream's R as acceptable for https support, and to propose > the patches necessary to its use to the R Core Team. I didn't succeed, but, > hey, I'm just a dentist that tries to *use* R... I don't care about that, and am not sure how I indicated that I am. I'm also not sure who accused you of being a "dentist" (also I know some very smart dentists...); I'm sorry if you feel on the defensive here because again I think we're almost entirely in agreement. I want what you want, for the same reasons, just from a different...perspective? I'm not sure what the disconnect is here except that you and the R developers seem to believe that it should be literally impossible to even build their software unless it has even the *capability* of connecting to CRAN, even though that's not necessary for any of its core functionality. This is very strange to me, even if we can all agree that CRAN is important to its ecosystem and typical user-base. Best, Erik -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
