[ceph-users] Ceph Mimic on Debian 9 Stretch
Hi Guys, When will the Ceph Mimic packages for Debian Stretch released? I could not find the packages even after changing the sources.list. Kind regards, Charles Alva Sent from Gmail Mobile ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On 04. juni 2018 06:41, Charles Alva wrote: Hi Guys, When will the Ceph Mimic packages for Debian Stretch released? I could not find the packages even after changing the sources.list. I am also eager to test mimic on my ceph debian-mimic only contains ceph-deploy atm. kind regards Ronny Aasen ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
There aren't any builds for Debian because the distro does not have compiler backports required for building Mimic On Mon, Jun 4, 2018 at 8:55 AM, Ronny Aasen wrote: > On 04. juni 2018 06:41, Charles Alva wrote: >> >> Hi Guys, >> >> When will the Ceph Mimic packages for Debian Stretch released? I could not >> find the packages even after changing the sources.list. >> >> > > I am also eager to test mimic on my ceph > > debian-mimic only contains ceph-deploy atm. > > kind regards > Ronny Aasen > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
[adding ceph-maintainers] On Mon, 4 Jun 2018, Charles Alva wrote: > Hi Guys, > > When will the Ceph Mimic packages for Debian Stretch released? I could not > find the packages even after changing the sources.list. The problem is that we're now using c++17, which requires a newer gcc than stretch or jessie provide, and Debian does not provide backports of the newer gcc packages. We currently can't build the latest Ceph for those releases. We raised this with the Debian package maintainers about a month ago[1][2] when the first release candidate was built and didn't get any response (beyond a "yes, we there are not gcc package backports"). Both ubuntu and fedora/rhel/centos (and I presume sles/opensuse) provide compiler backports we did not anticipate this being a problem. We'd love to build for stretch, but until there is a newer gcc for that distro it's not possible. We could build packages for 'testing', but I'm not sure if those will be usable on stretch. sage [1] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000603.html [2] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000611.html ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
My reaction when I read that there will be no Mimic soon on Stretch: https://pix.milkywan.fr/JDjOJWnx.png Anyway, thank you for the kind explanation, as well as for getting in touch with the Debian team about this issue On 06/04/2018 08:39 PM, Sage Weil wrote: > [adding ceph-maintainers] > > On Mon, 4 Jun 2018, Charles Alva wrote: >> Hi Guys, >> >> When will the Ceph Mimic packages for Debian Stretch released? I could not >> find the packages even after changing the sources.list. > > The problem is that we're now using c++17, which requires a newer gcc > than stretch or jessie provide, and Debian does not provide backports of > the newer gcc packages. We currently can't build the latest Ceph for > those releases. > > We raised this with the Debian package maintainers about a month ago[1][2] > when the first release candidate was built and didn't get any response > (beyond a "yes, we there are not gcc package backports"). Both ubuntu and > fedora/rhel/centos (and I presume sles/opensuse) provide compiler > backports we did not anticipate this being a problem. > > We'd love to build for stretch, but until there is a newer gcc for that > distro it's not possible. We could build packages for 'testing', but I'm > not sure if those will be usable on stretch. > > sage > > > [1] > http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000603.html > [2] > http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000611.html > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On 06/04/2018 07:39 PM, Sage Weil wrote: > [1] > http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000603.html > [2] > http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000611.html Just a heads up, seems the ceph-maintainers archives are not public. -Joao ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
Hi, 2018-06-04 20:39 GMT+02:00 Sage Weil : > We'd love to build for stretch, but until there is a newer gcc for that > distro it's not possible. We could build packages for 'testing', but I'm > not sure if those will be usable on stretch. > you can install gcc (and only gcc) from testing on Stretch to build Ceph for Stretch: echo 'deb http://ftp.de.debian.org/debian/ testing main' >> /etc/apt/sources.list echo 'APT::Default-Release "stable";' > /etc/apt/apt.conf.d/stable-as-default apt update apt install g++ -t testing That's how we are building our Debian packages. Still, Debian not providing gcc in backports is clearly something that needs to be fixed in Debian. Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
I see, thanks for the detailed information, Sage! Kind regards, Charles Alva Sent from Gmail Mobile On Tue, Jun 5, 2018 at 1:39 AM Sage Weil wrote: > [adding ceph-maintainers] > > On Mon, 4 Jun 2018, Charles Alva wrote: > > Hi Guys, > > > > When will the Ceph Mimic packages for Debian Stretch released? I could > not > > find the packages even after changing the sources.list. > > The problem is that we're now using c++17, which requires a newer gcc > than stretch or jessie provide, and Debian does not provide backports of > the newer gcc packages. We currently can't build the latest Ceph for > those releases. > > We raised this with the Debian package maintainers about a month ago[1][2] > when the first release candidate was built and didn't get any response > (beyond a "yes, we there are not gcc package backports"). Both ubuntu and > fedora/rhel/centos (and I presume sles/opensuse) provide compiler > backports we did not anticipate this being a problem. > > We'd love to build for stretch, but until there is a newer gcc for that > distro it's not possible. We could build packages for 'testing', but I'm > not sure if those will be usable on stretch. > > sage > > > [1] > http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000603.html > [2] > http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000611.html > > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On Tue, Jun 5, 2018 at 6:13 AM, Paul Emmerich wrote: > Hi, > > 2018-06-04 20:39 GMT+02:00 Sage Weil : >> >> We'd love to build for stretch, but until there is a newer gcc for that >> distro it's not possible. We could build packages for 'testing', but I'm >> not sure if those will be usable on stretch. > > > you can install gcc (and only gcc) from testing on Stretch to build Ceph for > Stretch: > > echo 'deb http://ftp.de.debian.org/debian/ testing main' >> > /etc/apt/sources.list > echo 'APT::Default-Release "stable";' > > /etc/apt/apt.conf.d/stable-as-default > apt update > apt install g++ -t testing > > That's how we are building our Debian packages. thanks for sharing this, Paul ! does the built binary require any runtime dependency offered the testing repo? if the answer is no, i think we should offer the pre-built package for debian stable then. > > Still, Debian not providing gcc in backports is clearly something that needs > to be > fixed in Debian. > > > Paul > > -- > Paul Emmerich > > Looking for help with your Ceph cluster? Contact us at https://croit.io > > croit GmbH > Freseniusstr. 31h > 81247 München > www.croit.io > Tel: +49 89 1896585 90 > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Regards Kefu Chai ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On 05/06/18 05:58, kefu chai wrote: > On Tue, Jun 5, 2018 at 6:13 AM, Paul Emmerich wrote: >> Hi, >> >> 2018-06-04 20:39 GMT+02:00 Sage Weil : >>> We'd love to build for stretch, but until there is a newer gcc for that >>> distro it's not possible. We could build packages for 'testing', but I'm >>> not sure if those will be usable on stretch. >> >> you can install gcc (and only gcc) from testing on Stretch to build Ceph for >> Stretch: >> >> echo 'deb http://ftp.de.debian.org/debian/ testing main' >> >> /etc/apt/sources.list >> echo 'APT::Default-Release "stable";' > >> /etc/apt/apt.conf.d/stable-as-default >> apt update >> apt install g++ -t testing >> >> That's how we are building our Debian packages. > thanks for sharing this, Paul ! does the built binary require any > runtime dependency offered the testing repo? if the answer is no, i > think we should offer the pre-built package for debian stable then. > > We have some debian developers in our team - let me see if we can make something happen with backporting gcc. It would be a shame for Ceph to be more cumbersome to install on Debian. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
2018-06-05 6:58 GMT+02:00 kefu chai : > > thanks for sharing this, Paul ! does the built binary require any > runtime dependency offered the testing repo? if the answer is no, i > think we should offer the pre-built package for debian stable then. > It will by default produce binaries linking against libc >= 2.25 and Stretch comes with 2.24. But that can be fixed at build time. (There's nothing in Ceph that technically requires that version) We just added a newer libc manually to our image at the moment, we'll do proper building/packaging later for our Mimic release. -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On 04.06.2018 21:08, Joao Eduardo Luis wrote: On 06/04/2018 07:39 PM, Sage Weil wrote: [1] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000603.html [2] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/2018-April/000611.html Just a heads up, seems the ceph-maintainers archives are not public. -Joao The debian-gcc list is public: https://lists.debian.org/debian-gcc/2018/04/msg00137.html Ronny Aasen ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
Hi, If anyone wants to play around with Ceph on Debian: I just made our mirror for our dev/test image builds public: wget -q -O- 'https://static.croit.io/keys/release.asc' | apt-key add - echo 'deb https://static.croit.io/debian-mimic/ stretch main' >> /etc/apt/sources.list apt update apt install ceph Caution: this is not yet qualified for production use by us, use at your own risk. The mirror also contains a newer version of libc because we are currently building against libc 2.27. Also, the URL will probably change to mirror.croit.io in the future, I've just put it on our static web server if anyone wants to play around with it. Paul 2018-06-05 17:35 GMT+02:00 Ronny Aasen : > On 04.06.2018 21:08, Joao Eduardo Luis wrote: > >> On 06/04/2018 07:39 PM, Sage Weil wrote: >> >>> [1] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/ >>> 2018-April/000603.html >>> [2] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/ >>> 2018-April/000611.html >>> >> Just a heads up, seems the ceph-maintainers archives are not public. >> >>-Joao >> >> The debian-gcc list is public: > https://lists.debian.org/debian-gcc/2018/04/msg00137.html > > Ronny Aasen > > > > > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
Hello Paul, Am 5. Juni 2018 22:17:15 MESZ schrieb Paul Emmerich : >Hi, > >If anyone wants to play around with Ceph on Debian: I just made our >mirror >for our >dev/test image builds public: > >wget -q -O- 'https://static.croit.io/keys/release.asc' | apt-key add - >echo 'deb https://static.croit.io/debian-mimic/ stretch main' >> >/etc/apt/sources.list >apt update >apt install ceph > That friendly Thank you very much. I Bet this is usefull for Debian User. - Mehmet >Caution: this is not yet qualified for production use by us, use at >your >own risk. The mirror >also contains a newer version of libc because we are currently building >against libc 2.27. > >Also, the URL will probably change to mirror.croit.io in the future, >I've >just put it on our >static web server if anyone wants to play around with it. > > >Paul > >2018-06-05 17:35 GMT+02:00 Ronny Aasen : > >> On 04.06.2018 21:08, Joao Eduardo Luis wrote: >> >>> On 06/04/2018 07:39 PM, Sage Weil wrote: >>> [1] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/ 2018-April/000603.html [2] http://lists.ceph.com/private.cgi/ceph-maintainers-ceph.com/ 2018-April/000611.html >>> Just a heads up, seems the ceph-maintainers archives are not public. >>> >>>-Joao >>> >>> The debian-gcc list is public: >> https://lists.debian.org/debian-gcc/2018/04/msg00137.html >> >> Ronny Aasen >> >> >> >> >> >> ___ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On Mon, Jun 04, 2018 at 06:39:08PM +, Sage Weil wrote: > [adding ceph-maintainers] [and ceph-devel] > > On Mon, 4 Jun 2018, Charles Alva wrote: > > Hi Guys, > > > > When will the Ceph Mimic packages for Debian Stretch released? I could not > > find the packages even after changing the sources.list. > > The problem is that we're now using c++17, which requires a newer gcc > than stretch or jessie provide, and Debian does not provide backports of > the newer gcc packages. We currently can't build the latest Ceph for > those releases. IMHO this is backwards. if you want to support distro X you should take care to not need toolchain features that are not included in distro X. Debian only provides one toolchain backport, and that is for Firefox, which has a stable update exception because it is such an important component for desktop systems and cannot be supported otherwise[1]. This package is also not intended as general purpose toolchain, but built solely for enabling a Firefox backport. > We raised this with the Debian package maintainers about a month ago[1][2] > when the first release candidate was built and didn't get any response > (beyond a "yes, we there are not gcc package backports"). this is not how Debian works, as you most likely know ;) > Both ubuntu and > fedora/rhel/centos (and I presume sles/opensuse) provide compiler > backports we did not anticipate this being a problem. this is also not very accurate. it is true that Canonical provides a toolchain PPA[2] which the Ceph build for Xenial seems to use, but there is (AFAICT) no official guarantee for the level of support, security or otherwise[3]. in fact, the PPA description states that it contains "Toolchain test builds", which seem to mean pretty automatic backports of whatever is in the current Ubuntu dev release, with a very short delay between upload to Cosmic and the PPA for Xenial. e.g., for the currently contained gcc-7 packages, there was less than a week between hitting Cosmic and Xenial. Cosmic at the current point in the release cycle is already not really exposed to public testing scrutiny in general, and for sure not to the level something like the core toolchain would require. effectively this means the current Xenial builds are about as safe and production-ready as doing your own gcc backports for Stretch - i.e., not very. > We'd love to build for stretch, but until there is a newer gcc for that > distro it's not possible. We could build packages for 'testing', but I'm > not sure if those will be usable on stretch. saying you'd love to build for a distro, while effectively making sure a build according to that distro's release policies is impossible without major effort by someone else does strike me as a bit of a hollow statement. in the end this is a further nail in the coffin of upstream support for the Debian(-based) distros, with only the latest (1.5 months old!) Ubuntu LTS being properly supported. I hope we find some way to support Mimic+ for Stretch without requiring a backport of gcc-7+, although it unfortunately seems unlikely at this point. 1: https://tracker.debian.org/pkg/gcc-mozilla 2: https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test 3: https://wiki.ubuntu.com/ToolChain#Toolchain_Updates ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On Wed, 13 Jun 2018, Fabian Grünbichler said: > I hope we find some way to support Mimic+ for Stretch without requiring > a backport of gcc-7+, although it unfortunately seems unlikely at this > point. Me too. I picked ceph luminous on debian stretch because I thought it would be maintained going forwards, and we're a debian shop. I appreciate Mimic is a non-LTS release, I hope issues of debian support are resolved by the time of the next LTS. Sean Purdy ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
Hi Fabian, On Wed, 13 Jun 2018, Fabian Grünbichler wrote: > On Mon, Jun 04, 2018 at 06:39:08PM +, Sage Weil wrote: > > [adding ceph-maintainers] > > [and ceph-devel] > > > > > On Mon, 4 Jun 2018, Charles Alva wrote: > > > Hi Guys, > > > > > > When will the Ceph Mimic packages for Debian Stretch released? I could not > > > find the packages even after changing the sources.list. > > > > The problem is that we're now using c++17, which requires a newer gcc > > than stretch or jessie provide, and Debian does not provide backports of > > the newer gcc packages. We currently can't build the latest Ceph for > > those releases. > > IMHO this is backwards. if you want to support distro X you should take > care to not need toolchain features that are not included in distro X. Well, I thought we did. When we were making the C++17 decision we verified that we could do builds on Ubuntu and CentOS using a newer compiler toolchain. My assumption was that since both of these distros had backports that pretty much everyone did. Clearly I was wrong. > [...] > effectively this means the current Xenial builds are about as safe and > production-ready as doing your own gcc backports for Stretch - i.e., not > very. I missed this nuance as well. > > We'd love to build for stretch, but until there is a newer gcc for that > > distro it's not possible. We could build packages for 'testing', but I'm > > not sure if those will be usable on stretch. > > saying you'd love to build for a distro, while effectively making sure a > build according to that distro's release policies is impossible without > major effort by someone else does strike me as a bit of a hollow > statement. in the end this is a further nail in the coffin of upstream > support for the Debian(-based) distros, with only the latest (1.5 months > old!) Ubuntu LTS being properly supported. I think we need to be clear about the use of the term "support" here. I was careful to say we'd like to *build* for Debian, but I'm not sure what organizations out there are offering formal *support* for any of the ceph.com packages (in the sense of providing technical support for bug escalations or any guarantees around stability etc). This incident is perhaps an indication that those organizations should become more involved in the upstream development and decision-making process. > I hope we find some way to support Mimic+ for Stretch without requiring > a backport of gcc-7+, although it unfortunately seems unlikely at this > point. It is not practical to reverse the dependency on C++17. A lot of C++17 code has already made it into the tree, and the use of projects that require it (notably Seastar) is a *big* part of our plans going forward to improve performance on flash devices. Again, I would still like to provide these builds. If the requirement to do so is a high level of assurance that the backported compiler used is solid and the resulting builds have been tested, then that will take some additional effort--at a minimum, adding stretch to the set of OSes that we test in the sepia lab. This is probably not that much work, TBH, but does require someone who cares to become engaged in the testing effort. There are daily and weekly standups you can join, and of course discussions take place on IRC (#sepia) and over email (ceph-devel and ceph-maintainers). The immediate practical issue/question, IIRC, is simply for someone to verify whether a build using the debian testing version of gcc will produce a package that can be installed without additional dependencies on stretch. Or maybe this question was already answered and a backport is definitely needed. Regardless, it seems to me like the compiler version in testing would be an appropriate choice for a gcc backport to stretch. One final note: we didn't move to C++17 for no reason: we did so because it is difficult to move the Ceph project forward without taking advantage of the new language features. It took us a *long* time to get to C++11 because of paranoia around distro support, but we finally realized that we could move more quickly if we rely on the compiler backports available in most distros (tho clearly not as many as we thought!). I wish we had moved much sooner with C++11, and I still think this was the right decision with respect to C++17. I'm sorry that this has caused problems with Debian, but I think with your (or someone's) help we can mitigate the damage with testing. Thanks! sage___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On 06/13/2018 02:01 PM, Sean Purdy wrote: > Me too. I picked ceph luminous on debian stretch because I thought > it would be maintained going forwards, and we're a debian shop. I > appreciate Mimic is a non-LTS release, I hope issues of debian > support are resolved by the time of the next LTS. There won't be a "next LTS" in the fashion that previous Ceph releases were published. See http://docs.ceph.com/docs/master/releases/schedule/#stable-releases-x-2-z for how this is handled now. Lenz -- SUSE Linux GmbH - Maxfeldstr. 5 - 90409 Nuernberg (Germany) GF:Felix Imendörffer,Jane Smithard,Graham Norton,HRB 21284 (AG Nürnberg) signature.asc Description: OpenPGP digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On Wed, Jun 13, 2018 at 12:36:50PM +, Sage Weil wrote: > Hi Fabian, thanks for your quick, and sorry for my delayed response (only having 1.5 usable arms atm). > > On Wed, 13 Jun 2018, Fabian Grünbichler wrote: > > On Mon, Jun 04, 2018 at 06:39:08PM +, Sage Weil wrote: > > > [adding ceph-maintainers] > > > > [and ceph-devel] > > > On Mon, 4 Jun 2018, Charles Alva wrote: > > > > Hi Guys, > > > > > > > > When will the Ceph Mimic packages for Debian Stretch released? I could > > > > not > > > > find the packages even after changing the sources.list. > > > > > > The problem is that we're now using c++17, which requires a newer gcc > > > than stretch or jessie provide, and Debian does not provide backports of > > > the newer gcc packages. We currently can't build the latest Ceph for > > > those releases. > > > > IMHO this is backwards. if you want to support distro X you should take > > care to not need toolchain features that are not included in distro X. > > Well, I thought we did. When we were making the C++17 decision we > verified that we could do builds on Ubuntu and CentOS using a newer > compiler toolchain. My assumption was that since both of these distros > had backports that pretty much everyone did. Clearly I was wrong. just to make this explicit: I did not mean to imply any malicious intent on your part. I am aware this whole issue is more a question of priorities and capacities than anything else (on all sides, not just yours). I do appreciate all the work you and the rest of the main Ceph developers and the community as a whole has done and continues to do, and we see ourselves very much as part of this community! > > > [...] > > effectively this means the current Xenial builds are about as safe and > > production-ready as doing your own gcc backports for Stretch - i.e., not > > very. > > I missed this nuance as well. just as another point, install-deps.sh script from ceph.git will upgrade(!) the following packages on Xenial: - libatomic1 - libcc1-0 - libcilkrts5 - libgcc1 - libgomp1 - libitm1 - liblsan0 - libquadmath0 - libstdc++6 - libtsan0 - libubsan0 some of them will even be upgraded to versions built from gcc-8. so not only is this backport not very production-grade from a testing and support POV, it is also not self-contained (in contrast to the old Wheezy Mozilla GCC backport, or RedHat's DTS, at least from my understanding?). IMHO both issues should be mentioned in the appropriate places (regular docs for the former, dev docs for the latter?). > > > We'd love to build for stretch, but until there is a newer gcc for that > > > distro it's not possible. We could build packages for 'testing', but I'm > > > not sure if those will be usable on stretch. > > > > saying you'd love to build for a distro, while effectively making sure a > > build according to that distro's release policies is impossible without > > major effort by someone else does strike me as a bit of a hollow > > statement. in the end this is a further nail in the coffin of upstream > > support for the Debian(-based) distros, with only the latest (1.5 months > > old!) Ubuntu LTS being properly supported. > > I think we need to be clear about the use of the term "support" here. I > was careful to say we'd like to *build* for Debian, but I'm not sure what > organizations out there are offering formal *support* for any of the > ceph.com packages (in the sense of providing technical support for bug > escalations or any guarantees around stability etc). This incident is > perhaps an indication that those organizations should become more involved > in the upstream development and decision-making process. I am aware that there is no formal support (in the sense of commercial agreements, etc.pp.) for the packages on download.ceph.com, and the fact that most of the testing the packages for Debian get are just a side-effect of you testing Ubuntu Xenial and now Bionic. we are already rolling our own .deb packages for Proxmox VE (based on the upstream debian/ directory) because we have been bitten in the past by issues not caught in the upstream CI infrastructure. we try to stay involved in the community, e.g. by opening or forwarding bugs after initial triaging, and contributing fixes when possible. we do spread the "gospel of Ceph" and promote it quite heaviliy, we do develop integration and management layers that probably allow end users to setup and use Ceph that would otherwise not dare to because of the complexity involved. but in the end, we (as in Ceph the upstream project, and Proxmox as downstream distro) both face a similar dilemma - given limited developer time, we need to make a choice on how to use it. we will continue discussing internally, and try to ramp up our involvement in one way or another - especially since we expect Ceph to become an ever more important component of your virtualization stack. > > I hope we find some way to support Mimic+ for Stretch without requiring > > a backp
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On Mon, 18 Jun 2018, Fabian Grünbichler wrote: > it's of course within your purview as upstream project (lead) to define > certain platforms/architectures/distros as fully supported, and others > as best-effort/community-driven/... . there was no clear public > communication (AFAICT, only the one thread on ceph-maintainers, which is > rather low visibility) that Debian moves from somewhere in the middle[2] > to the latter category with Mimic, and has now (at least for the time > being) effectively joined FreeBSD (which has at least one community > member pouring in enormous amounts of work) and the various community > Linux distros like Arch, Gentoo, ... (where I frankly have no idea about > the status quo). there is also no mention in the docs or the release > notes about the lack of Debian packages (and the state of Xenial > packages) for Mimic. all of which gives off more of an "unintended > consequence" vibe, rather than "conscious decision to drop Debian". This is a fair assessment, and it's good to hear that there is some path forward. It looks like the Buster release date is Feb '19 (give or take), which corresponds to Nautilus, so it should be possible for Debian users to skip mimic and upgrade directly from luminous to nautilus as long as we build some buster packages for luminous as well right around the end of its lifetime (and/or mimic point releases once buster gets closer to stable). Is this reasonable? https://github.com/ceph/ceph/pull/22602 sage___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
Hi Fabian, Hope your arm is doing well :) unless such a backport is created and tested fairly well (and we will spend some more time investigating this internally despite the caveats above), our plan B will probably involve: - building Luminous for Buster to ease the upgrade from Stretch+Luminous (upgrading both base distro release and Ceph major version in one go did not work out in the past) - keeping our Stretch-based release on Luminous even once Luminous is EoL upstream - strongly recommending to those of our users that rely on Ceph to upgrade to our (future/next) Buster-based release (which will likely get Mimic or Nautilus as default Ceph version, depending on whether the Ceph release schedule holds or not) - hope this whole story does not repeat itself too often because of the inherent misalignment between Ceph and Debian release cycles especially the second and third point will irritate some of our users, but sometimes life only hands you lemons. We're responsible of about 6 small clusters of Proxmox + Ceph; I think this is the path to take. Use the time to "extend" Luminous support, maybe you can do this together with others, maybe even with some support from Ceph upstream. I think it should be less work than the gcc backport just for a few months. Just skip Mimic like you did with non LTS releases in the past. It's also less work for the Proxmox admins, as we'll be able to skip a Ceph upgrade easily. Cheers Eneko -- Zuzendari Teknikoa / Director Técnico Binovo IT Human Project, S.L. Telf. 943569206 Astigarraga bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa) www.binovo.es ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
FWIW: we'll be shipping Mimic on Debian with an upgraded libc6. This is of course somewhat of a zombie Debian: our build system initially fell apart quite a few times. But for the build system it's not a problem -- it will be thrown away after the build finishes anyway. These problems will occur when installation other packages that now depend on the wrong versions/pull in half of buster. Our OS simply comes with an upgraded libc6 - also a potential problem. However, all of our deployments are based on the concept of immutable infrastructure. We do not update images with apt while they are running*. We ship one image as squashfs and upgrading to a newer version means rebooting your server with our new image via an automated rolling reboot process. We will keep our mirror for Debian updated and public, but we will not officially recommend it or "real" Debian installations that are not based on our live image. We've looked a little bit into building with an older libc, but it's not worth the effort for us at the moment. We expect to ship this with Mimic 13.2.1 assuming that it will be stable enough. 13.2.0 with Debian is working fine so far on our test cluster. Paul (* except temporarily to ensure proper order during Ceph upgrades and security fixes of course) 2018-06-19 10:02 GMT+02:00 Eneko Lacunza : > Hi Fabian, > > Hope your arm is doing well :) > > unless such a backport is created and tested fairly well (and we will >> spend some more time investigating this internally despite the caveats >> above), our plan B will probably involve: >> - building Luminous for Buster to ease the upgrade from Stretch+Luminous >> (upgrading both base distro release and Ceph major version in one go >> did not work out in the past) >> - keeping our Stretch-based release on Luminous even once Luminous is EoL >> upstream >> - strongly recommending to those of our users that rely on Ceph to >> upgrade to our (future/next) Buster-based release (which will likely >> get Mimic or Nautilus as default Ceph version, depending on whether >> the Ceph release schedule holds or not) >> - hope this whole story does not repeat itself too often because of the >> inherent misalignment between Ceph and Debian release cycles >> >> especially the second and third point will irritate some of our users, >> but sometimes life only hands you lemons. >> > We're responsible of about 6 small clusters of Proxmox + Ceph; I think > this is the path to take. > > Use the time to "extend" Luminous support, maybe you can do this together > with others, maybe even with some support from Ceph upstream. I think it > should be less work than the gcc backport just for a few months. > > Just skip Mimic like you did with non LTS releases in the past. It's also > less work for the Proxmox admins, as we'll be able to skip a Ceph upgrade > easily. > > Cheers > Eneko > > -- > Zuzendari Teknikoa / Director Técnico > Binovo IT Human Project, S.L. > Telf. 943569206 > Astigarraga bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa) > www.binovo.es > > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On Mon, Jun 18, 2018 at 07:15:49PM +, Sage Weil wrote: > On Mon, 18 Jun 2018, Fabian Grünbichler wrote: > > it's of course within your purview as upstream project (lead) to define > > certain platforms/architectures/distros as fully supported, and others > > as best-effort/community-driven/... . there was no clear public > > communication (AFAICT, only the one thread on ceph-maintainers, which is > > rather low visibility) that Debian moves from somewhere in the middle[2] > > to the latter category with Mimic, and has now (at least for the time > > being) effectively joined FreeBSD (which has at least one community > > member pouring in enormous amounts of work) and the various community > > Linux distros like Arch, Gentoo, ... (where I frankly have no idea about > > the status quo). there is also no mention in the docs or the release > > notes about the lack of Debian packages (and the state of Xenial > > packages) for Mimic. all of which gives off more of an "unintended > > consequence" vibe, rather than "conscious decision to drop Debian". > > This is a fair assessment, and it's good to hear that there is some path > forward. > > It looks like the Buster release date is Feb '19 (give or take), which > corresponds to Nautilus, so it should be possible for Debian users to skip > mimic and upgrade directly from luminous to nautilus as long as we build > some buster packages for luminous as well right around the end of its > lifetime (and/or mimic point releases once buster gets closer to stable). > > Is this reasonable? yes. like I already indicated, this is our "Plan B" in case Mimic on Stretch is not feasible. we'll likely skip Mimic entirely (except for some internal testing to catch and fix issue before Nautlius) in that case, and jump straight to Nautilus with Buster and keep Luminous on "life support" for Stretch and upgrading. > https://github.com/ceph/ceph/pull/22602 LGTM. still would like to see some note about the Xenial toolchain stability issues, but that is more for the sake of others (I am not an Ubuntu user). ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
