Re: libmlx4 and libmlx5 git trees? Who is handling those?
On 09/28/2015 06:08 PM, Woodruff, Robert J wrote: > Jason wrote, >> IMHO - we should be talking about getting to the point where we can >> deliver the kernel rc and uapi rc together to someplace like UNH >> and vendor internal labs and >expect them to test that pair. >> Regularly, ideally on the kernel release time line. > > I am all for doing more testing earlier, but I am not sure that > people (like UNH and the vendor validation teams will sign up to do > it. This would essentially double their testing effort (and costs), > first testing the user-space packages with the upstream kernel that > is under development, but also still having to test what comes out of > the distro or OFED. This is because it is what is in OFED and/or the > distro that their customers will actually run. Very few customers > actually run the kernel.org kernel, they run what is in the distro. > And BTW, UNH gets paid to do testing, so someone would have to cough > up the $$ to get them to add this additional testing. I disagree Bob. > So it all really comes down to resources and cost and being able to > justify that doing more testing of upstream code is worth it. Hopefully this isn't too blunt, but the EWG has the open source paradigm ass-backwards. Right now, OS distros (us and SuSE) do their own releases with their own QE testing. Upstream we have some people from different companies participating and contributing code, but with admittedly thin testing resources devoted to upcoming upstream kernels. We catch some things, but I've also see something slip through the cracks on mlx4 hardware in both of the last two upstream kernel releases. The EWG does its own integration and testing work for OFED. Finally, the UNH-IOL does an entirely different type of testing, but no fixes. If an end users wants to know if bug X is fixed, then it matters who found the bug because it is likely fixed in that group's product, but not necessarily in the other group's products. If a user needs both bug X and bug Y fixed, and they were fixed by different groups, then things are screwed. That's not how open source is supposed to work. That is not the best use of all of our time. First and foremost, we should all be concentrating on upstream, because that is the work that pays off for all of the people distributing this code. You say end users don't use upstream? Of course they do. No matter whether they use OFED or Red Hat or SuSE or something else, all of them start with upstream. Put the effort where you get the most payback. -- Doug Ledford GPG KeyID: 0E572FDD signature.asc Description: OpenPGP digital signature
RE: libmlx4 and libmlx5 git trees? Who is handling those?
Jason wrote, >I was commenting specifically on the idea that we'd ever release a libverbs >that forced a kernel upgrade. I hope we all agree that is not acceptable. This means that user-space and kernel space packages can be released on separate schedules, so there is no need to tie the two together. That is good. >IMHO - we should be talking about getting to the point where we can deliver >the kernel rc and uapi rc together to someplace like UNH and vendor internal >labs and >expect them to test that pair. Regularly, ideally on the kernel >release time line. I am all for doing more testing earlier, but I am not sure that people (like UNH and the vendor validation teams will sign up to do it. This would essentially double their testing effort (and costs), first testing the user-space packages with the upstream kernel that is under development, but also still having to test what comes out of the distro or OFED. This is because it is what is in OFED and/or the distro that their customers will actually run. Very few customers actually run the kernel.org kernel, they run what is in the distro. And BTW, UNH gets paid to do testing, so someone would have to cough up the $$ to get them to add this additional testing. So it all really comes down to resources and cost and being able to justify that doing more testing of upstream code is worth it. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, Sep 28, 2015 at 04:22:50PM -0400, Doug Ledford wrote: > >> just a bug fix. Further, that new stuff might even require new > >> kernel code, so it could not just be replaced as a new user-space > >> package by a distro w/o updating the kernel. > > > > We are not going to make a change like that, that violates the spirit > > of how the uabi side is supposed to work. > None of that violates the spirit of the libibverbs development, but > it still doesn't necessarily match up with distro needs. I was commenting specifically on the idea that we'd ever release a libverbs that forced a kernel upgrade. I hope we all agree that is not acceptable. > Anyway, I had intended, and I've started on, making a change in how > libibverbs is done anyway. The idea that a new release is just a script > that throws a version on and we go is naive. I *will* be doing > pre-release rc tarballs and there will be testing and there will be a > release process that helps to make sure we don't see things like what we > saw with librdmacm (sorry to pick on you Sean): 1.0.17/1.0.17.1, > 1.0.18/1.0.18.1, and 1.0.19/1.0.19.1. I don't know about other people, but I've built the mess enough times to know I don't want to do it. Especially if it starts to be "git HEAD on all the bits doesn't work together" Making rcs is fine, but think of the audience? Who will build them, who will test them, and is it *as easy as possible* to do that? Make it hard and people will ignore git, ignore rc release, ignore release tar balls and just wait for OFED to put something out. IMHO - we should be talking about getting to the point where we can deliver the kernel rc and uapi rc together to someplace like UNH and vendor internal labs and expect them to test that pair. Regularly, ideally on the kernel release time line. To that end, in my view, exactly two source releases is the ideal. Regarding bug fixes: I've seen other communities emulate the kernel scheme of tagging patches in git for backport, and distros use that guidance to make sure bug fixes filter back into their releases. Just like the kernel. Much easier to do that in one place than many. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: libmlx4 and libmlx5 git trees? Who is handling those?
> >The issue is that basically no one tests the releases of the packages. > > We assume that the maintainer of the package tests it before they release > it :) No one has access to that many pieces of hardware and system configurations. The "community" can't dump the responsibility for testing onto a single person. OFA has always had the problem of testing long after the fact -- months, if not years, after changes are merged upstream (kernel or user space). -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On 09/28/2015 04:40 PM, Woodruff, Robert J wrote: > Sean wrote, >> The issue is that basically no one tests the releases of the packages. > > We assume that the maintainer of the package tests it before they release it > :) Every package maintainer I know of does. But there isn't a single package maintainer that has a 100% test setup that covers all of the various ways in which their code is used. Maintainer testing should be considered good, but *never* sufficient for release testing. There needs to be more done. > The OFED testing then tests all the currently released packages This is backwards IMO. Testing after the release results in exactly what I mentioned in my email. The right time to test is during the release process, not after it is already complete. -- Doug Ledford GPG KeyID: 0E572FDD signature.asc Description: OpenPGP digital signature
RE: libmlx4 and libmlx5 git trees? Who is handling those?
Sean wrote, >The issue is that basically no one tests the releases of the packages. We assume that the maintainer of the package tests it before they release it :) The OFED testing then tests all the currently released packages as a bundle with a released kernel version backported to a distro kernel. Sometimes this does find bugs in a package that the maintainer testing did not find that requires the maintainer to release a newer package if they feel that bug is critical to fix now and sometimes they just do the fix in the next package if it is not critical. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: libmlx4 and libmlx5 git trees? Who is handling those?
> Anyway, I had intended, and I've started on, making a change in how > libibverbs is done anyway. The idea that a new release is just a script > that throws a version on and we go is naive. I *will* be doing > pre-release rc tarballs and there will be testing and there will be a > release process that helps to make sure we don't see things like what we > saw with librdmacm (sorry to pick on you Sean): 1.0.17/1.0.17.1, > 1.0.18/1.0.18.1, and 1.0.19/1.0.19.1. I don’t feel picked on. :) The issue is that basically no one tests the releases of the packages. They only test the OFED bundle, and OFED doesn't want to pick up anything that's not already a release. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On 09/28/2015 03:00 PM, Jason Gunthorpe wrote: > On Mon, Sep 28, 2015 at 06:18:52PM +, Woodruff, Robert J wrote: >> On Mon, Sep 28, 2015 at 10:42:08AM -0700, Jason Gunthorpe wrote: >>> We are well past the point of doing experimental breaking stuff in the core >>> uapi libraries. If it is in the master git it should be shippable by a >>> distro, and it is so easy >to slap a version number on the HEAD if a >>> distro/ofed/etc wants one. >> >> This may be true, but since it is the master, there could be all >> kinds of new stuff in it and not just the simple bug fix a H/W >> vendor might want to put into his/her user-space H/W specific >> library. Consider the situation when some H/W vendor just wants to >> provide a simple fix to their library. In the current model, they >> simply fix it and release a new H/W specific library that a customer >> or distro can pick up and ship. > > It makes very little difference, once a distro has gone stable they > will review and inspect all patches - checking a new tar release for a > sub component or backporting a git patch for the same subcomponent are > very similar end results. > > I would guess all-together is actually less work for the distro side > because the whole candidate patch stream for backporting is right > there, easy to see, not sprinkled across all manner of git trees. > > And again, if the master is unusable, then we have not done our job > right. The master is *NOT* for stuff that doesn't work. > >> just a bug fix. Further, that new stuff might even require new >> kernel code, so it could not just be replaced as a new user-space >> package by a distro w/o updating the kernel. > > We are not going to make a change like that, that violates the spirit > of how the uabi side is supposed to work. Actually, that's not true. Master is for things that have reached a state where we think they work and are ready to commit to them. Between releases, you can add a number of new features. The on demand paging feature went into master for libibverbs a while ago. The checksum support patches didn't go in until after those. The on demand paging feature is pretty invasive, while the checksum patch set is very minimally invasive. As a distro, I can take the checksum patches, but the on demand patches need bake time. None of that violates the spirit of the libibverbs development, but it still doesn't necessarily match up with distro needs. Anyway, I had intended, and I've started on, making a change in how libibverbs is done anyway. The idea that a new release is just a script that throws a version on and we go is naive. I *will* be doing pre-release rc tarballs and there will be testing and there will be a release process that helps to make sure we don't see things like what we saw with librdmacm (sorry to pick on you Sean): 1.0.17/1.0.17.1, 1.0.18/1.0.18.1, and 1.0.19/1.0.19.1. As for bundling the packages together, I'm fine either way. I don't think there is a huge benefit to one method over the other as long as all the maintainers can coordinate work that needs to be coordinated. It might be a little clunky to write all the autotools checks to make sure you don't compile a version against another incompatible version, but the tradeoff is that you can do separate updates easily to fix bugs in the dependent packages. -- Doug Ledford GPG KeyID: 0E572FDD signature.asc Description: OpenPGP digital signature
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On 9/28/2015 12:39 PM, Jason Gunthorpe wrote: On Mon, Sep 28, 2015 at 05:28:20PM +, Woodruff, Robert J wrote: On Mon, 28 Sep 2015, Christoph Lameter wrote: Right. Its really nasty when you are trying to add features that require libibverbs and libmlx? changes. Plus it may depend on kernel changes. On the other hand, combining everything into one package limits the ability of the maintainer of the individual components to release packages with simple bug fixes or enhancements that are component specific and don't require kernel core or libibverbs changes. Thus, they would have to wait till a new combined package gets released and makes the maintainer of that package the bottleneck for getting new code out. That is certainly the minority of work these days, by my observation. Nearly everything is being driven by kernel side changes now and requires cross-library work. Or it is maintenance activity, which is so hard now (I've done a few patches over the years) it isn't worth doing unless it is really important. Jason I've released many libcxb4 releases that fixed bugs, and even added new device support w/o any libibverbs changes. So I'm not sure I see the benefit of munging all the RDMA libraries into some uber-release... Steve. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, 28 Sep 2015, Jason Gunthorpe wrote: > I would guess all-together is actually less work for the distro side > because the whole candidate patch stream for backporting is right > there, easy to see, not sprinkled across all manner of git trees. This is true not only for the distro but also for those needing to make changes because f.e. some things are broken. Its much easier to send out a single patch vs the "big library" instead of 3 separate ones against libibverbs, libmlx5, libmlx4. Keeping track of that is a nightmare. Ideally there should be a lib directory in the kernel where the stuff lives so that a single patch can make the necessary changes. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, Sep 28, 2015 at 06:18:52PM +, Woodruff, Robert J wrote: > On Mon, Sep 28, 2015 at 10:42:08AM -0700, Jason Gunthorpe wrote: > >We are well past the point of doing experimental breaking stuff in the core > >uapi libraries. If it is in the master git it should be shippable by a > >distro, and it is so easy >to slap a version number on the HEAD if a > >distro/ofed/etc wants one. > > This may be true, but since it is the master, there could be all > kinds of new stuff in it and not just the simple bug fix a H/W > vendor might want to put into his/her user-space H/W specific > library. Consider the situation when some H/W vendor just wants to > provide a simple fix to their library. In the current model, they > simply fix it and release a new H/W specific library that a customer > or distro can pick up and ship. It makes very little difference, once a distro has gone stable they will review and inspect all patches - checking a new tar release for a sub component or backporting a git patch for the same subcomponent are very similar end results. I would guess all-together is actually less work for the distro side because the whole candidate patch stream for backporting is right there, easy to see, not sprinkled across all manner of git trees. And again, if the master is unusable, then we have not done our job right. The master is *NOT* for stuff that doesn't work. > just a bug fix. Further, that new stuff might even require new > kernel code, so it could not just be replaced as a new user-space > package by a distro w/o updating the kernel. We are not going to make a change like that, that violates the spirit of how the uabi side is supposed to work. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, Sep 28, 2015 at 10:42:08AM -0700, Jason Gunthorpe wrote: >We are well past the point of doing experimental breaking stuff in the core >uapi libraries. If it is in the master git it should be shippable by a distro, >and it is so easy >to slap a version number on the HEAD if a distro/ofed/etc >wants one. This may be true, but since it is the master, there could be all kinds of new stuff in it and not just the simple bug fix a H/W vendor might want to put into his/her user-space H/W specific library. Consider the situation when some H/W vendor just wants to provide a simple fix to their library. In the current model, they simply fix it and release a new H/W specific library that a customer or distro can pick up and ship. In the merged library case, the fix has to go into master, which may have other new stuff already in it. If you simply build a new package and roll the version number, it now has to be retested and revalidated by the distro and perhaps other ISVs that have coded and validated with the previous package, since it has new stuff and not just a bug fix. Further, that new stuff might even require new kernel code, so it could not just be replaced as a new user-space package by a distro w/o updating the kernel. There are definitely pros and cons to each model. Since I am not a maintainer of any of the code, I really don't have a strong opinion either way. I think it is really up to all the various H/W vendors and maintainers of the other libraries, umad, rdmacm, ibcm, etc. and the Linux distros which way they would prefer it be done. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, Sep 28, 2015 at 10:32:08AM -0700, Christoph Hellwig wrote: > > Hi Robert, > > getting a package out should not be an issue. master should always be > in releasable state, and cutting a release should be a simple shell > script doing all the tagging and uploading. +1 We are well past the point of doing experimental breaking stuff in the core uapi libraries. If it is in the master git it should be shippable by a distro, and it is so easy to slap a version number on the HEAD if a distro/ofed/etc wants one. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, Sep 28, 2015 at 05:28:20PM +, Woodruff, Robert J wrote: > On Mon, 28 Sep 2015, Christoph Lameter wrote: > > > Right. Its really nasty when you are trying to add features that require > > libibverbs and libmlx? changes. Plus it may depend on kernel changes. > > On the other hand, combining everything into one package limits the > ability of the maintainer of the individual components to release > packages with simple bug fixes or enhancements that are component > specific and don't require kernel core or libibverbs changes. Thus, > they would have to wait till a new combined package gets released > and makes the maintainer of that package the bottleneck for getting > new code out. That is certainly the minority of work these days, by my observation. Nearly everything is being driven by kernel side changes now and requires cross-library work. Or it is maintenance activity, which is so hard now (I've done a few patches over the years) it isn't worth doing unless it is really important. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
Hi Robert, getting a package out should not be an issue. master should always be in releasable state, and cutting a release should be a simple shell script doing all the tagging and uploading. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, 28 Sep 2015, Christoph Lameter wrote: > Right. Its really nasty when you are trying to add features that require > libibverbs and libmlx? changes. Plus it may depend on kernel changes. On the other hand, combining everything into one package limits the ability of the maintainer of the individual components to release packages with simple bug fixes or enhancements that are component specific and don't require kernel core or libibverbs changes. Thus, they would have to wait till a new combined package gets released and makes the maintainer of that package the bottleneck for getting new code out. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, Sep 28, 2015 at 11:17:24AM -0600, Jason Gunthorpe wrote: > Should we combine the user side of the kapi 'core' stack (libverbs, > all open source providers, libumad, libcm) into one source > package? Many projects have been working in that model lately with > some success, IMHO. > > Many of these components are one or two C files, and the overhead of > maintaining N copies of all the auto-crap is more than the package > itself :| > > Right now we even have the situation where some providers won't build > with some verbs's, so it isn't even really the case they are actually > independent. I'm just a bystander in terms of userspace support, but the current model always seemed a complete trainwreck to me. Especially as it implied a public ABI between the core libibvers and the providers, which caused all kinds of problems. Especially if you userspace guys want to keep up with the improvements we're doing on the kernel side you better get up to speed an create a coherent environment :) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, 28 Sep 2015, Jason Gunthorpe wrote: > Should we combine the user side of the kapi 'core' stack (libverbs, > all open source providers, libumad, libcm) into one source > package? Many projects have been working in that model lately with > some success, IMHO. Yes please. > Right now we even have the situation where some providers won't build > with some verbs's, so it isn't even really the case they are actually > independent. Right. Its really nasty when you are trying to add features that require libibverbs and libmlx? changes. Plus it may depend on kernel changes. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, Sep 28, 2015 at 11:43:47AM -0400, Doug Ledford wrote: > On 09/28/2015 11:42 AM, Christoph Lameter wrote: > > Where are these trees and who is maintaining them? I see that there is a > > libibverbs on kernel.org that is updated by Doug. > > > > There are some mlx4/5 trees around but those have no recent commits. There > > is a libmlx4 on kernel.org as well but the last merge there was by Roland > > in May 2014. > > > > git://git.openfabrics.org/~yishaih/libmlx4.git > > git://git.openfabrics.org/~eli/libmlx5.git We've talked about this around and around a few times over the years, but throwing it out there again.. Should we combine the user side of the kapi 'core' stack (libverbs, all open source providers, libumad, libcm) into one source package? Many projects have been working in that model lately with some success, IMHO. Many of these components are one or two C files, and the overhead of maintaining N copies of all the auto-crap is more than the package itself :| Right now we even have the situation where some providers won't build with some verbs's, so it isn't even really the case they are actually independent. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: libmlx4 and libmlx5 git trees? Who is handling those?
Doug Ledford wrote: >git://git.openfabrics.org/~yishaih/libmlx4.git >git://git.openfabrics.org/~eli/libmlx5.git The OFA maintainers list contains this for the maintainers of these trees. ibmlx4 (upstream): Yishai Hadas http://www.openfabrics.org/downloads/mlx4 git://git.openfabrics.org/~yishaih/libmlx4.git libmlx5: Eli Cohen http://www.openfabrics.org/downloads/mlx5 git://git.openfabrics.org/~eli/libmlx5.git -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On Mon, 28 Sep 2015, Doug Ledford wrote: > On 09/28/2015 11:42 AM, Christoph Lameter wrote: > > Where are these trees and who is maintaining them? I see that there is a > > libibverbs on kernel.org that is updated by Doug. > > > > There are some mlx4/5 trees around but those have no recent commits. There > > is a libmlx4 on kernel.org as well but the last merge there was by Roland > > in May 2014. > > > > git://git.openfabrics.org/~yishaih/libmlx4.git Ahhh... There is active development here. > git://git.openfabrics.org/~eli/libmlx5.git More than 6 months of no activity. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libmlx4 and libmlx5 git trees? Who is handling those?
On 09/28/2015 11:42 AM, Christoph Lameter wrote: > Where are these trees and who is maintaining them? I see that there is a > libibverbs on kernel.org that is updated by Doug. > > There are some mlx4/5 trees around but those have no recent commits. There > is a libmlx4 on kernel.org as well but the last merge there was by Roland > in May 2014. > git://git.openfabrics.org/~yishaih/libmlx4.git git://git.openfabrics.org/~eli/libmlx5.git -- Doug Ledford GPG KeyID: 0E572FDD signature.asc Description: OpenPGP digital signature
