[dpdk-dev] Why nothing since 1.8.0?
On Sun, Jan 18, 2015 at 09:48:33PM +, O'driscoll, Tim wrote: > > -Original Message- > > From: Neil Horman [mailto:nhorman at tuxdriver.com] > > Sent: Sunday, January 18, 2015 6:25 PM > > To: O'driscoll, Tim > > Cc: Thomas Monjalon; dev at dpdk.org > > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > > > On Sat, Jan 17, 2015 at 07:57:04PM +, O'driscoll, Tim wrote: > > > I'm going to risk the wrath of the open source purists amongst you by top- > > posting. The reason is that there has been lots of email on this subject, > > and I > > want to summarise the key issue and clearly state our position on it. > > Hoperfully nobody is offended by this! > > > > > Acutally, I am a bit upset by your doing this. While top posting can be an > > acceptible response form, doing so in an interleaved thread gives you the > > opportunity to effectively rewrite the conversation on your own terms. > > While > > you might have summarized your position accurately in your mind, you've > > discarded all the evidence that I presented in opposition. I don't > > appreciate > > that. But we can rebuild from here, no worries. > > > > > This thread has generated lots of debate, which is good, and there are a > > number of items that I believe everybody agrees on (having a MAINTAINERS > > file, better tools etc.). However, the key issue that we do not agree on is > > the > > granularity of the repositories within DPDK. > > > > > No, thats not really the crux of the debate in my mind anymore, though that > > is > > certainly part of it, as it effects the convienience of developers to > > contribute > > to the project. More to the point, the area of disagreement here is the > > best, > > most efficient way to integrate patches to various pieces of the dpdk that > > balances developer convienience, effiency and transparency (I've not > > ennumerated > > that last part before, as I've not thought of the right word, and that may > > still > > not be quite right, but more on it later). > > > > > The current state is: > > > - One master repository > > > - A small number of sub-repositories, each with a separate > > maintainer/SME, including: i40e, fm10k, bnx2x, docs, dts. These are used to > > generate pull requests to the main repo. > > > > > No, this is not the current state. The current state is: > > - One master repository > > We have a repository for docs, with a separate maintainer that was used to > generate pull requests for 1.8. From our perspective that worked well. > For i40e, 1.8 development was done in the main repository, and we agreed to > transition to a separate repo for 2.0. 2.0 development is currently in > progress. > We haven't upstreamed anything for fm10k yet, but that will be done in its > own repo from the start, for the 2.0 release. > > > We have lots of cloned dpdk trees on dpdk.org, but there is no path from > > them to > > the master repository. Nothing has been comitted to any of the subtrees, > > despite having been open for a few months now. > > The plan is to use the i40e and fm10k repos for 2.0 development, which is in > progress. > > > I don't see any > > documentation > > indicating who the maintainer of each tree is, and so don't have the > > foggiest > > idea who to contact if I want to get something merged to these trees. They > > aren't subtrees, they're just clones of the master repository. > > A MAINTAINERS file to clarify responsibilities is a good idea. > > > > You're proposing the following: > > > - Remove the individual PMD repositories, and replace them with a single > > repository containing all PMDs, plus parts of the core code that are closely > > linked to the PMDs, with you as the maintainer and SMEs for each PMD. > > > > > Not necesarily me, mind you (though yes, I've volunteered). I'm happy for > > someone else to take on this role if they volunteer. The point is not to > > have > > a > > separate repo to integrate patches for any one PMD (as theres no need, and > > it > > makes development harder). I want one repository that I can use to target > > development against all PMDs, just like we do with net/net-next in the linux > > community. > > > > > As I've said before, and as Venky has also explained in private email on > > > this, > > we do not agree with this proposed change. We believe it would be a &
[dpdk-dev] Why nothing since 1.8.0?
> -Original Message- > From: Neil Horman [mailto:nhorman at tuxdriver.com] > Sent: Sunday, January 18, 2015 6:25 PM > To: O'driscoll, Tim > Cc: Thomas Monjalon; dev at dpdk.org > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > On Sat, Jan 17, 2015 at 07:57:04PM +, O'driscoll, Tim wrote: > > I'm going to risk the wrath of the open source purists amongst you by top- > posting. The reason is that there has been lots of email on this subject, and > I > want to summarise the key issue and clearly state our position on it. > Hoperfully nobody is offended by this! > > > Acutally, I am a bit upset by your doing this. While top posting can be an > acceptible response form, doing so in an interleaved thread gives you the > opportunity to effectively rewrite the conversation on your own terms. > While > you might have summarized your position accurately in your mind, you've > discarded all the evidence that I presented in opposition. I don't appreciate > that. But we can rebuild from here, no worries. > > > This thread has generated lots of debate, which is good, and there are a > number of items that I believe everybody agrees on (having a MAINTAINERS > file, better tools etc.). However, the key issue that we do not agree on is > the > granularity of the repositories within DPDK. > > > No, thats not really the crux of the debate in my mind anymore, though that > is > certainly part of it, as it effects the convienience of developers to > contribute > to the project. More to the point, the area of disagreement here is the best, > most efficient way to integrate patches to various pieces of the dpdk that > balances developer convienience, effiency and transparency (I've not > ennumerated > that last part before, as I've not thought of the right word, and that may > still > not be quite right, but more on it later). > > > The current state is: > > - One master repository > > - A small number of sub-repositories, each with a separate > maintainer/SME, including: i40e, fm10k, bnx2x, docs, dts. These are used to > generate pull requests to the main repo. > > > No, this is not the current state. The current state is: > - One master repository We have a repository for docs, with a separate maintainer that was used to generate pull requests for 1.8. From our perspective that worked well. For i40e, 1.8 development was done in the main repository, and we agreed to transition to a separate repo for 2.0. 2.0 development is currently in progress. We haven't upstreamed anything for fm10k yet, but that will be done in its own repo from the start, for the 2.0 release. > We have lots of cloned dpdk trees on dpdk.org, but there is no path from > them to > the master repository. Nothing has been comitted to any of the subtrees, > despite having been open for a few months now. The plan is to use the i40e and fm10k repos for 2.0 development, which is in progress. > I don't see any > documentation > indicating who the maintainer of each tree is, and so don't have the foggiest > idea who to contact if I want to get something merged to these trees. They > aren't subtrees, they're just clones of the master repository. A MAINTAINERS file to clarify responsibilities is a good idea. > > You're proposing the following: > > - Remove the individual PMD repositories, and replace them with a single > repository containing all PMDs, plus parts of the core code that are closely > linked to the PMDs, with you as the maintainer and SMEs for each PMD. > > > Not necesarily me, mind you (though yes, I've volunteered). I'm happy for > someone else to take on this role if they volunteer. The point is not to have > a > separate repo to integrate patches for any one PMD (as theres no need, and > it > makes development harder). I want one repository that I can use to target > development against all PMDs, just like we do with net/net-next in the linux > community. > > > As I've said before, and as Venky has also explained in private email on > > this, > we do not agree with this proposed change. We believe it would be a > backward step, and would have an adverse impact on DPDK. > > > You've asserted that several times, but not once have you provided any > supporting evidence or data to back that assertion. Conversely I've provided > several bits of evidence to support my assertion that using the linux > workflow > model would work perfectly well here and handle all our needs (which > include, > but are not limited to, yours). The reason is to give as much control as possible to the people most familiar with the PMD code and the correspon
[dpdk-dev] Why nothing since 1.8.0?
patches than will be present in > any individual PMD. > That's true, but it's also not relevant. Our goal is not to make the > SME/maintainer for i40e, fm10k etc. a full-time position. Our goal is to have > an expert in this position, so that we can move quickly and still ensure good > quality software. > Please re-read your above statement. I think you're contradicting yourself 1) You agree that a tree maintainer can handle a higher volume of patches than any one PMD presents, implying that a tree maintainer can, when properly focused, efficiently take the feedback of SME's and integrate many patches quickly. 2) You claim (1) is irellevant because because your goal is to put an expert in the position of tree maintainer so that you can move more quickly. If you agree that (1) allows for a fast, efficient and transparent workflow, how can you both claim it is both irrelevant and that you need a merged SME/tree maintainer role to achieve the same goal? Question: How exactly do you believe putting an SME in the role of tree maintainer will improve code quality and make code integration faster? > 4. We shouldn't give maintainer work to an SME as it detracts from their > other tasks. > We don't anticipate a problem with workload for the people that we have in > these positions. You're having that problem right now, you just refuse to acknoweldge it. Let me present it for you: http://dpdk.org/dev/patchwork/project/dpdk/list/?q=I40e That shows the only 6 patches that have been posted for I40e since 1.8 released, ranging dating back as far as November 20th. These patches have been sittting on the list since then unacted upon. If fact, taking a closer look, its a bit worse than that. The X710 patch in that list has been integrated, but a version different from the one in patchwork. Its probably just an oversight, but the fact remains, whoever is doing your subtree maintenence is focused on your internal needs and is ignoring their community responsibilities. Thats a problem. > > 5. There will be extra overhead for developers who want to implement changes > spanning multiple PMDs. > That's true, but it's also something of a hypothetical argument. The people > who've spoken against your proposal on the mailing list are from Intel and > 6Wind, who are also the people contributing to most of these PMDs. I had a > quick scan of the commits to see if I could spot anything from another > contributor that might fall into this category, and I couldn't (admittedly it > wasn't an exhaustive search, which someone else is free to do if they want). > If this situation does arise, then Thomas has previously outlined how it can > be handled. Its not a hypothetical argument at all. We have this situation upstream every time someone makes a patch that crosses subsystems, and its managed by the maintainers co-ordinating their efforts when merge time comes along. Thats why its so important to find the right granularity for subtrees, so that extra effort is minimized. And yes, you're right, most of the contributors currently are from 6wind and Intel. The goal here is to spread that participation beyond just the two of you. If you don't have a tree-maintainer that is actively handling patches on the list, getting things integrated in a timely fashion, no one is going to bother participating. As for handling it using the workaround thomas has proposed, I've made my arguments to that effect already several times, but you've neglected to summarize them here with your top post, so let me re-iterate: Patch sets that cross subtrees are a challenge no matter how you slice them. If you have subtrees at a reasonable granularity, subtree maintainers can work together to ensure that the upstream merge goes smoothly. If you force a tree spanning patch to be done in the parent tree you will have merge work to do for each subtree that sends you a pull request. Having a few subtree maintainers handle that work is preferable than having to do merge fixups in the main tree. The main tree should whenever possible have clean merges. > > > In terms of where we go from here, I'd suggest the following: > - Thomas has already asked us about a maintainer for older Intel NICs, which > we're looking into. We may choose to have a single repository with a single > maintainer/SME for e1000/igb/ixgbe combined, which would limit the overall > number of repositories. > - You could pursue a path of having a single repository for non-Intel NICs. > This would obviously need to be worked with those responsible (Stephen, > Sujith etc.). > - As Thomas previously suggested, we should review this in future, possibly > after 2.0. Maybe you'll be proven right and we'll all have to apologise for > disagreeing! It sounds like you'
[dpdk-dev] Why nothing since 1.8.0?
Neil Horman [mailto:nhorman at tuxdriver.com] > Sent: Friday, January 16, 2015 4:51 PM > To: O'driscoll, Tim > Cc: Thomas Monjalon; dev at dpdk.org > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > On Thu, Jan 15, 2015 at 09:55:00PM +, O'driscoll, Tim wrote: > > > -Original Message- > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > > > Sent: Thursday, January 15, 2015 6:51 PM > > > To: Thomas Monjalon > > > Cc: dev at dpdk.org > > > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > > > > > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > > > 2015-01-15 08:06, Neil Horman: > > > > > On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: > > > > > > 2015-01-15 04:27, Ouyang, Changchun: > > > > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, > Helin > > > > > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil > > > Horman > > > > > > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen > Hemminger > > > wrote: > > > > > > > > > > Ok, so 1.8.0 came out almost a month ago and none of the > > > patches > > > > > > > > > > that were deferred waiting for the release got merged since > > > then. > > > > > > > > > > Last commit in git is the 1.8.0 release. > > > > > > > > > > > > > > > > > > > > Where is the post-merge window bundle, where are the > later > > > commits? > > > > > > > > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > > > > > > > > > > > +1, I've had the same questions. > > > > > > > > > Neil > > > > > > > > > > > > > > > > +1, Some patch set might be ready for being merged. > > > > > > > > > > > > > > +1, the earlier some patches are merged into mainline, and the > easier > > > those > > > > > > > sequent patch sets can resolve their conflicts. > > > > > > > > > > > > +1, there are some patches which are properly reviewed > > > > > > > > > > > > Reminder: sub-tree to manage specific part of DPDK can be open on > > > request > > > > > > > > > > Ok, I think what you're saying here is you're too busy to handle all > > > > > the > > > patches > > > > > comming in at the moment. As such I'd like to propose a sub-tree > > > encompassing > > > > > all the pmds in DPDK. I would envision that including all the acutal > pmd's > > > in > > > > > the tree, as well as the infrastructure that is used to interface > > > > > them to > the > > > > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the > > > > > patch > pool > > > > > and send you pull requests. > > > > > > > > The list of PMDs is increasing: > > > > librte_pmd_af_packet > > > > librte_pmd_bond > > > > librte_pmd_e1000 > > > > librte_pmd_enic > > > > librte_pmd_i40e > > > > librte_pmd_ixgbe > > > > librte_pmd_pcap > > > > librte_pmd_ring > > > > librte_pmd_virtio > > > > librte_pmd_vmxnet3 > > > > librte_pmd_xenvirt > > > > There is already some sub-trees for bnx2x, fm10k and i40e: > > > > http://dpdk.org/browse/ > > > > > > > Yes, and I've mentioned before that that is an absolutely silly way to > break > > > out > > > subtrees. You have to find a balance of workload distribution and > developer > > > convienience. > > > > > > I also note that these are problematic because you're not merging > anything > > > from them. Is it your intention to keep bnx2 and fm10k separate in > > > perpituity? > > > If so, thats a real problem, because then we effectively just have several > out > > > of tree drivers, and thats just unacceptible. > > > > > > > > If you could set me up with a login to dpdk.org, I'd appreciate it. > > > > > >
[dpdk-dev] Why nothing since 1.8.0?
I am all for having multiple staging trees that get MERGED into one upstream tree. Let me make it simple, no out of tree drivers. I am not supporting, maintaining or submitting drivers that are not in the mainline DPDK product. If you want me to submit drivers, then they should go into the mainstream. Developers: Want to be able to make changes to core infrastructure during merge window and fix all the related drivers. For many devices, developer wants to be able to submit a driver and have it fixed by others in the main tree. Vendors: Want to be able to extend, change, and enhance the existing DPDK tree and then build an application with that. It is a increase in technical overhead to merge in drivers from multiple sources.
[dpdk-dev] Why nothing since 1.8.0?
2015-01-16 12:20, Neil Horman: > On Thu, Jan 15, 2015 at 11:23:28PM +0100, Thomas Monjalon wrote: > > 2015-01-15 13:51, Neil Horman: > > > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > > > 2015-01-15 08:06, Neil Horman: > > > > > Ok, I think what you're saying here is you're too busy to handle all > > > > > the patches > > > > > comming in at the moment. As such I'd like to propose a sub-tree > > > > > encompassing > > > > > all the pmds in DPDK. I would envision that including all the acutal > > > > > pmd's in > > > > > the tree, as well as the infrastructure that is used to interface > > > > > them to the > > > > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the > > > > > patch pool > > > > > and send you pull requests. > > > > > > > > The list of PMDs is increasing: > > > > librte_pmd_af_packet > > > > librte_pmd_bond > > > > librte_pmd_e1000 > > > > librte_pmd_enic > > > > librte_pmd_i40e > > > > librte_pmd_ixgbe > > > > librte_pmd_pcap > > > > librte_pmd_ring > > > > librte_pmd_virtio > > > > librte_pmd_vmxnet3 > > > > librte_pmd_xenvirt > > > > There is already some sub-trees for bnx2x, fm10k and i40e: > > > > http://dpdk.org/browse/ > > > > > > > Yes, and I've mentioned before that that is an absolutely silly way to > > > break out > > > subtrees. You have to find a balance of workload distribution and > > > developer > > > convienience. > > > > Intel requested fm10k and i40e sub-trees because there are many developments > > in progress. We want to experience this model. > > > Ok, but thats not the point. Just because a given pmd has lots of changes > doesn't mean it itself needs its own tree. With the right separation of > responsibilities all the pmds can be managed from one tree more easily and > with > less distractions to the developers doing the work for those libraries. > > > > I also note that these are problematic because you're not merging anything > > > from them. Is it your intention to keep bnx2 and fm10k separate in > > > perpituity? > > > > No, I'll merge them on pull requests. > > Note that they are planned for version 2.0 but not available yet. > > > Ok, good on the pull request, but I don't really see that happening. > According > to this: > http://git.dpdk.org/dev/roadmap#cycle > If we plan a 2.0 release for mid march, counting backwards, we should be at > the > review period stage. ?? We are January 16 today, so according to http://dpdk.org/dev/roadmap#dates, we are not yet in review period. > As of today in patchwork, I see 6 patches with I40e in the > title. Of those 3 have been acked, only one by someone who I think is likely a > subject matter expert on I40e. > > Some of those patches have been sitting on the list since November 20th of > last > year. > > I think we're missing the point of a subtree. Its created to both take some > of > the load off of the upstream tree maintainer when the volume gets too high, > and > it provides a location for developers to get bleeding edge code for a given > aspect of a project they might be interested in. Neither of these things are > happening here. Please let the things happen. If our experience shows that this subtree is not needed, it is possible to close it. I feel it can be convenient for first releases of new drivers. > > > > > If so, thats a real problem, because then we effectively just have > > > several out > > > of tree drivers, and thats just unacceptible. > > > > I don't understand what make you thinking that. They are -next tree, not > > out of tree. > > > If they are the -next tree, then I apologize, because it certainly doesn't > seem > that way so far. But if they are, so be it. That still leaves the > outstanding > question though of, why one tree per pmd? As I noted in other notes, the > roles > of tree maintainer and driver maintainer (or as I would refer to it, subject > matter expert, for clairity) are separate ones. The former is focused on the > merging of patches and general SCM process, while the latter is focused on > reviewing code within their purview. When done properly, a single tree > maintainer can simply rely on the ACKs of the SME's to gate the merging of > code, > and both parties can do their jobs much more efficiently. > > > > > > If you could set me up with a login to dpdk.org, I'd appreciate it. > > > > > > > > It is preferred to have 1 sub-tree per module. > > > > What do you think of managing contributions for af_packet and/or virtio? > > > > It would make sense as virtio is a RedHat technology. > > > > Maybe it could include vhost lib and example. > > > > > > > No, for reasons I've mentioned before. If you take each pmd/library and > > > create > > > a subtree for it, you've created the most fine grained control of > > > subtrees you > > > could ask for, but you've created a nighmare of a burden on deve
[dpdk-dev] Why nothing since 1.8.0?
On Fri, Jan 16, 2015 at 12:38:48PM -0800, Matthew Hall wrote: > On Fri, Jan 16, 2015 at 03:00:57PM -0500, Neil Horman wrote: > > Like Gerrit: > > https://code.google.com/p/gerrit/ > > Maybe we could work on setting up a community copy? I'd prefer if we could > avoid n=1 and make our community as strong as possible. > Sure, Its a bit orthogonal to this conversation, but I think its a fine idea to have if it aids in general reviewing. Thomas can it be setup on dpdk.org? Neil > Matthew. >
[dpdk-dev] Why nothing since 1.8.0?
On 15/01/15 19:51, Neil Horman wrote: > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: >> 2015-01-15 08:06, Neil Horman: >>> On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: 2015-01-15 04:27, Ouyang, Changchun: > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman >>> On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: Ok, so 1.8.0 came out almost a month ago and none of the patches that were deferred waiting for the release got merged since then. Last commit in git is the 1.8.0 release. Where is the post-merge window bundle, where are the later commits? Lots of patches are sitting rotting in patchwork... >>> +1, I've had the same questions. >>> Neil >> +1, Some patch set might be ready for being merged. > +1, the earlier some patches are merged into mainline, and the easier > those > sequent patch sets can resolve their conflicts. +1, there are some patches which are properly reviewed Reminder: sub-tree to manage specific part of DPDK can be open on request >>> Ok, I think what you're saying here is you're too busy to handle all the >>> patches >>> comming in at the moment. As such I'd like to propose a sub-tree >>> encompassing >>> all the pmds in DPDK. I would envision that including all the acutal pmd's >>> in >>> the tree, as well as the infrastructure that is used to interface them to >>> the >>> core (i.e. the ethdev/rte_ether library). I'll gladly maintain the patch >>> pool >>> and send you pull requests. >> [snip] >> And that doesn't account for the ~500 patches that come in via pull >> request from the wireless subtree. Nor does it account for the merge >> window for net-next being 2 months instead of dpdk's 6 months. Neil, I don't want to hinder the discussion but: could you please point me out where this wireless subtree is? Maybe I am too blind, but I cannot see it here: http://dpdk.org/browse/ We are interested in acceleration for wireless NICs. Marc [snip]
[dpdk-dev] Why nothing since 1.8.0?
On Fri, Jan 16, 2015 at 10:58:52AM -0800, Matthew Hall wrote: > On Fri, Jan 16, 2015 at 07:18:19PM +0100, Thomas Monjalon wrote: > > I'd like to try solving the review challenge first and see what else can be > > done after that. Step by step. > > FWIW, I know the kernel guys seem to really love it, but not everybody else > has much fun trying to do the reviews reading huge patch emails. I lose a lot > of context trying to stare at them in mutt 80x25 console etc. Well, ok, then don't use mutt, no one mandates it. You can setup outlook/thunderbird/evolution/MTA of choice to format email properly for lkml pretty easily. > It would be nice > if we could have a visual interface with syntax highlighting and comment > capabilities, that's easier to read through quickly and clearly, like > ReviewBoard, GitHub Pull Request UI, etc. If it had email integration to > reply > to the patch threads that'd be great too. > Like Gerrit: https://code.google.com/p/gerrit/ Its easy enough to setup your own instance and point it at your own tree for review purposes. > Also if we had some branches available where conceptually related changes are > grouped, somebody could check out the branch with some feature they wanted to > try, get all the related patches, integrate with their app of choice, and see > if the app works successfully with the new feature. > That would be the master branch of a subtree, if the granularity was correct. > Some of these things like DPDK, it isn't obvious how the feature will help or > hurt, until you write some code against it and/or benchmark it first, because > some of these features are kind of complicated. > > Another thing... if we had some kind of wiki page, where some of the backend > coders could mark themselves as maintainers of all the different features > they > work on, and more client-side network stack guys like me could express > interest in certain features, we could connect the two sides so any given guy > knows who can review his bugfix he found, or try out his new patchset to see > if it works well in an app. > Thats what the MAINTAINERS file and --subject-prefix options in git-send-email are commonly used for Neil
[dpdk-dev] Why nothing since 1.8.0?
On Fri, Jan 16, 2015 at 07:18:19PM +0100, Thomas Monjalon wrote: > 2015-01-16 12:20, Neil Horman: > > On Thu, Jan 15, 2015 at 11:23:28PM +0100, Thomas Monjalon wrote: > > > 2015-01-15 13:51, Neil Horman: > > > > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > > > > 2015-01-15 08:06, Neil Horman: > > > > > > Ok, I think what you're saying here is you're too busy to handle > > > > > > all the patches > > > > > > comming in at the moment. As such I'd like to propose a sub-tree > > > > > > encompassing > > > > > > all the pmds in DPDK. I would envision that including all the > > > > > > acutal pmd's in > > > > > > the tree, as well as the infrastructure that is used to interface > > > > > > them to the > > > > > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the > > > > > > patch pool > > > > > > and send you pull requests. > > > > > > > > > > The list of PMDs is increasing: > > > > > librte_pmd_af_packet > > > > > librte_pmd_bond > > > > > librte_pmd_e1000 > > > > > librte_pmd_enic > > > > > librte_pmd_i40e > > > > > librte_pmd_ixgbe > > > > > librte_pmd_pcap > > > > > librte_pmd_ring > > > > > librte_pmd_virtio > > > > > librte_pmd_vmxnet3 > > > > > librte_pmd_xenvirt > > > > > There is already some sub-trees for bnx2x, fm10k and i40e: > > > > > http://dpdk.org/browse/ > > > > > > > > > Yes, and I've mentioned before that that is an absolutely silly way to > > > > break out > > > > subtrees. You have to find a balance of workload distribution and > > > > developer > > > > convienience. > > > > > > Intel requested fm10k and i40e sub-trees because there are many > > > developments > > > in progress. We want to experience this model. > > > > > Ok, but thats not the point. Just because a given pmd has lots of changes > > doesn't mean it itself needs its own tree. With the right separation of > > responsibilities all the pmds can be managed from one tree more easily and > > with > > less distractions to the developers doing the work for those libraries. > > > > > > I also note that these are problematic because you're not merging > > > > anything > > > > from them. Is it your intention to keep bnx2 and fm10k separate in > > > > perpituity? > > > > > > No, I'll merge them on pull requests. > > > Note that they are planned for version 2.0 but not available yet. > > > > > Ok, good on the pull request, but I don't really see that happening. > > According > > to this: > > http://git.dpdk.org/dev/roadmap#cycle > > If we plan a 2.0 release for mid march, counting backwards, we should be at > > the > > review period stage. > > ?? We are January 16 today, so according to http://dpdk.org/dev/roadmap#dates, > we are not yet in review period. > Ok, you still have two weeks, but my point still stands. At least for I40e, several of the outstanding patches have been on the list since November of last year, and have been Acked. If we're separating the trees out, why are we waiting so long to merge them, shouldn't that have been done by the subtree maintainer by now? The merge window is called a merge window for a reason, its the time that code gets merged. > > As of today in patchwork, I see 6 patches with I40e in the > > title. Of those 3 have been acked, only one by someone who I think is > > likely a > > subject matter expert on I40e. > > > > Some of those patches have been sitting on the list since November 20th of > > last > > year. > > > > I think we're missing the point of a subtree. Its created to both take > > some of > > the load off of the upstream tree maintainer when the volume gets too high, > > and > > it provides a location for developers to get bleeding edge code for a given > > aspect of a project they might be interested in. Neither of these things > > are > > happening here. > > Please let the things happen. > If our experience shows that this subtree is not needed, it is possible to > close it. > I feel it can be convenient for first releases of new drivers. > I'm not disagreeing, I'm advocating for the fact that theres no need to open and close trees as it becomes (in)convienient. At least not at such a fine granularity. A single subtree for all the pmds with one tree maintainer and several subject matter experts to do reviews has proven to be highly efficient and flexible in many other projects (most notably the linux kernel). > > > > > > > If so, thats a real problem, because then we effectively just have > > > > several out > > > > of tree drivers, and thats just unacceptible. > > > > > > I don't understand what make you thinking that. They are -next tree, not > > > out of tree. > > > > > If they are the -next tree, then I apologize, because it certainly doesn't > > seem > > that way so far. But if they are, so be it. That still leaves the > > outstanding > > question though of, why one tree per pmd? As I noted in other notes, the > > ro
[dpdk-dev] Why nothing since 1.8.0?
On Fri, Jan 16, 2015 at 04:14:38PM -0500, Neil Horman wrote: > Sure, Its a bit orthogonal to this conversation, but I think its a fine idea > to > have if it aids in general reviewing. Thomas can it be setup on dpdk.org? > > Neil Admittedly I'm not a PMD expert to comment on all the specifics of what you said about subtrees. But I do like to think out of the box and look at the big picture of what people have to say in the various threads. Your points about making the community strong seemed important, so I thought about the various subproblems to solve the whole topic: 1) some logical subtrees as you advised, 2) a clarification of who runs the subtrees and who does the -next mergeups, 3) feature branches or some other way for end users to validate new related functionality all together as a kind of integration testing, 4) a MAINTAINERS file, and maybe a TESTERS file, or tester / end-user entries in MAINTAINERS, 5) a really good friend way to review the new code like Gerrit as you advised I think if we attack all of these we should be in good shape as the community continues to grow and mature. Matthew.
[dpdk-dev] Why nothing since 1.8.0?
On Fri, Jan 16, 2015 at 03:00:57PM -0500, Neil Horman wrote: > Like Gerrit: > https://code.google.com/p/gerrit/ Maybe we could work on setting up a community copy? I'd prefer if we could avoid n=1 and make our community as strong as possible. Matthew.
[dpdk-dev] Why nothing since 1.8.0?
On Thu, Jan 15, 2015 at 11:23:28PM +0100, Thomas Monjalon wrote: > 2015-01-15 13:51, Neil Horman: > > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > > 2015-01-15 08:06, Neil Horman: > > > > Ok, I think what you're saying here is you're too busy to handle all > > > > the patches > > > > comming in at the moment. As such I'd like to propose a sub-tree > > > > encompassing > > > > all the pmds in DPDK. I would envision that including all the acutal > > > > pmd's in > > > > the tree, as well as the infrastructure that is used to interface them > > > > to the > > > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the > > > > patch pool > > > > and send you pull requests. > > > > > > The list of PMDs is increasing: > > > librte_pmd_af_packet > > > librte_pmd_bond > > > librte_pmd_e1000 > > > librte_pmd_enic > > > librte_pmd_i40e > > > librte_pmd_ixgbe > > > librte_pmd_pcap > > > librte_pmd_ring > > > librte_pmd_virtio > > > librte_pmd_vmxnet3 > > > librte_pmd_xenvirt > > > There is already some sub-trees for bnx2x, fm10k and i40e: > > > http://dpdk.org/browse/ > > > > > Yes, and I've mentioned before that that is an absolutely silly way to > > break out > > subtrees. You have to find a balance of workload distribution and developer > > convienience. > > Intel requested fm10k and i40e sub-trees because there are many developments > in progress. We want to experience this model. > Ok, but thats not the point. Just because a given pmd has lots of changes doesn't mean it itself needs its own tree. With the right separation of responsibilities all the pmds can be managed from one tree more easily and with less distractions to the developers doing the work for those libraries. > > I also note that these are problematic because you're not merging anything > > from them. Is it your intention to keep bnx2 and fm10k separate in > > perpituity? > > No, I'll merge them on pull requests. > Note that they are planned for version 2.0 but not available yet. > Ok, good on the pull request, but I don't really see that happening. According to this: http://git.dpdk.org/dev/roadmap#cycle If we plan a 2.0 release for mid march, counting backwards, we should be at the review period stage. As of today in patchwork, I see 6 patches with I40e in the title. Of those 3 have been acked, only one by someone who I think is likely a subject matter expert on I40e. Some of those patches have been sitting on the list since November 20th of last year. I think we're missing the point of a subtree. Its created to both take some of the load off of the upstream tree maintainer when the volume gets too high, and it provides a location for developers to get bleeding edge code for a given aspect of a project they might be interested in. Neither of these things are happening here. > > If so, thats a real problem, because then we effectively just have several > > out > > of tree drivers, and thats just unacceptible. > > I don't understand what make you thinking that. They are -next tree, not out > of tree. > If they are the -next tree, then I apologize, because it certainly doesn't seem that way so far. But if they are, so be it. That still leaves the outstanding question though of, why one tree per pmd? As I noted in other notes, the roles of tree maintainer and driver maintainer (or as I would refer to it, subject matter expert, for clairity) are separate ones. The former is focused on the merging of patches and general SCM process, while the latter is focused on reviewing code within their purview. When done properly, a single tree maintainer can simply rely on the ACKs of the SME's to gate the merging of code, and both parties can do their jobs much more efficiently. > > > > If you could set me up with a login to dpdk.org, I'd appreciate it. > > > > > > It is preferred to have 1 sub-tree per module. > > > What do you think of managing contributions for af_packet and/or virtio? > > > It would make sense as virtio is a RedHat technology. > > > Maybe it could include vhost lib and example. > > > > > No, for reasons I've mentioned before. If you take each pmd/library and > > create > > a subtree for it, you've created the most fine grained control of subtrees > > you > > could ask for, but you've created a nighmare of a burden on developers who > > want > > to update any code, especially if they have patches that hit multiple trees. > > It's not planned to have a sub-tree for each library. > And some sub-trees can be closed when activity decrease. > But that need not happen. If you create a single sub-tree for all the PMD's, it will have a long life, and developers will have a long lived canonical source from which to get the latest pmd code, and will enjoy the benefits of more rapidly merged/reviewed code, if we follow the dual role approach that I've been advocating. > > Look at some of the stats in the dpdk tree: > > > > Library Comm
[dpdk-dev] Why nothing since 1.8.0?
On Fri, Jan 16, 2015 at 03:51:55PM +0100, Marc Sune wrote: > > On 15/01/15 19:51, Neil Horman wrote: > >On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > >>2015-01-15 08:06, Neil Horman: > >>>On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: > 2015-01-15 04:27, Ouyang, Changchun: > >From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > >>From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > >>>On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > Ok, so 1.8.0 came out almost a month ago and none of the patches > that were deferred waiting for the release got merged since then. > Last commit in git is the 1.8.0 release. > > Where is the post-merge window bundle, where are the later commits? > Lots of patches are sitting rotting in patchwork... > >>>+1, I've had the same questions. > >>>Neil > >>+1, Some patch set might be ready for being merged. > >+1, the earlier some patches are merged into mainline, and the easier > >those > >sequent patch sets can resolve their conflicts. > +1, there are some patches which are properly reviewed > > Reminder: sub-tree to manage specific part of DPDK can be open on request > >>>Ok, I think what you're saying here is you're too busy to handle all the > >>>patches > >>>comming in at the moment. As such I'd like to propose a sub-tree > >>>encompassing > >>>all the pmds in DPDK. I would envision that including all the acutal > >>>pmd's in > >>>the tree, as well as the infrastructure that is used to interface them to > >>>the > >>>core (i.e. the ethdev/rte_ether library). I'll gladly maintain the patch > >>>pool > >>>and send you pull requests. > >>[snip] > >>And that doesn't account for the ~500 patches that come in via pull > >>request from the wireless subtree. Nor does it account for the merge > >>window for net-next being 2 months instead of dpdk's 6 months. > > Neil, > > I don't want to hinder the discussion but: could you please point me out > where this wireless subtree is? > There is none for DPDK, I'm drawing a comparison here. The DPDK is trying to split out subtrees at a granularity of 1 tree per pmd, which I am aruging against because a maintainer for a pmd doesn't need their own tree to push to. Instead I'm proposing a large granularity of where all drivers (or drivers of a certain type) are housed in the same tree, like net-next: https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git/ or wireless: https://git.kernel.org/cgit/linux/kernel/git/kvalo/wireless-drivers.git/ where one tree maintainer relies on the review of many subject matter experts to validate the patches that they merge. I'm making the comparison to argue for workflow process, nothing more. You won't find any wireless pmds anywhere at the moment. Neil > Maybe I am too blind, but I cannot see it here: > > http://dpdk.org/browse/ > > We are interested in acceleration for wireless NICs. > > Marc > > [snip] >
[dpdk-dev] Why nothing since 1.8.0?
On Thu, Jan 15, 2015 at 09:55:00PM +, O'driscoll, Tim wrote: > > -Original Message- > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > > Sent: Thursday, January 15, 2015 6:51 PM > > To: Thomas Monjalon > > Cc: dev at dpdk.org > > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > > > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > > 2015-01-15 08:06, Neil Horman: > > > > On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: > > > > > 2015-01-15 04:27, Ouyang, Changchun: > > > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > > > > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil > > Horman > > > > > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger > > wrote: > > > > > > > > > Ok, so 1.8.0 came out almost a month ago and none of the > > patches > > > > > > > > > that were deferred waiting for the release got merged since > > then. > > > > > > > > > Last commit in git is the 1.8.0 release. > > > > > > > > > > > > > > > > > > Where is the post-merge window bundle, where are the later > > commits? > > > > > > > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > > > > > > > > > +1, I've had the same questions. > > > > > > > > Neil > > > > > > > > > > > > > > +1, Some patch set might be ready for being merged. > > > > > > > > > > > > +1, the earlier some patches are merged into mainline, and the > > > > > > easier > > those > > > > > > sequent patch sets can resolve their conflicts. > > > > > > > > > > +1, there are some patches which are properly reviewed > > > > > > > > > > Reminder: sub-tree to manage specific part of DPDK can be open on > > request > > > > > > > > Ok, I think what you're saying here is you're too busy to handle all the > > patches > > > > comming in at the moment. As such I'd like to propose a sub-tree > > encompassing > > > > all the pmds in DPDK. I would envision that including all the acutal > > > > pmd's > > in > > > > the tree, as well as the infrastructure that is used to interface them > > > > to the > > > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the > > > > patch pool > > > > and send you pull requests. > > > > > > The list of PMDs is increasing: > > > librte_pmd_af_packet > > > librte_pmd_bond > > > librte_pmd_e1000 > > > librte_pmd_enic > > > librte_pmd_i40e > > > librte_pmd_ixgbe > > > librte_pmd_pcap > > > librte_pmd_ring > > > librte_pmd_virtio > > > librte_pmd_vmxnet3 > > > librte_pmd_xenvirt > > > There is already some sub-trees for bnx2x, fm10k and i40e: > > > http://dpdk.org/browse/ > > > > > Yes, and I've mentioned before that that is an absolutely silly way to break > > out > > subtrees. You have to find a balance of workload distribution and developer > > convienience. > > > > I also note that these are problematic because you're not merging anything > > from them. Is it your intention to keep bnx2 and fm10k separate in > > perpituity? > > If so, thats a real problem, because then we effectively just have several > > out > > of tree drivers, and thats just unacceptible. > > > > > > If you could set me up with a login to dpdk.org, I'd appreciate it. > > > > > > It is preferred to have 1 sub-tree per module. > > > What do you think of managing contributions for af_packet and/or virtio? > > > It would make sense as virtio is a RedHat technology. > > > Maybe it could include vhost lib and example. > > > > > No, for reasons I've mentioned before. If you take each pmd/library and > > create > > a subtree for it, you've created the most fine grained control of subtrees > > you > > could ask for, but you've created a nighmare of a burden on developers who > > want > > to update any code, especially if they have patches that hit multiple trees. > > > >
[dpdk-dev] Why nothing since 1.8.0?
On Fri, Jan 16, 2015 at 07:18:19PM +0100, Thomas Monjalon wrote: > I'd like to try solving the review challenge first and see what else can be > done after that. Step by step. FWIW, I know the kernel guys seem to really love it, but not everybody else has much fun trying to do the reviews reading huge patch emails. I lose a lot of context trying to stare at them in mutt 80x25 console etc. It would be nice if we could have a visual interface with syntax highlighting and comment capabilities, that's easier to read through quickly and clearly, like ReviewBoard, GitHub Pull Request UI, etc. If it had email integration to reply to the patch threads that'd be great too. Also if we had some branches available where conceptually related changes are grouped, somebody could check out the branch with some feature they wanted to try, get all the related patches, integrate with their app of choice, and see if the app works successfully with the new feature. Some of these things like DPDK, it isn't obvious how the feature will help or hurt, until you write some code against it and/or benchmark it first, because some of these features are kind of complicated. Another thing... if we had some kind of wiki page, where some of the backend coders could mark themselves as maintainers of all the different features they work on, and more client-side network stack guys like me could express interest in certain features, we could connect the two sides so any given guy knows who can review his bugfix he found, or try out his new patchset to see if it works well in an app. Matthew.
[dpdk-dev] Why nothing since 1.8.0?
2015-01-15 17:46, Matthew Hall: > On Thu, Jan 15, 2015 at 09:55:00PM +, O'driscoll, Tim wrote: > > As you said, there's a balance to be struck, and too many subtrees may > > become unmanageable. With respect to your concern about developers having > > to > > potentially develop patches against multiple subtrees, this has never been > > raised as a concern by any of our development team. Is there any historical > > data on the number of changes that would fall into this category so we can > > see if it's a real problem or not? > > Hi Tim, > > What happens when a core API like rte_mbuf gets some changes, and you have to > update the PMD's to fit? > > Do I have to make 10-20 odd random patches to separate PMD maintainers > instead > of one set of patches to the PMD subtree? Then the patchset is core-wide and must be managed in the main tree. > To me it doesn't sound very nice for the guys maintaining the core. Given > most > of the changes seem to be mbuf or eal this seems like a scaling issue to me. In previous release, there were a lot of changes related to i40e. And we expect to have the same level of activity for fm10k. > But maybe I misunderstood the process. No problem, we are starting experiencing this model and will write some guidelines. -- Thomas
[dpdk-dev] Why nothing since 1.8.0?
2015-01-15 13:51, Neil Horman: > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > 2015-01-15 08:06, Neil Horman: > > > Ok, I think what you're saying here is you're too busy to handle all the > > > patches > > > comming in at the moment. As such I'd like to propose a sub-tree > > > encompassing > > > all the pmds in DPDK. I would envision that including all the acutal > > > pmd's in > > > the tree, as well as the infrastructure that is used to interface them to > > > the > > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the patch > > > pool > > > and send you pull requests. > > > > The list of PMDs is increasing: > > librte_pmd_af_packet > > librte_pmd_bond > > librte_pmd_e1000 > > librte_pmd_enic > > librte_pmd_i40e > > librte_pmd_ixgbe > > librte_pmd_pcap > > librte_pmd_ring > > librte_pmd_virtio > > librte_pmd_vmxnet3 > > librte_pmd_xenvirt > > There is already some sub-trees for bnx2x, fm10k and i40e: > > http://dpdk.org/browse/ > > > Yes, and I've mentioned before that that is an absolutely silly way to break > out > subtrees. You have to find a balance of workload distribution and developer > convienience. Intel requested fm10k and i40e sub-trees because there are many developments in progress. We want to experience this model. > I also note that these are problematic because you're not merging anything > from them. Is it your intention to keep bnx2 and fm10k separate in perpituity? No, I'll merge them on pull requests. Note that they are planned for version 2.0 but not available yet. > If so, thats a real problem, because then we effectively just have several out > of tree drivers, and thats just unacceptible. I don't understand what make you thinking that. They are -next tree, not out of tree. > > > If you could set me up with a login to dpdk.org, I'd appreciate it. > > > > It is preferred to have 1 sub-tree per module. > > What do you think of managing contributions for af_packet and/or virtio? > > It would make sense as virtio is a RedHat technology. > > Maybe it could include vhost lib and example. > > > No, for reasons I've mentioned before. If you take each pmd/library and > create > a subtree for it, you've created the most fine grained control of subtrees you > could ask for, but you've created a nighmare of a burden on developers who > want > to update any code, especially if they have patches that hit multiple trees. It's not planned to have a sub-tree for each library. And some sub-trees can be closed when activity decrease. > Look at some of the stats in the dpdk tree: > > Library Commits between 1.7.0 and 1.8.0 > librte_acl5 > librte_cfgfile0 > librte_cmdline4 > librte_compat 0 > librte_distributor5 > librte_eal125 > librte_ether 31 > librte_hash 1 > librte_ip_frag5 > librte_ivshmem0 > librte_kni2 > librte_kvargs 0 > librte_lpm1 > librte_malloc 1 > librte_mbuf 39 > librte_mempool4 > librte_meter 0 > librte_net4 > librte_pipeline 0 > librte_pmd_af_packet 4 > librte_pmd_bond 20 > librte_pmd_e1000 21 > librte_pmd_enic 12 > librte_pmd_i40e 90 > librte_pmd_ixgbe 83 > librte_pmd_pcap 4 > librte_pmd_ring 0 > librte_pmd_virtio 21 > librte_pmd_vmxnet321 > librte_pmd_xenvirt6 > librte_port 6 > librte_power 3 > librte_ring 2 > librte_sched 1 > librte_table 7 > librte_timer 0 > librte_vhost 30 > > If you look at all of the pmds in the dpdk tree, we're talking about ~300 > patches per release. If you look at the net-next tree for the linux kernel, > Dave Miller merged 569 patches on his own (based on the following command: > git log --pretty=format:%H v3.17..v3.18 -- drivers/net/ethernet/ net/core/ | > wc > -l) > > And that doesn't account for the ~500 patches that come in via pull request > from > the wireless subtree. Nor does it account for the merge window for net-next > being 2 months instead of dpdk's 6 months. Theres no need in any way for 12 > maintainers to be twiddling their thumbs waiting on ~20 patches each, and for > that split, you've forced developers to potentially develop patches against 12 > trees (12 being the current number of PMD's that are in the dpdk). Please stop on this wrong assumption. We keep only 1 mailing-list and use only few sub-trees. > The right answer here is balance. Let me split out the pmd's and ethernet > infrastructure libraries to a subtree. I'll pull in patches posted regarding > pmd's and librte_ether/ip_frag etc, and send you a pull requests after each > release so you get all the latest bits, and then pulls for stabilization on > each > -rc. I can manage 300 patches without issu
[dpdk-dev] Why nothing since 1.8.0?
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > Sent: Thursday, January 15, 2015 6:51 PM > To: Thomas Monjalon > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > 2015-01-15 08:06, Neil Horman: > > > On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: > > > > 2015-01-15 04:27, Ouyang, Changchun: > > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > > > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil > Horman > > > > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger > wrote: > > > > > > > > Ok, so 1.8.0 came out almost a month ago and none of the > patches > > > > > > > > that were deferred waiting for the release got merged since > then. > > > > > > > > Last commit in git is the 1.8.0 release. > > > > > > > > > > > > > > > > Where is the post-merge window bundle, where are the later > commits? > > > > > > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > > > > > > > +1, I've had the same questions. > > > > > > > Neil > > > > > > > > > > > > +1, Some patch set might be ready for being merged. > > > > > > > > > > +1, the earlier some patches are merged into mainline, and the easier > those > > > > > sequent patch sets can resolve their conflicts. > > > > > > > > +1, there are some patches which are properly reviewed > > > > > > > > Reminder: sub-tree to manage specific part of DPDK can be open on > request > > > > > > Ok, I think what you're saying here is you're too busy to handle all the > patches > > > comming in at the moment. As such I'd like to propose a sub-tree > encompassing > > > all the pmds in DPDK. I would envision that including all the acutal > > > pmd's > in > > > the tree, as well as the infrastructure that is used to interface them to > > > the > > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the patch > > > pool > > > and send you pull requests. > > > > The list of PMDs is increasing: > > librte_pmd_af_packet > > librte_pmd_bond > > librte_pmd_e1000 > > librte_pmd_enic > > librte_pmd_i40e > > librte_pmd_ixgbe > > librte_pmd_pcap > > librte_pmd_ring > > librte_pmd_virtio > > librte_pmd_vmxnet3 > > librte_pmd_xenvirt > > There is already some sub-trees for bnx2x, fm10k and i40e: > > http://dpdk.org/browse/ > > > Yes, and I've mentioned before that that is an absolutely silly way to break > out > subtrees. You have to find a balance of workload distribution and developer > convienience. > > I also note that these are problematic because you're not merging anything > from them. Is it your intention to keep bnx2 and fm10k separate in > perpituity? > If so, thats a real problem, because then we effectively just have several out > of tree drivers, and thats just unacceptible. > > > > If you could set me up with a login to dpdk.org, I'd appreciate it. > > > > It is preferred to have 1 sub-tree per module. > > What do you think of managing contributions for af_packet and/or virtio? > > It would make sense as virtio is a RedHat technology. > > Maybe it could include vhost lib and example. > > > No, for reasons I've mentioned before. If you take each pmd/library and > create > a subtree for it, you've created the most fine grained control of subtrees you > could ask for, but you've created a nighmare of a burden on developers who > want > to update any code, especially if they have patches that hit multiple trees. > > Look at some of the stats in the dpdk tree: > > Library Commits between 1.7.0 and 1.8.0 > librte_acl5 > librte_cfgfile0 > librte_cmdline4 > librte_compat 0 > librte_distributor5 > librte_eal125 > librte_ether 31 > librte_hash 1 > librte_ip_frag5 > librte_ivshmem0 > librte_kni2 > librte_kvargs 0 > librte_lpm1 > librte_malloc 1 >
[dpdk-dev] Why nothing since 1.8.0?
2015-01-15 08:06, Neil Horman: > On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: > > 2015-01-15 04:27, Ouyang, Changchun: > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > > > > > > Ok, so 1.8.0 came out almost a month ago and none of the patches > > > > > > that were deferred waiting for the release got merged since then. > > > > > > Last commit in git is the 1.8.0 release. > > > > > > > > > > > > Where is the post-merge window bundle, where are the later commits? > > > > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > > > +1, I've had the same questions. > > > > > Neil > > > > > > > > +1, Some patch set might be ready for being merged. > > > > > > +1, the earlier some patches are merged into mainline, and the easier > > > those > > > sequent patch sets can resolve their conflicts. > > > > +1, there are some patches which are properly reviewed > > > > Reminder: sub-tree to manage specific part of DPDK can be open on request > > Ok, I think what you're saying here is you're too busy to handle all the > patches > comming in at the moment. As such I'd like to propose a sub-tree encompassing > all the pmds in DPDK. I would envision that including all the acutal pmd's in > the tree, as well as the infrastructure that is used to interface them to the > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the patch pool > and send you pull requests. The list of PMDs is increasing: librte_pmd_af_packet librte_pmd_bond librte_pmd_e1000 librte_pmd_enic librte_pmd_i40e librte_pmd_ixgbe librte_pmd_pcap librte_pmd_ring librte_pmd_virtio librte_pmd_vmxnet3 librte_pmd_xenvirt There is already some sub-trees for bnx2x, fm10k and i40e: http://dpdk.org/browse/ > If you could set me up with a login to dpdk.org, I'd appreciate it. It is preferred to have 1 sub-tree per module. What do you think of managing contributions for af_packet and/or virtio? It would make sense as virtio is a RedHat technology. Maybe it could include vhost lib and example. Thanks for proposing -- Thomas
[dpdk-dev] Why nothing since 1.8.0?
On Thu, Jan 15, 2015 at 09:55:00PM +, O'driscoll, Tim wrote: > As you said, there's a balance to be struck, and too many subtrees may > become unmanageable. With respect to your concern about developers having to > potentially develop patches against multiple subtrees, this has never been > raised as a concern by any of our development team. Is there any historical > data on the number of changes that would fall into this category so we can > see if it's a real problem or not? Hi Tim, What happens when a core API like rte_mbuf gets some changes, and you have to update the PMD's to fit? Do I have to make 10-20 odd random patches to separate PMD maintainers instead of one set of patches to the PMD subtree? To me it doesn't sound very nice for the guys maintaining the core. Given most of the changes seem to be mbuf or eal this seems like a scaling issue to me. But maybe I misunderstood the process. Matthew.
[dpdk-dev] Why nothing since 1.8.0?
On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > 2015-01-15 08:06, Neil Horman: > > On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: > > > 2015-01-15 04:27, Ouyang, Changchun: > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > > > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > > > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > > > > > > > Ok, so 1.8.0 came out almost a month ago and none of the patches > > > > > > > that were deferred waiting for the release got merged since then. > > > > > > > Last commit in git is the 1.8.0 release. > > > > > > > > > > > > > > Where is the post-merge window bundle, where are the later > > > > > > > commits? > > > > > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > > > > > +1, I've had the same questions. > > > > > > Neil > > > > > > > > > > +1, Some patch set might be ready for being merged. > > > > > > > > +1, the earlier some patches are merged into mainline, and the easier > > > > those > > > > sequent patch sets can resolve their conflicts. > > > > > > +1, there are some patches which are properly reviewed > > > > > > Reminder: sub-tree to manage specific part of DPDK can be open on request > > > > Ok, I think what you're saying here is you're too busy to handle all the > > patches > > comming in at the moment. As such I'd like to propose a sub-tree > > encompassing > > all the pmds in DPDK. I would envision that including all the acutal pmd's > > in > > the tree, as well as the infrastructure that is used to interface them to > > the > > core (i.e. the ethdev/rte_ether library). I'll gladly maintain the patch > > pool > > and send you pull requests. > > The list of PMDs is increasing: > librte_pmd_af_packet > librte_pmd_bond > librte_pmd_e1000 > librte_pmd_enic > librte_pmd_i40e > librte_pmd_ixgbe > librte_pmd_pcap > librte_pmd_ring > librte_pmd_virtio > librte_pmd_vmxnet3 > librte_pmd_xenvirt > There is already some sub-trees for bnx2x, fm10k and i40e: > http://dpdk.org/browse/ > Yes, and I've mentioned before that that is an absolutely silly way to break out subtrees. You have to find a balance of workload distribution and developer convienience. I also note that these are problematic because you're not merging anything from them. Is it your intention to keep bnx2 and fm10k separate in perpituity? If so, thats a real problem, because then we effectively just have several out of tree drivers, and thats just unacceptible. > > If you could set me up with a login to dpdk.org, I'd appreciate it. > > It is preferred to have 1 sub-tree per module. > What do you think of managing contributions for af_packet and/or virtio? > It would make sense as virtio is a RedHat technology. > Maybe it could include vhost lib and example. > No, for reasons I've mentioned before. If you take each pmd/library and create a subtree for it, you've created the most fine grained control of subtrees you could ask for, but you've created a nighmare of a burden on developers who want to update any code, especially if they have patches that hit multiple trees. Look at some of the stats in the dpdk tree: Library Commits between 1.7.0 and 1.8.0 librte_acl 5 librte_cfgfile 0 librte_cmdline 4 librte_compat 0 librte_distributor 5 librte_eal 125 librte_ether31 librte_hash 1 librte_ip_frag 5 librte_ivshmem 0 librte_kni 2 librte_kvargs 0 librte_lpm 1 librte_malloc 1 librte_mbuf 39 librte_mempool 4 librte_meter0 librte_net 4 librte_pipeline 0 librte_pmd_af_packet4 librte_pmd_bond 20 librte_pmd_e100021 librte_pmd_enic 12 librte_pmd_i40e 90 librte_pmd_ixgbe83 librte_pmd_pcap 4 librte_pmd_ring 0 librte_pmd_virtio 21 librte_pmd_vmxnet3 21 librte_pmd_xenvirt 6 librte_port 6 librte_power3 librte_ring 2 librte_sched1 librte_table7 librte_timer0 librte_vhost30 If you look at all of the pmds in the dpdk tree, we're talking about ~300 patches per release. If you look at the net-next tree for the linux kernel, Dave Miller merged 569 patches on his own (based on the following command: git log --pretty=format:%H v3.17..v3.18 -- drivers/net/ethernet/ net/core/ | wc -l) And that doesn't account for the ~500 patches that come in via pull request from the wireless subtree. Nor does it account for the merge window for net-next being 2 months instead of dpdk's 6 months. Theres no need in any way for 12 maintainers to be twiddling their thumbs waiting on ~20 patches each, and for that split, you've forced developers to poten
[dpdk-dev] Why nothing since 1.8.0?
2015-01-15 04:27, Ouyang, Changchun: > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > > > > Ok, so 1.8.0 came out almost a month ago and none of the patches > > > > that were deferred waiting for the release got merged since then. > > > > Last commit in git is the 1.8.0 release. > > > > > > > > Where is the post-merge window bundle, where are the later commits? > > > > Lots of patches are sitting rotting in patchwork... > > > > > > +1, I've had the same questions. > > > Neil > > > > +1, Some patch set might be ready for being merged. > > +1, the earlier some patches are merged into mainline, and the easier those > sequent patch sets can resolve their conflicts. +1, there are some patches which are properly reviewed Reminder: sub-tree to manage specific part of DPDK can be open on request -- Thomas, back on track
[dpdk-dev] Why nothing since 1.8.0?
On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote: > 2015-01-15 04:27, Ouyang, Changchun: > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > > > > > Ok, so 1.8.0 came out almost a month ago and none of the patches > > > > > that were deferred waiting for the release got merged since then. > > > > > Last commit in git is the 1.8.0 release. > > > > > > > > > > Where is the post-merge window bundle, where are the later commits? > > > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > +1, I've had the same questions. > > > > Neil > > > > > > +1, Some patch set might be ready for being merged. > > > > +1, the earlier some patches are merged into mainline, and the easier those > > sequent patch sets can resolve their conflicts. > > +1, there are some patches which are properly reviewed > > Reminder: sub-tree to manage specific part of DPDK can be open on request > Ok, I think what you're saying here is you're too busy to handle all the patches comming in at the moment. As such I'd like to propose a sub-tree encompassing all the pmds in DPDK. I would envision that including all the acutal pmd's in the tree, as well as the infrastructure that is used to interface them to the core (i.e. the ethdev/rte_ether library). I'll gladly maintain the patch pool and send you pull requests. If you could set me up with a login to dpdk.org, I'd appreciate it. Thanks! Neil > -- > Thomas, back on track >
[dpdk-dev] Why nothing since 1.8.0?
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin > Sent: Thursday, January 15, 2015 12:15 PM > To: Neil Horman; Stephen Hemminger > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > +1, Some patch set might be ready for being merged. > > > -Original Message- > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > > Sent: Thursday, January 15, 2015 5:02 AM > > To: Stephen Hemminger > > Cc: dev at dpdk.org > > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > > > Ok, so 1.8.0 came out almost a month ago and none of the patches > > > that were deferred waiting for the release got merged since then. > > > Last commit in git is the 1.8.0 release. > > > > > > Where is the post-merge window bundle, where are the later commits? > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > > > +1, I've had the same questions. > > Neil +1, the earlier some patches are merged into mainline, and the easier those sequent patch sets can resolve their conflicts. Thanks Changchun
[dpdk-dev] Why nothing since 1.8.0?
+1, Some patch set might be ready for being merged. > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > Sent: Thursday, January 15, 2015 5:02 AM > To: Stephen Hemminger > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > > Ok, so 1.8.0 came out almost a month ago and none of the patches that > > were deferred waiting for the release got merged since then. > > Last commit in git is the 1.8.0 release. > > > > Where is the post-merge window bundle, where are the later commits? > > Lots of patches are sitting rotting in patchwork... > > > > > > +1, I've had the same questions. > Neil
[dpdk-dev] Why nothing since 1.8.0?
On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote: > Ok, so 1.8.0 came out almost a month ago and none of the patches > that were deferred waiting for the release got merged since then. > Last commit in git is the 1.8.0 release. > > Where is the post-merge window bundle, where are the later commits? > Lots of patches are sitting rotting in patchwork... > > +1, I've had the same questions. Neil
[dpdk-dev] Why nothing since 1.8.0?
Ok, so 1.8.0 came out almost a month ago and none of the patches that were deferred waiting for the release got merged since then. Last commit in git is the 1.8.0 release. Where is the post-merge window bundle, where are the later commits? Lots of patches are sitting rotting in patchwork...
