Re: linux rdma 3.14 merge plans
On Wed, Jan 22, 2014 at 3:43 AM, Roland Dreier wrote: [...] > I'd really rather spend my time on something actually useful like cleaning up > softroce. Hi Roland, Were you referring to the code that was posted here few years ago, or you're working on something new? two comments 1. make sure it uses IP based GIDs... 2. I think we don't want to have a 3rd SW implementation in the kernel of IB transport, (1=ipath, 2=qib) right? so the relevant qib code could/should be refactored into a module which implements the IB transport. thoughts? Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: linux rdma 3.14 merge plans
Thanks Roland, My response is inline below. -Regards Devesh -Original Message- From: rol...@purestorage.com [mailto:rol...@purestorage.com] On Behalf Of Roland Dreier Sent: Saturday, March 08, 2014 1:02 AM To: Devesh Sharma Cc: Nicholas A. Bellinger; Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; target-devel; Sagi Grimberg; linux-kernel Subject: Re: linux rdma 3.14 merge plans Sure, no problem. Do you have a git tree with the latest versions of all the changes you want for 3.15 in a branch? That would be helpful as I catch up on applying things, so that I don't miss anything. [DS]: Yes, I do have a cloned copy of your git tree migrated to for-next branch, with all the patches we have submitted to linux-rdma (patch series: [PATCH for-next 00/17] ocrdma driver code sync-up). However, it is on my local server. In case of urgency (and if OFED EWG permits me) I can set it up on benay dot openfabrics dot org, for now and give a link to pull. If you don't have one, taking a little time to set one up on github or wherever would be nice. You can base your set of changes on Linus's latest tree. [DS]: Currently we do not have such tree. I will have to discuss within my organization on how to go about this request. Thanks! Roland On Thu, Mar 6, 2014 at 9:07 PM, Devesh Sharma wrote: > Hi Roland, > > Is it okay to send next series of patches even if previous series is not > accepted yet in your tree? Off-course I will cut patches on top of previous > series of patches. > > -Regards > Devesh > > -Original Message- > From: linux-rdma-ow...@vger.kernel.org > [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Nicholas A. > Bellinger > Sent: Thursday, March 06, 2014 12:34 AM > To: Roland Dreier > Cc: Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; > target-devel; Sagi Grimberg; linux-kernel > Subject: Re: linux rdma 3.14 merge plans > > On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote: >> On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger >> wrote: >> > That all said, do you have an objection wrt taking this bits >> > through target-pending..? Given the dependencies involved, that >> > would seem the most logical path to take. >> >> Perhaps not surprisingly, I would prefer to get a chance to review a >> major change to the core RDMA midlayer rather than having you merge >> it through your tree. So yes I do object. Please give me a chance >> to review and merge it. I am working on that this week. >> > > Great. We'll be looking for a response by the end of the week. > > Otherwise if you end up not having time, we'd still like to move forward for > v3.15 given the amount of review the series has already gotten on the list. > > Thank you, > > --nab > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" > in the body of a message to majord...@vger.kernel.org More majordomo > info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
Sure, no problem. Do you have a git tree with the latest versions of all the changes you want for 3.15 in a branch? That would be helpful as I catch up on applying things, so that I don't miss anything. If you don't have one, taking a little time to set one up on github or wherever would be nice. You can base your set of changes on Linus's latest tree. Thanks! Roland On Thu, Mar 6, 2014 at 9:07 PM, Devesh Sharma wrote: > Hi Roland, > > Is it okay to send next series of patches even if previous series is not > accepted yet in your tree? Off-course I will cut patches on top of previous > series of patches. > > -Regards > Devesh > > -Original Message- > From: linux-rdma-ow...@vger.kernel.org > [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Nicholas A. Bellinger > Sent: Thursday, March 06, 2014 12:34 AM > To: Roland Dreier > Cc: Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; target-devel; > Sagi Grimberg; linux-kernel > Subject: Re: linux rdma 3.14 merge plans > > On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote: >> On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger >> wrote: >> > That all said, do you have an objection wrt taking this bits through >> > target-pending..? Given the dependencies involved, that would seem >> > the most logical path to take. >> >> Perhaps not surprisingly, I would prefer to get a chance to review a >> major change to the core RDMA midlayer rather than having you merge it >> through your tree. So yes I do object. Please give me a chance to >> review and merge it. I am working on that this week. >> > > Great. We'll be looking for a response by the end of the week. > > Otherwise if you end up not having time, we'd still like to move forward for > v3.15 given the amount of review the series has already gotten on the list. > > Thank you, > > --nab > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the > body of a message to majord...@vger.kernel.org More majordomo info at > http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: linux rdma 3.14 merge plans
Hi Roland, Is it okay to send next series of patches even if previous series is not accepted yet in your tree? Off-course I will cut patches on top of previous series of patches. -Regards Devesh -Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Nicholas A. Bellinger Sent: Thursday, March 06, 2014 12:34 AM To: Roland Dreier Cc: Or Gerlitz; Hefty Sean; linux-rdma; Martin K. Petersen; target-devel; Sagi Grimberg; linux-kernel Subject: Re: linux rdma 3.14 merge plans On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote: > On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger > wrote: > > That all said, do you have an objection wrt taking this bits through > > target-pending..? Given the dependencies involved, that would seem > > the most logical path to take. > > Perhaps not surprisingly, I would prefer to get a chance to review a > major change to the core RDMA midlayer rather than having you merge it > through your tree. So yes I do object. Please give me a chance to > review and merge it. I am working on that this week. > Great. We'll be looking for a response by the end of the week. Otherwise if you end up not having time, we'd still like to move forward for v3.15 given the amount of review the series has already gotten on the list. Thank you, --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html N�r��yb�X��ǧv�^�){.n�+{��ٚ�{ay�ʇڙ�,j��f���h���z��w��� ���j:+v���w�j�mzZ+�ݢj"��!�i
Re: linux rdma 3.14 merge plans
On Wed, 2014-03-05 at 07:18 -0800, Roland Dreier wrote: > On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger > wrote: > > That all said, do you have an objection wrt taking this bits through > > target-pending..? Given the dependencies involved, that would seem the > > most logical path to take. > > Perhaps not surprisingly, I would prefer to get a chance to review a > major change to the core RDMA midlayer rather than having you merge it > through your tree. So yes I do object. Please give me a chance to > review and merge it. I am working on that this week. > Great. We'll be looking for a response by the end of the week. Otherwise if you end up not having time, we'd still like to move forward for v3.15 given the amount of review the series has already gotten on the list. Thank you, --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 05/03/2014 17:18, Roland Dreier wrote: On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger wrote: >That all said, do you have an objection wrt taking this bits through >target-pending..? Given the dependencies involved, that would seem the >most logical path to take. Perhaps not surprisingly, I would prefer to get a chance to review a major change to the core RDMA midlayer rather than having you merge it through your tree. So yes I do object. Please give me a chance to review and merge it. I am working on that this week. Hi Roland, we're very happy to hear that!! As you know, we are soon marking whole five months!! (patches posted Oct 15th/2013) of sitting and waiting to your feedback, lost few upstream releases on the way. Let's get this into the works. Or. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Wed, Mar 5, 2014 at 1:54 AM, Nicholas A. Bellinger wrote: > That all said, do you have an objection wrt taking this bits through > target-pending..? Given the dependencies involved, that would seem the > most logical path to take. Perhaps not surprisingly, I would prefer to get a chance to review a major change to the core RDMA midlayer rather than having you merge it through your tree. So yes I do object. Please give me a chance to review and merge it. I am working on that this week. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, 2014-02-06 at 16:04 -0800, Roland Dreier wrote: > On Thu, Feb 6, 2014 at 4:02 PM, Nicholas A. Bellinger > wrote: > > Can you give us an estimate of when you'll have some time to give > > feedback on the outstanding patches..? > > I hope to get to it in the next few weeks. > > - R. Hi Roland, We'd very much like to move forward with the verbs + mlx5 + LIO target PI related patches for v3.15 code. Given the verbs + mlx5 changes have undergone ~5 months of review at this point, I'd like to go ahead and get them in target-pending/for-next ASAP to continue making forward progress towards a mainline merge. I'm happy with the state of the iser-target related changes, and Sagi & Co are making good progress on the iser-initiator related patches with Mike. That all said, do you have an objection wrt taking this bits through target-pending..? Given the dependencies involved, that would seem the most logical path to take. --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Fri, Feb 7, 2014 at 2:04 AM, Roland Dreier wrote: > On Thu, Feb 6, 2014 at 4:02 PM, Nicholas A. Bellinger > wrote: >> Can you give us an estimate of when you'll have some time to give >> feedback on the outstanding patches..? > > I hope to get to it in the next few weeks. Hi Roland, So days, weeks and months are passing by and we still didn't get any real feedback from you on the patches which now make a complete story: verbs, driver, target and initiator nor on the detailed response/s sent to you as replies to the few on the surface quick questions and doubts you raised. 3.15 is coming soon and there's no reason to miss it just for that lack of feedback as happened for 3.13 and 3.14 -- are you picking on this or maybe prefer that Nic will push it upstream through his tree? all the patches are now on the rdma-dif of the target-pending tree, please let us know. > > - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, Feb 6, 2014 at 4:02 PM, Nicholas A. Bellinger wrote: > Can you give us an estimate of when you'll have some time to give > feedback on the outstanding patches..? I hope to get to it in the next few weeks. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
Hi Roland, On Tue, 2014-01-21 at 23:27 -0800, Nicholas A. Bellinger wrote: > Roland & Co, > > On Tue, 2014-01-21 at 16:43 -0800, Roland Dreier wrote: > > On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz wrote: > > > Roland, ping! the signature patches were posted > three months ago. We > > > deserve a response from the maintainer that goes beyond "I need to > > > think on that". > > > > > > Responsiveness was stated by Linus to be the #1 requirement from > > > kernel maintainers. > > > > Or, I'm not sure what response you're after from me. Linus has also > > said that maintainers should say "no" a lot more > > (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I > > won't merge this patch set, since it adds a bunch of complexity to > > support a feature no one really cares about." Is that it? > > The patch set proposed by Sagi + Or is modest in terms of LOC to core IB > code, and includes mostly mlx5 specific driver changes that enables HW > offloads. > > > (And yes I > > am skeptical about this stuff — I work at an enterprise storage > > company and even here it's hard to find anyone who cares about > > DIF/DIX, especially offload features that stop it from being > > end-to-end) > > > > My understanding is most HBAs capable of T10 PI offload in DIX PASS + > VERIFY mode are already implementing DIX INSERT + STRIP modes in various > capacities to support legacy environments. > > Beyond the DIX INSERT + STRIP case for enterprise storage, the amount of > FC + SAS HBAs that already support T10 PI metadata is substantial. > > > I'm sure you're not expecting me to say, "Sure, I'll merge it without > > understanding the problem it's solving or how it's doing that," > > especially given the your recent history of pushing me to merge stuff > > like the IP-RoCE patches back when they broke the userspace ABI. > > With the merge window now upon us, there is a understandable reluctance > to merge new features. Given the amount of time the series has spent on > the list, it is however a good candidate to consider for an exception. > > Short of that, are you planning to accept the series for the next round > once the current merge window closes..? > > We'd really like to start enabling fabrics with these types of offloads > for v3.15. > Now with the initial DIF backend taraget support in place for v3.14-rc1 code, we'd like to move forward on iser-target related pieces for T10 PI. Can you give us an estimate of when you'll have some time to give feedback on the outstanding patches..? --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 30/01/2014 12:07, Bart Van Assche wrote: On 01/30/14 09:19, Or Gerlitz wrote: Thanks for narrowing this down, I see your point, however the solution I propose if to remove this copy altogether... for those rare cases where fast-registration can't be done -- in SRP I think the code goes to indirect mode under which there no copy (right?). As for iSER, we will add to our TODO enhancing the code to avoid using bounce-buffer, few designs might be possible, one way would be to create set of FMRS/Fast-reg element which are capable to map chunks which are < PAGE_SIZE, e.g in the order of single block, and hence can map what ever arbitrary set of blocks provided by the upper layer. I'm not sure how important this issue is on x86 systems since the page size on these systems is 4 KB and since many applications use an I/O size >= 4 KB. So if the I/O buffers are aligned on page boundaries the buffer memory can be registered as a single memory region without having to copy any data. However, on PPC systems, which have a page size of 64 KB, if e.g. a database issues a readv() or writev() call or if an I/O scheduler coalesces several small I/O requests into a single I/O request the data buffer of these I/O requests is discontiguous. I think it is important that data copying can be avoided for such I/O requests. We know that... and hence iser uses fast registration in chucks of 4KB even when the page size is 64KB -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 01/30/14 09:19, Or Gerlitz wrote: > On 29/01/2014 19:56, Bart Van Assche wrote: >> On 01/29/14 16:06, Sagi Grimberg wrote: >>> Didn't understand why should it matter where the copy is done >>> (iser/block)? >> In the Linux kernel community it is considered important to avoid code >> duplication. Hence the proposal to keep code that copies data buffers >> in the block layer core and to avoid that such functionality has to be >> reimplemented in every block driver or SCSI LLD. > > Thanks for narrowing this down, I see your point, however the solution I > propose if to remove this copy altogether... for those rare cases where > fast-registration can't be done -- in SRP I think the code goes to > indirect mode under which there no copy (right?). As for iSER, we will > add to our TODO enhancing the code to avoid using bounce-buffer, few > designs might be possible, one way would be to create set of > FMRS/Fast-reg element which are capable to map chunks which are < > PAGE_SIZE, e.g in the order of single block, and hence can map what ever > arbitrary set of blocks provided by the upper layer. I'm not sure how important this issue is on x86 systems since the page size on these systems is 4 KB and since many applications use an I/O size >= 4 KB. So if the I/O buffers are aligned on page boundaries the buffer memory can be registered as a single memory region without having to copy any data. However, on PPC systems, which have a page size of 64 KB, if e.g. a database issues a readv() or writev() call or if an I/O scheduler coalesces several small I/O requests into a single I/O request the data buffer of these I/O requests is discontiguous. I think it is important that data copying can be avoided for such I/O requests. By the way, is it documented somewhere what the alignment requirements are for FMR and FR requests ? As far as I can see the mlx4 driver requires page alignment for FMR requests but not for FR requests (see e.g. mlx4_check_fmr()). I haven't found a page aligment requirement in e.g. ehca_fmr_check_page_list(). Does this mean that alignment restrictions for FMR are HCA-dependent ? Would it be a good idea to add an attribute in e.g. ib_device_attr that allows ULP drivers to retrieve FMR alignment requirements in a generic way ? Thanks, Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 29/01/2014 19:56, Bart Van Assche wrote: On 01/29/14 16:06, Sagi Grimberg wrote: Didn't understand why should it matter where the copy is done (iser/block)? In the Linux kernel community it is considered important to avoid code duplication. Hence the proposal to keep code that copies data buffers in the block layer core and to avoid that such functionality has to be reimplemented in every block driver or SCSI LLD. Hi Bart, Thanks for narrowing this down, I see your point, however the solution I propose if to remove this copy altogether... for those rare cases where fast-registration can't be done -- in SRP I think the code goes to indirect mode under which there no copy (right?). As for iSER, we will add to our TODO enhancing the code to avoid using bounce-buffer, few designs might be possible, one way would be to create set of FMRS/Fast-reg element which are capable to map chunks which are < PAGE_SIZE, e.g in the order of single block, and hence can map what ever arbitrary set of blocks provided by the upper layer. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 29/01/2014 17:06, Sagi Grimberg wrote: Non-contiguous buffers are a well known problem/challenge for RDMA transports, each handles them differently: iSER uses bounce buffers, SRP registers multiple contiguous regions. To be precise, we do it only on very special cases, normally the payload **each** SCSI command of size > PAGE_SIZE is scattered between random set of pages allocated by the kernel buffer cache and this set is mapped through fast-reg or fmr mechanism to one RDMA virtual address. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 1/29/2014 4:13 PM, Bart Van Assche wrote: On 01/28/14 22:02, Or Gerlitz wrote: Roland, ping! the signature patches were posted > three months ago. I have a question about RDMA and T10-DIF support. The Linux block layer, the Linux SCSI core, Linux filesystems and block drivers all support discontiguous buffers. These are buffers that cannot be mapped onto a single contiguous range of virtual memory addresses. I think the RDMA FMR and FR memory registration mechanisms only support mapping a range of addresses that can be mapped onto a contiguous region of virtual addresses. This means that either multiple RDMA memory regions are needed to map a discontiguous buffer or that the buffer contents has to be copied before associating it with a memory region. What I understood from the mlx5 T10-DIF patch series is that the API introduced in that patch series is such that only a single memory region per data transfer operation is supported. This made me wonder how to support T10-DIF for discontiguous buffers. Hey Bart, Thanks for the observation, You are correct, the T10-PI RDMA offload API requires a single memory region that describes the data and a single memory region that describes the protection information. Non-contiguous buffers are a well known problem/challenge for RDMA transports, each handles them differently: iSER uses bounce buffers, SRP registers multiple contiguous regions. The T10-PI offload API does not impose any other constraint on this situation, each ULP will act the same: iSER will copy to contiguous bounce buffers and SRP will register multiple "Signature" memory regions (each for a contiguous chunk of data) and send the rkeys to the target. Would one of the options below perhaps be an appropriate solution ? - Add a flag in struct request_queue that tells the block layer to use a bounce buffer (data copying) for I/O requests with a discontiguous data buffer. The block layer already supports copying data buffers for which e.g. DMA alignment requirements are not met. Rework patch "IB/iser: Introduce fast memory registration model" (commit 5587856c9659ac2d6ab201141aa8a5c2ff3be4cd) such that it takes advantage of that new flag. See also blk_rq_map_user_iov() for an example of how the block layer decides when copying a data buffer is necessary. Didn't understand why should it matter where the copy is done (iser/block)? - Modify the patches that add T10-DIF such that discontiguous buffers are supported. I thought about adding some kind of logic to T10-PI RDMA API to try and solve the alignment issue at the same time, but the fact is that the two are unrelated, and I figured it is not such a good idea. Non-contiguous memory registration is separate problem (hint: under development) and should be addressed separately, T10-PI RDMA offload should play well with it. Sorry that I had not realized this before. Bart. Hops this helps, Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Wed, Jan 22, 2014, Sagi Grimberg wrote: > On 1/22/2014 2:43 AM, Roland Dreier wrote: >> On Tue, Jan 21, 2014, Or Gerlitz wrote: >>> Roland, ping! the signature patches were posted > three months ago. We >>> deserve a response from the maintainer that goes beyond "I need to >>> think on that". Responsiveness was stated by Linus to be the #1 requirement >>> from kernel maintainers. > Hi Roland, I'll try to respond here. removing LKML and adding Linux-scsi. Sorry, it seems we will not getting responses unless coping LKML, so lets do that again -- Roland, below is the detailed response Sagi wrote you following your "adds a bunch of complexity to support a feature no one really cares about" comment, can we get you to respond on that? >> Or, I'm not sure what response you're after from me. Linus has also >> said that maintainers should say "no" a lot more >> (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I >> won't merge this patch set, since it adds a bunch of complexity to >> support a feature no one really cares about." > 1. I disagree about no-one cares about DIF/DIX. We are witnessing growing > interests in this especially for RDMA. > 2. We put a lot of efforts to avoid complexity here and plug-in as simple as > possible. > Application that will choose to use DIF will implement only 3 steps: > a. allocate signature enabled MR. > b. register signature enabled MR with DIF attributes (via post_send) and > then do RDMA. > c. check MR status after transaction is completed (_lightweight_ verb that > can be called from interrupt context). >> Is that it? (And yes I am skeptical about this stuff -- I work at an >> enterprise >> storage company and even here it's hard to find anyone who cares about >> DIF/DIX, especially offload features that stop it from being end-to-end) > 1. RDMA verbs are _NOT_ stopping DIF from being end-to-end. > OS (or SCSI in our specific case) passes LLD 2 scatterlists: data {block1, > block2, block3,...}, and protection {DIF1, DIF2, DIF3}. > LLD is required to verify the data integrity (block guards) and to > interleave over the wire {block1, DIF1, block2, DIF2}. > You must support that in HW, you rather iSER/SRP will use giant copy's to > interleave by itself? or in case OS asked LLD > to INSERT DIF iSER/SRP will compute CRC for each data-block? RDMA storage > ULPs are transports - they should have no business with > data processing. > 2. HW DIF offload also gives you protection across the PCI. the > data-validation is done (hopefully offloaded) also > when data+protection are written to the back-end device. end-to-end is > preserved. > 3. SAS & FC have T10-PI offload. This is just adding RDMA into the game. > With this set of verbs iSER, SRP, FCoE Initiators and targets will be able > to support T10-PI. >> I'm sure you're not expecting me to say, "Sure, I'll merge it without >> understanding the problem it's solving > Problem: T10-PI offload support for RDMA based initiators. Supporting > end-to-end data integrity while sustaining high RDMA performance. >> or how it's doing that," > How it's doing that: > - We introduce a new type of memory region that posses protection attributes > suited for data integrity offload. > - We Introduce a new fast registration method that can bind all the relevant > info for verify/generate of protection information: > * describe if/how to interleave data with protection. > * describe what method of data integrity is used (DIF type X, CRC, XOR...) > and the seeds that HW should start calculation from. > * describe how to verify the data. > - We Introduce a new lightweight check of the data-integrity status to check > if there were any integrity errors and get information on them. > Note: We made MR allocation routine generic enough to lay a framework to > unite all MR allocation > methods (get_dma_mr, alloc_fast_reg_mr, reg_phys, reg_user_mr, fmrs, and > probably more in the future...). > We defined ib_create_mr that can actually get mr_init_attr which can be > easily extended as opposed to the specific calls exists today. > So I would say this even reduces complexity. > Hope this helps, > Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 1/22/2014 2:43 AM, Roland Dreier wrote: On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz wrote: Roland, ping! the signature patches were posted > three months ago. We deserve a response from the maintainer that goes beyond "I need to think on that". Responsiveness was stated by Linus to be the #1 requirement from kernel maintainers. Hi Roland, I'll try to respond here. removing LKML and adding Linux-scsi. Or, I'm not sure what response you're after from me. Linus has also said that maintainers should say "no" a lot more (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I won't merge this patch set, since it adds a bunch of complexity to support a feature no one really cares about." 1. I disagree about no-one cares about DIF/DIX. We are witnessing growing interests in this especially for RDMA. 2. We put a lot of efforts to avoid complexity here and plug-in as simple as possible. Application that will choose to use DIF will implement only 3 steps: a. allocate signature enabled MR. b. register signature enabled MR with DIF attributes (via post_send) and then do RDMA. c. check MR status after transaction is completed (_lightweight_ verb that can be called from interrupt context). Is that it? (And yes I am skeptical about this stuff — I work at an enterprise storage company and even here it's hard to find anyone who cares about DIF/DIX, especially offload features that stop it from being end-to-end) 1. RDMA verbs are _NOT_ stopping DIF from being end-to-end. OS (or SCSI in our specific case) passes LLD 2 scatterlists: data {block1, block2, block3,...}, and protection {DIF1, DIF2, DIF3}. LLD is required to verify the data integrity (block guards) and to interleave over the wire {block1, DIF1, block2, DIF2}. You must support that in HW, you rather iSER/SRP will use giant copy's to interleave by itself? or in case OS asked LLD to INSERT DIF iSER/SRP will compute CRC for each data-block? RDMA storage ULPs are transports - they should have no business with data processing. 2. HW DIF offload also gives you protection across the PCI. the data-validation is done (hopefully offloaded) also when data+protection are written to the back-end device. end-to-end is preserved. 3. SAS & FC have T10-PI offload. This is just adding RDMA into the game. With this set of verbs iSER, SRP, FCoE Initiators and targets will be able to support T10-PI. I'm sure you're not expecting me to say, "Sure, I'll merge it without understanding the problem it's solving Problem: T10-PI offload support for RDMA based initiators. Supporting end-to-end data integrity while sustaining high RDMA performance. or how it's doing that," How it's doing that: - We introduce a new type of memory region that posses protection attributes suited for data integrity offload. - We Introduce a new fast registration method that can bind all the relevant info for verify/generate of protection information: * describe if/how to interleave data with protection. * describe what method of data integrity is used (DIF type X, CRC, XOR...) and the seeds that HW should start calculation from. * describe how to verify the data. - We Introduce a new lightweight check of the data-integrity status to check if there were any integrity errors and get information on them. Note: We made MR allocation routine generic enough to lay a framework to unite all MR allocation methods (get_dma_mr, alloc_fast_reg_mr, reg_phys, reg_user_mr, fmrs, and probably more in the future...). We defined ib_create_mr that can actually get mr_init_attr which can be easily extended as opposed to the specific calls exists today. So I would say this even reduces complexity. Hope this helps, Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
Roland & Co, On Tue, 2014-01-21 at 16:43 -0800, Roland Dreier wrote: > On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz wrote: > > Roland, ping! the signature patches were posted > three months ago. We > > deserve a response from the maintainer that goes beyond "I need to > > think on that". > > > > Responsiveness was stated by Linus to be the #1 requirement from > > kernel maintainers. > > Or, I'm not sure what response you're after from me. Linus has also > said that maintainers should say "no" a lot more > (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I > won't merge this patch set, since it adds a bunch of complexity to > support a feature no one really cares about." Is that it? The patch set proposed by Sagi + Or is modest in terms of LOC to core IB code, and includes mostly mlx5 specific driver changes that enables HW offloads. > (And yes I > am skeptical about this stuff — I work at an enterprise storage > company and even here it's hard to find anyone who cares about > DIF/DIX, especially offload features that stop it from being > end-to-end) > My understanding is most HBAs capable of T10 PI offload in DIX PASS + VERIFY mode are already implementing DIX INSERT + STRIP modes in various capacities to support legacy environments. Beyond the DIX INSERT + STRIP case for enterprise storage, the amount of FC + SAS HBAs that already support T10 PI metadata is substantial. > I'm sure you're not expecting me to say, "Sure, I'll merge it without > understanding the problem it's solving or how it's doing that," > especially given the your recent history of pushing me to merge stuff > like the IP-RoCE patches back when they broke the userspace ABI. With the merge window now upon us, there is a understandable reluctance to merge new features. Given the amount of time the series has spent on the list, it is however a good candidate to consider for an exception. Short of that, are you planning to accept the series for the next round once the current merge window closes..? We'd really like to start enabling fabrics with these types of offloads for v3.15. > I'd really rather spend my time on something actually useful like > cleaning up softroce. > +1 for softroce + T10 PI support! --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Wed, Jan 22, 2014 at 2:43 AM, Roland Dreier wrote: > On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz wrote: >> Roland, ping! the signature patches were posted > three months ago. We >> deserve a response from the maintainer that goes beyond "I need to >> think on that". >> >> Responsiveness was stated by Linus to be the #1 requirement from >> kernel maintainers. > > Or, I'm not sure what response you're after from me. Roland, what I am after is a r-e-s-p-o-n-s-e from you, and let it contain what ever justified and/or unjustified mud as below. We posted the V0 series on Oct 15 2013 and since that time not a word from you, except for an "I need to think on that" comment last week after we nudged million times. You can't leave us clueless in the air for whole three months without any concrete or unconcrete comment. There's no way to carry kernel development like that. I am old enough to hear and face "no" and "wTF is this" or "yTF you do it this way" etc etc, this happened few times with e.g with networking patches we sent and we either improved things or did them differently or whatever needed to be done. There's no way on earth to face plain ignoring of your work, and this is what happens here. I had no way to get your below response except for going to LKML, why? > Linus has also said that maintainers should say "no" a lot more > (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I > won't merge this patch set, since it adds a bunch of complexity to > support a feature no one really cares about." Is that it? (And yes I > am skeptical about this stuff — I work at an enterprise storage > company and even here it's hard to find anyone who cares about > DIF/DIX, especially offload features that stop it from being > end-to-end) > > I'm sure you're not expecting me to say, "Sure, I'll merge it without > understanding the problem it's solving or how it's doing that," > especially given the your recent history of pushing me to merge stuff > like the IP-RoCE patches back when they broke the userspace ABI. > > I'd really rather spend my time on something actually useful like > cleaning up softroce. > > - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Tue, Jan 21, 2014 at 2:00 PM, Or Gerlitz wrote: > Roland, ping! the signature patches were posted > three months ago. We > deserve a response from the maintainer that goes beyond "I need to > think on that". > > Responsiveness was stated by Linus to be the #1 requirement from > kernel maintainers. Or, I'm not sure what response you're after from me. Linus has also said that maintainers should say "no" a lot more (http://lwn.net/Articles/571995/) so maybe you want me to say, "No, I won't merge this patch set, since it adds a bunch of complexity to support a feature no one really cares about." Is that it? (And yes I am skeptical about this stuff — I work at an enterprise storage company and even here it's hard to find anyone who cares about DIF/DIX, especially offload features that stop it from being end-to-end) I'm sure you're not expecting me to say, "Sure, I'll merge it without understanding the problem it's solving or how it's doing that," especially given the your recent history of pushing me to merge stuff like the IP-RoCE patches back when they broke the userspace ABI. I'd really rather spend my time on something actually useful like cleaning up softroce. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Mon, Jan 20, 2014, Or Gerlitz wrote: > On Sun, Jan 19, 2014, sagi grimberg wrote: >> Thanks Nic, let me elaborate on this, [...] >> Hope this helps, > Hi Roland, with Nic's && Sagi's answers @ hand, were your questions resolved? Roland, ping! the signature patches were posted > three months ago. We deserve a response from the maintainer that goes beyond "I need to think on that". Responsiveness was stated by Linus to be the #1 requirement from kernel maintainers. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Sun, Jan 19, 2014 at 1:20 PM, sagi grimberg wrote: > Thanks Nic, let me elaborate on this, > > It is true that T10-PI aims for end-to-end data-integrity, the verbs API > offer HW offload for protection > information processing which is VERY expensive for CPU computation (CRC > verify for each block). T10-PI > is intended to be offloaded, both on the backend devices which supports this > feature and for fabric > transports (such as Qlogic/Emulex FC drivers), Verbs API just adds RDMA to > the game... > > T10-PI specifies protection information scheme in a way that over the wire > each protection interval is > followed by 8 bytes of data-integrity (interleaved). In the memory on the > other hand, the data and > protection block-guards may lie in separated buffers (for example that is the > preferred approach in block > and SCSI layers). > > Newly introduced REG_SIG_MR work request allows to (fast) register a > "signature" memory key which > incorporates the protection method and pattern used: > 1. Where is the data? (data sge) > 2. Where is the protection block-guards? (protection sge) > 3. What are the signature attributes? (T10-PI method, crc/reftag/apptag > seeds, verify mask, memory pattern, wire pattern) > When doing the actual data-transfer the HCA will enforce T10-PI scheme (See > my cover-letter for a more detailed explanation). > > If you take a look in SCSI implementation you will see that SCSI signals the > transport of protection attributes in > scsi_cmnd (prot SG-list, protection type, guard type, protection operation). > Verbs API allows full offload of all > T10-PI operations: > 1. INSERT - HCA computes/generates data-integrity block-guards and writes > them according to the specified > pattern (interleaved with the data or in a separated buffer). > 2. STRIP (and VERIFY) - HCA verifies incoming data-integrity block-guards and > strip them from the data stream. > 3. PASS (and VERIFY) - HCA verifies incoming data-integrity block-guards and > passes them forward according to > the specified pattern (interleaved/separated). > > In addition, Verbs API can be easily extended to support other data-integrity > methods (XOR-32, CRC-32, etc...) > so that an application interested in data-integrity has signature verbs in > its tool-box. This is why we use "Signature" > notation and refer to T10-PI as a specific signature method. > > Hope this helps, Hi Roland, So with Nic's && Sagi's answers @ hand, were your questions resolved? Also, Nic put the V4 patches in a branch on his tree for 0-day testing and we had one hit from Dan Carpenter which Sagi addressed with incremental fix he sent you. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 1/19/2014 5:42 AM, Nicholas A. Bellinger wrote: On Sat, 2014-01-18 at 13:42 -0800, Roland Dreier wrote: On Thu, Jan 16, 2014 at 1:14 PM, Nicholas A. Bellinger wrote: I've reviewed the API from the perspective of what's required for implementing protection support in iser, and currently don't have any recommendations or objections beyond what has been proposed by Sagi & Co in PATCH-v4 code. I guess I'm a little confused about why we need verbs support for this to implement DIF/DIX in iser. Isn't the whole point of protection to have end-to-end checksums, rather than having checksums computed by the transport after there's a chance for corruption? So to my knowledge, there are three target side DIX HBA modes of operation: - TARGET PASS: Fabric + Backend support PI - TARGET INSERT: Fabric does not support PI, backend supports PI - TARGET STRIP: Fabric supports PI, backend does not support PI The scenario your thinking about above is the 'TARGET INSERT' case, where the initiator does not generate PI, but the backend device on the target side expects PI, so the target fabric ends up generating PI on incoming WRITEs, and verifying + striping PI on outgoing READs. The scenario for 'TARGET STRIP' is when the initiator generates PI but the backend device does not support/process PI, so the target verifies + strips PI on incoming WRITESs, and inserts PI on outgoing READs. Your correct that both of these modes don't provide true end-to-end protection, and my understanding is that they are provided as a way to accommodate existing fabrics + backend devices where PI is not supported all the way through the stack. The 'TARGET PASS' is the scenario that provides true end-to-end guarantees, where for WRITEs PI is generated by the Host OS, verified + passed on the initiator side HBA, verified + passed on the target HBA, and verified + stored on the device backend. For READs, PI is retrieved from the backend device, verified + passed on the target HBA, verified + passed on the initiator HBA, and finally verified on the Host OS. So in the proposed RDMA VERBs changes these three modes of target DIX operation are supported. Also it's my understanding (Sagi & Co, please correct me), that the proposed changes are implemented to be independent of target/initiator mode DIX operation. Correct. Verbs API allow all supported protection operations without any peer dependencies. --nab Thanks Nic, let me elaborate on this, It is true that T10-PI aims for end-to-end data-integrity, the verbs API offer HW offload for protection information processing which is VERY expensive for CPU computation (CRC verify for each block). T10-PI is intended to be offloaded, both on the backend devices which supports this feature and for fabric transports (such as Qlogic/Emulex FC drivers), Verbs API just adds RDMA to the game... T10-PI specifies protection information scheme in a way that over the wire each protection interval is followed by 8 bytes of data-integrity (interleaved). In the memory on the other hand, the data and protection block-guards may lie in separated buffers (for example that is the preferred approach in block and SCSI layers). Newly introduced REG_SIG_MR work request allows to (fast) register a "signature" memory key which incorporates the protection method and pattern used: 1. Where is the data? (data sge) 2. Where is the protection block-guards? (protection sge) 3. What are the signature attributes? (T10-PI method, crc/reftag/apptag seeds, verify mask, memory pattern, wire pattern) When doing the actual data-transfer the HCA will enforce T10-PI scheme (See my cover-letter for a more detailed explanation). If you take a look in SCSI implementation you will see that SCSI signals the transport of protection attributes in scsi_cmnd (prot SG-list, protection type, guard type, protection operation). Verbs API allows full offload of all T10-PI operations: 1. INSERT - HCA computes/generates data-integrity block-guards and writes them according to the specified pattern (interleaved with the data or in a separated buffer). 2. STRIP (and VERIFY) - HCA verifies incoming data-integrity block-guards and strip them from the data stream. 3. PASS (and VERIFY) - HCA verifies incoming data-integrity block-guards and passes them forward according to the specified pattern (interleaved/separated). In addition, Verbs API can be easily extended to support other data-integrity methods (XOR-32, CRC-32, etc...) so that an application interested in data-integrity has signature verbs in its tool-box. This is why we use "Signature" notation and refer to T10-PI as a specific signature method. Hope this helps, Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Sat, 2014-01-18 at 13:42 -0800, Roland Dreier wrote: > On Thu, Jan 16, 2014 at 1:14 PM, Nicholas A. Bellinger > wrote: > > I've reviewed the API from the perspective of what's required for > > implementing protection support in iser, and currently don't have any > > recommendations or objections beyond what has been proposed by Sagi & Co > > in PATCH-v4 code. > > I guess I'm a little confused about why we need verbs support for this > to implement DIF/DIX in iser. Isn't the whole point of protection to > have end-to-end checksums, rather than having checksums computed by > the transport after there's a chance for corruption? > So to my knowledge, there are three target side DIX HBA modes of operation: - TARGET PASS: Fabric + Backend support PI - TARGET INSERT: Fabric does not support PI, backend supports PI - TARGET STRIP: Fabric supports PI, backend does not support PI The scenario your thinking about above is the 'TARGET INSERT' case, where the initiator does not generate PI, but the backend device on the target side expects PI, so the target fabric ends up generating PI on incoming WRITEs, and verifying + striping PI on outgoing READs. The scenario for 'TARGET STRIP' is when the initiator generates PI but the backend device does not support/process PI, so the target verifies + strips PI on incoming WRITESs, and inserts PI on outgoing READs. Your correct that both of these modes don't provide true end-to-end protection, and my understanding is that they are provided as a way to accommodate existing fabrics + backend devices where PI is not supported all the way through the stack. The 'TARGET PASS' is the scenario that provides true end-to-end guarantees, where for WRITEs PI is generated by the Host OS, verified + passed on the initiator side HBA, verified + passed on the target HBA, and verified + stored on the device backend. For READs, PI is retrieved from the backend device, verified + passed on the target HBA, verified + passed on the initiator HBA, and finally verified on the Host OS. So in the proposed RDMA VERBs changes these three modes of target DIX operation are supported. Also it's my understanding (Sagi & Co, please correct me), that the proposed changes are implemented to be independent of target/initiator mode DIX operation. --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, Jan 16, 2014 at 1:14 PM, Nicholas A. Bellinger wrote: > I've reviewed the API from the perspective of what's required for > implementing protection support in iser, and currently don't have any > recommendations or objections beyond what has been proposed by Sagi & Co > in PATCH-v4 code. I guess I'm a little confused about why we need verbs support for this to implement DIF/DIX in iser. Isn't the whole point of protection to have end-to-end checksums, rather than having checksums computed by the transport after there's a chance for corruption? - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
> "nab" == Nicholas A Bellinger writes: nab> So there is not a interface in SCSI land for interacting (directly) nab> with hardware protection support, as it's primarily just telling nab> SCSI what protection modes are supported while the rest is nab> implemented in vendor specific firmware interfaces. (CC'ing MKP) I've worked on getting my DIX1.1 changes ready for submission the last couple of days. That'll take the magic out of firmware and into the SCSI stack. I'll get those patches out early next week. They are fairly trivial but do require some changes to mptNsas, lpfc and qla2xxx. -- Martin K. Petersen Oracle Linux Engineering -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, 2014-01-16 at 23:42 +0200, Or Gerlitz wrote: > On Thu, Jan 16, 2014 at 11:37 PM, Greg Kroah-Hartman > wrote: > > On Thu, Jan 16, 2014 at 01:14:12PM -0800, Nicholas A. Bellinger wrote: > >> On Thu, 2014-01-16 at 12:13 -0800, Roland Dreier wrote: > >> > I haven't really had a chance to look at the new API and decide if it > >> > makes sense as the way to expose these features. Have you looked at > >> > the new kernel API? Any thoughts? > >> > >> I've reviewed the API from the perspective of what's required for > >> implementing protection support in iser, and currently don't have any > >> recommendations or objections beyond what has been proposed by Sagi & Co > >> in PATCH-v4 code. > >> > >> > How does it compare with how other subsystems have exposed protection > >> > info? > >> > > >> > >> So there is not a interface in SCSI land for interacting (directly) with > >> hardware protection support, as it's primarily just telling SCSI what > >> protection modes are supported while the rest is implemented in vendor > >> specific firmware interfaces. (CC'ing MKP) > >> > >> > Right now I'm dealing with fixing the fallout from picking up the "IP > >> > addressing for IBoE" and other patch sets that were supposedly "ready > >> > to merge for a long time" so I'm not sure I'll get to the protection > >> > changes in time for 3.14. > >> > > >> > >> Pretty please for v3.14..? > > > > It's _really_ late in the development cycle for new stuff for 3.14. My > > trees have been closed for almost a week now, for major stuff, and for > > anything else for a few days now. It would be good if you could get > > your patchsets into the 0-day testing bot earlier to shake out any build > > issues that might happen with them, please work with that developer to > > help make the merging of the code easier. > > Greg, > > The T10 patches were posted whole three months ago (V0 Oct 15th). I > don't see why another cycle should get lost just because there was no > maintainer feedback on them throughout this whole period. There's > enough time for us to fix things that will show up in the testing bot > before Roland sends his pull request over the two weeks merge window. > So as Greg noted, it's still useful to get this into the 0-day build testing now. That said, pushing the -v4 series into target-pending/rdma-dif, and Fengguang's scripts should be picking it up shortly. --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, Jan 16, 2014 at 11:42:19PM +0200, Or Gerlitz wrote: > On Thu, Jan 16, 2014 at 11:37 PM, Greg Kroah-Hartman > wrote: > > On Thu, Jan 16, 2014 at 01:14:12PM -0800, Nicholas A. Bellinger wrote: > >> On Thu, 2014-01-16 at 12:13 -0800, Roland Dreier wrote: > >> > > On Mon, 2014-01-13 at 12:32 -0800, Nicholas A. Bellinger wrote: > >> > > > Hi Roland & Co, > >> > > > Just curious if your planning to take a look at this series soon for > >> > > > v3.14 code..? > >> > > > > >> > > > As you might imagine, I'd like to see it merged this round in order > >> > > > to > >> > > > move forward on iser-target protection for v3.15. > >> > > > > >> > > > Are there any specific issues that you'd like to see addressed for an > >> > > > initial merge..? > >> > > >> > I haven't really had a chance to look at the new API and decide if it > >> > makes sense as the way to expose these features. Have you looked at > >> > the new kernel API? Any thoughts? > >> > >> I've reviewed the API from the perspective of what's required for > >> implementing protection support in iser, and currently don't have any > >> recommendations or objections beyond what has been proposed by Sagi & Co > >> in PATCH-v4 code. > >> > >> > How does it compare with how other subsystems have exposed protection > >> > info? > >> > > >> > >> So there is not a interface in SCSI land for interacting (directly) with > >> hardware protection support, as it's primarily just telling SCSI what > >> protection modes are supported while the rest is implemented in vendor > >> specific firmware interfaces. (CC'ing MKP) > >> > >> > Right now I'm dealing with fixing the fallout from picking up the "IP > >> > addressing for IBoE" and other patch sets that were supposedly "ready > >> > to merge for a long time" so I'm not sure I'll get to the protection > >> > changes in time for 3.14. > >> > > >> > >> Pretty please for v3.14..? > > > > It's _really_ late in the development cycle for new stuff for 3.14. My > > trees have been closed for almost a week now, for major stuff, and for > > anything else for a few days now. It would be good if you could get > > your patchsets into the 0-day testing bot earlier to shake out any build > > issues that might happen with them, please work with that developer to > > help make the merging of the code easier. > > Greg, > > The T10 patches were posted whole three months ago (V0 Oct 15th). I > don't see why another cycle should get lost just because there was no > maintainer feedback on them throughout this whole period. There's > enough time for us to fix things that will show up in the testing bot > before Roland sends his pull request over the two weeks merge window. There's the rule that things need to be in linux-next for a week or so to shake things out before going to Linus, and that new things can't be added to a maintainer tree after Linus does a release before -rc1 is out. We can't break those rules unless you want to answer to the linux-next developer and Linus for it. Yes, I know your patches have been pending for a long time, this merge window was tough in that it covered two major holidays, your patches aren't the only thing that is going to miss 3.14 by a long-shot, there are lots of other developers who also didn't get stuff in due to maintainers taking long vacations. Which is fine, and is why we have 3 month release cycles, there's nothing wrong with waiting for 3.15 if there are build issues that are cropping up this late in the cycle. That's why I suggest you get your trees into the 0-day bot test systems, so that a maintainer has a semblance of confidence that the build will not break, and the "obvious" security issues are resolved, if they take your patches. thanks, greg k-h -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, Jan 16, 2014 at 11:37 PM, Greg Kroah-Hartman wrote: > On Thu, Jan 16, 2014 at 01:14:12PM -0800, Nicholas A. Bellinger wrote: >> On Thu, 2014-01-16 at 12:13 -0800, Roland Dreier wrote: >> > > On Mon, 2014-01-13 at 12:32 -0800, Nicholas A. Bellinger wrote: >> > > > Hi Roland & Co, >> > > > Just curious if your planning to take a look at this series soon for >> > > > v3.14 code..? >> > > > >> > > > As you might imagine, I'd like to see it merged this round in order to >> > > > move forward on iser-target protection for v3.15. >> > > > >> > > > Are there any specific issues that you'd like to see addressed for an >> > > > initial merge..? >> > >> > I haven't really had a chance to look at the new API and decide if it >> > makes sense as the way to expose these features. Have you looked at >> > the new kernel API? Any thoughts? >> >> I've reviewed the API from the perspective of what's required for >> implementing protection support in iser, and currently don't have any >> recommendations or objections beyond what has been proposed by Sagi & Co >> in PATCH-v4 code. >> >> > How does it compare with how other subsystems have exposed protection >> > info? >> > >> >> So there is not a interface in SCSI land for interacting (directly) with >> hardware protection support, as it's primarily just telling SCSI what >> protection modes are supported while the rest is implemented in vendor >> specific firmware interfaces. (CC'ing MKP) >> >> > Right now I'm dealing with fixing the fallout from picking up the "IP >> > addressing for IBoE" and other patch sets that were supposedly "ready >> > to merge for a long time" so I'm not sure I'll get to the protection >> > changes in time for 3.14. >> > >> >> Pretty please for v3.14..? > > It's _really_ late in the development cycle for new stuff for 3.14. My > trees have been closed for almost a week now, for major stuff, and for > anything else for a few days now. It would be good if you could get > your patchsets into the 0-day testing bot earlier to shake out any build > issues that might happen with them, please work with that developer to > help make the merging of the code easier. Greg, The T10 patches were posted whole three months ago (V0 Oct 15th). I don't see why another cycle should get lost just because there was no maintainer feedback on them throughout this whole period. There's enough time for us to fix things that will show up in the testing bot before Roland sends his pull request over the two weeks merge window. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, Jan 16, 2014 at 01:14:12PM -0800, Nicholas A. Bellinger wrote: > On Thu, 2014-01-16 at 12:13 -0800, Roland Dreier wrote: > > > > > On Mon, 2014-01-13 at 12:32 -0800, Nicholas A. Bellinger wrote: > > > > Hi Roland & Co, > > > > > > > > Just curious if your planning to take a look at this series soon for > > > > v3.14 code..? > > > > > > > > As you might imagine, I'd like to see it merged this round in order to > > > > move forward on iser-target protection for v3.15. > > > > > > > > Are there any specific issues that you'd like to see addressed for an > > > > initial merge..? > > > > I haven't really had a chance to look at the new API and decide if it > > makes sense as the way to expose these features. Have you looked at > > the new kernel API? Any thoughts? > > I've reviewed the API from the perspective of what's required for > implementing protection support in iser, and currently don't have any > recommendations or objections beyond what has been proposed by Sagi & Co > in PATCH-v4 code. > > > How does it compare with how other subsystems have exposed protection > > info? > > > > So there is not a interface in SCSI land for interacting (directly) with > hardware protection support, as it's primarily just telling SCSI what > protection modes are supported while the rest is implemented in vendor > specific firmware interfaces. (CC'ing MKP) > > > Right now I'm dealing with fixing the fallout from picking up the "IP > > addressing for IBoE" and other patch sets that were supposedly "ready > > to merge for a long time" so I'm not sure I'll get to the protection > > changes in time for 3.14. > > > > Pretty please for v3.14..? It's _really_ late in the development cycle for new stuff for 3.14. My trees have been closed for almost a week now, for major stuff, and for anything else for a few days now. It would be good if you could get your patchsets into the 0-day testing bot earlier to shake out any build issues that might happen with them, please work with that developer to help make the merging of the code easier. thanks, greg k-h -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Thu, 2014-01-16 at 12:13 -0800, Roland Dreier wrote: > > > On Mon, 2014-01-13 at 12:32 -0800, Nicholas A. Bellinger wrote: > > > Hi Roland & Co, > > > > > > Just curious if your planning to take a look at this series soon for > > > v3.14 code..? > > > > > > As you might imagine, I'd like to see it merged this round in order to > > > move forward on iser-target protection for v3.15. > > > > > > Are there any specific issues that you'd like to see addressed for an > > > initial merge..? > > I haven't really had a chance to look at the new API and decide if it > makes sense as the way to expose these features. Have you looked at > the new kernel API? Any thoughts? I've reviewed the API from the perspective of what's required for implementing protection support in iser, and currently don't have any recommendations or objections beyond what has been proposed by Sagi & Co in PATCH-v4 code. > How does it compare with how other subsystems have exposed protection > info? > So there is not a interface in SCSI land for interacting (directly) with hardware protection support, as it's primarily just telling SCSI what protection modes are supported while the rest is implemented in vendor specific firmware interfaces. (CC'ing MKP) > Right now I'm dealing with fixing the fallout from picking up the "IP > addressing for IBoE" and other patch sets that were supposedly "ready > to merge for a long time" so I'm not sure I'll get to the protection > changes in time for 3.14. > Pretty please for v3.14..? Sagi & Co are committed to supporting the changes, and are ready to do what's necessary for a merge. AFAICT the series has gone through enough list review over the last months without (serious) objections, so I'm not sure what else is necessary to move forward for a merge, aside from your input of course. :-) --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Mon, 2014-01-13 at 12:32 -0800, Nicholas A. Bellinger wrote: > Hi Roland & Co, > > Just curious if your planning to take a look at this series soon for > v3.14 code..? > > As you might imagine, I'd like to see it merged this round in order to > move forward on iser-target protection for v3.15. > > Are there any specific issues that you'd like to see addressed for an > initial merge..? Hi again Roland, Ping^2..? We're quite eager to see this code merged this round. Is there a specific issue on these patches preventing a v3.14 merge..? Thank you, --nab -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
Hi Roland & Co, Just curious if your planning to take a look at this series soon for v3.14 code..? As you might imagine, I'd like to see it merged this round in order to move forward on iser-target protection for v3.15. Are there any specific issues that you'd like to see addressed for an initial merge..? Thank you, --nab On Wed, 2014-01-08 at 11:37 +0200, Or Gerlitz wrote: > On 08/01/2014 02:51, Roland Dreier wrote: > > The data integrity stuff I'm not so sure about. Sean raised some I > think legitimate questions about whether all this should be added to > the verbs API and I want to see more discussion or at least have a > deep think about this myself before comitting. > > Well, re the deep think thing, we have problem here -- the patches were > posted three months ago, on Oct 15th 2013, and so far not a single bit > of input from you. Can you take a look on that as soon as possible and > let us know your thoughts? > > As for the questions posed by Sean -- Sagi, Tzahi and myself provided > detailed answers. Taking into account the small scale of the rdma > community, I don't see any other choice other than the guy wearing the > maintainer hat to step in and provide his say or follow up questions > that would cut things out or revive the discussion. > > Or. > > > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 08/01/2014 02:51, Roland Dreier wrote: The data integrity stuff I'm not so sure about. Sean raised some I think legitimate questions about whether all this should be added to the verbs API and I want to see more discussion or at least have a deep think about this myself before comitting. Well, re the deep think thing, we have problem here -- the patches were posted three months ago, on Oct 15th 2013, and so far not a single bit of input from you. Can you take a look on that as soon as possible and let us know your thoughts? As for the questions posed by Sean -- Sagi, Tzahi and myself provided detailed answers. Taking into account the small scale of the rdma community, I don't see any other choice other than the guy wearing the maintainer hat to step in and provide his say or follow up questions that would cut things out or revive the discussion. Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 08/01/2014 02:51, Roland Dreier wrote: I am definitely planning on merging the new IBoE IP addressing stuff, since we seem to have solved the ABI issues. The UD flow steering patches seem good and I will take a closer look soon. This sounds good, repeating myself, with 3.13-rc7 being out, 3.13 might be released Monday, so in case something need to be fixed, can you take this closer look asap so we have enough time for changes? also when you expect this merge to happen? -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On 1/8/2014 2:51 AM, Roland Dreier wrote: On Tue, Jan 7, 2014 at 1:02 PM, Or Gerlitz wrote: Currently there is single patch for 3.14 on your for-next branch, the usnic driver. With 3.13 being on rc7 and likely to be released next week, are you planning any other merges for 3.14? we have patches waiting for weeks and months without any comment from you. I am definitely planning on merging the new IBoE IP addressing stuff, since we seem to have solved the ABI issues. The UD flow steering patches seem good and I will take a closer look soon. And there are quite a few usnic patches still to pick up. I'm confident that will all make it. The data integrity stuff I'm not so sure about. Sean raised some I think legitimate questions about whether all this should be added to the verbs API and I want to see more discussion or at least have a deep think about this myself before comitting. Hey Roland, I don't think that Sean didn't question weather data-integrity support should or shouldn't be added to Verbs API (Sean correct me if I'm wrong), but rather the way it should be added. From our discussion on this, the only conflict that Sean and I had was weather the protection setup should "ride" on ib_post_send. Sean suggested a separate routine that would post on the SQ. I think that in the current framework where placing a fast-path operation is done via ib_post_send, we keep current implementation, and open a discussion if it is a good idea to migrate "non-send" work-requests out of ib_post_send (also fast-registration and memory-windows). Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: linux rdma 3.14 merge plans
On Tue, Jan 7, 2014 at 1:02 PM, Or Gerlitz wrote: > Currently there is single patch for 3.14 on your for-next branch, the > usnic driver. With 3.13 being on rc7 and likely to be released next > week, are you planning any other merges for 3.14? we have patches > waiting for weeks and months without any comment from you. I am definitely planning on merging the new IBoE IP addressing stuff, since we seem to have solved the ABI issues. The UD flow steering patches seem good and I will take a closer look soon. And there are quite a few usnic patches still to pick up. I'm confident that will all make it. The data integrity stuff I'm not so sure about. Sean raised some I think legitimate questions about whether all this should be added to the verbs API and I want to see more discussion or at least have a deep think about this myself before comitting. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html