RE: [RFC PATCH 00/16] ib_mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
[RFC PATCH 07/16] ib/mad: create a jumbo MAD kmem_cache
why not use a single kmem-cache instance with a non hard coded
element size, 256B (or whatever we use today) or 2KB?
I wanted to be able to adjust the element count of the caches separately to
better tune overall memory usage. However, I stopped short of adding
additional module parameters to adjust the 2K cache at this time.
I tend to think that the resulted code is too much of a special purpose one
under a (jumbo == 2K) assumption. See some more comments in the individual
patches and we'll take it from there.
Ok, I'll address those comments in the other email threads.
Also (nit), please change the prefix for all patches to be IB/mad:
and not
ib/mad: to comply with the existing habit of patch titles for the IB
subsystem
I will thanks.
Good. See below another easy-to-fix nitpicking comment, but before that, for
the sake of easier review and post-robustness of the code to future
bisections,
please do a re-ordering of the series such that all general refactoring and
pre-
patches come before the OPApatches.
This goes to re-order the current series such tat patches 8/9/10 are located
after patch 14, as listed here:
[RFC PATCH 01/16] ib/mad: rename is_data_mad to is_rmpp_data_mad
[RFC PATCH 02/16] ib/core: add IB_DEVICE_JUMBO_MAD_SUPPORT device
cap
[RFC PATCH 03/16] ib/mad: Add check for jumbo MADs support on a device
[RFC PATCH 04/16] ib/mad: add base version parameter to
[RFC PATCH 05/16] ib/mad: Add MAD size parameters to process_mad
[RFC PATCH 06/16] ib/mad: Create jumbo_mad data structures
[RFC PATCH 07/16] ib/mad: create a jumbo MAD kmem_cache
[RFC PATCH 11/16] ib/mad: create helper function for
[RFC PATCH 12/16] ib/mad: create helper function for
[RFC PATCH 13/16] ib/mad: create helper function for
[RFC PATCH 14/16] ib/mad: Create helper function for SMI processing
[RFC PATCH 08/16] ib/mad: Add Intel Omni-Path Architecture defines
[RFC PATCH 09/16] ib/mad: Implement support for Intel Omni-Path
[RFC PATCH 10/16] ib/mad: Add registration check for Intel Omni-Path
[RFC PATCH 15/16] ib/mad: Implement Intel Omni-Path Architecture SMP
[RFC PATCH 16/16] ib/mad: Implement Intel Omni-Path Architecture General
Done.
Another easy-to-fix nitpicking comment would be to have all the patches be
consistent w.r.t to the capitalization of the 1st letter in the 1st word
after the
IB/core: or IB/mad: prefix, e.g
ib/mad: create helper function for smi_handle_dr_smp_send
becomes
IB/mad: Create helper function for smi_handle_dr_smp_send
Done.
BTW, here my personal preference is Add helper and not Create helper
IB/mad: Add helper function for smi_handle_dr_smp_send
Done.
Ira
N�r��yb�X��ǧv�^�){.n�+{��ٚ�{ay� ʇڙ�,j��f���h���z� �w���
���j:+v���w�j�mzZ+�ݢj��!�i
Re: [RFC PATCH 00/16] ib_mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
On 11/25/2014 11:52 PM, Weiny, Ira wrote: On Thu, Nov 13, 2014 at 9:54 PM, ira.we...@intel.com wrote: The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to process Intel Omni-Path Architecture MADs on devices which support them. So this series allows to process such mad when it arrives or goes beyond that and allows to send such mad too? Both send and receive is supported. My apologies for not being clear. In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These jumbo MADs increase the performance of management traffic. Can you provide 1-2 use cases where such mads will be sent and by what entity? I recall 2KB mads were mentioned over our LWG talks 8 years ago on IB routers... The Intel Omni-Path driver and SM will be the entities using these MADs. The patch series is written such that other devices could use jumbo MAD's but there is no attempt to predict how other technologies would do so. [snip] [RFC PATCH 07/16] ib/mad: create a jumbo MAD kmem_cache why not use a single kmem-cache instance with a non hard coded element size, 256B (or whatever we use today) or 2KB? I wanted to be able to adjust the element count of the caches separately to better tune overall memory usage. However, I stopped short of adding additional module parameters to adjust the 2K cache at this time. I tend to think that the resulted code is too much of a special purpose one under a (jumbo == 2K) assumption. See some more comments in the individual patches and we'll take it from there. Also (nit), please change the prefix for all patches to be IB/mad: and not ib/mad: to comply with the existing habit of patch titles for the IB subsystem I will thanks. Good. See below another easy-to-fix nitpicking comment, but before that, for the sake of easier review and post-robustness of the code to future bisections, please do a re-ordering of the series such that all general refactoring and pre-patches come before the OPApatches. This goes to re-order the current series such tat patches 8/9/10 are located after patch 14, as listed here: [RFC PATCH 01/16] ib/mad: rename is_data_mad to is_rmpp_data_mad [RFC PATCH 02/16] ib/core: add IB_DEVICE_JUMBO_MAD_SUPPORT device cap [RFC PATCH 03/16] ib/mad: Add check for jumbo MADs support on a device [RFC PATCH 04/16] ib/mad: add base version parameter to [RFC PATCH 05/16] ib/mad: Add MAD size parameters to process_mad [RFC PATCH 06/16] ib/mad: Create jumbo_mad data structures [RFC PATCH 07/16] ib/mad: create a jumbo MAD kmem_cache [RFC PATCH 11/16] ib/mad: create helper function for [RFC PATCH 12/16] ib/mad: create helper function for [RFC PATCH 13/16] ib/mad: create helper function for [RFC PATCH 14/16] ib/mad: Create helper function for SMI processing [RFC PATCH 08/16] ib/mad: Add Intel Omni-Path Architecture defines [RFC PATCH 09/16] ib/mad: Implement support for Intel Omni-Path [RFC PATCH 10/16] ib/mad: Add registration check for Intel Omni-Path [RFC PATCH 15/16] ib/mad: Implement Intel Omni-Path Architecture SMP [RFC PATCH 16/16] ib/mad: Implement Intel Omni-Path Architecture General Another easy-to-fix nitpicking comment would be to have all the patches be consistent w.r.t to the capitalization of the 1st letter in the 1st word after the IB/core: or IB/mad: prefix, e.g ib/mad: create helper function for smi_handle_dr_smp_send becomes IB/mad: Create helper function for smi_handle_dr_smp_send BTW, here my personal preference is Add helper and not Create helper IB/mad: Add helper function for smi_handle_dr_smp_send -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [RFC PATCH 00/16] ib_mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
On Thu, Nov 13, 2014 at 9:54 PM, ira.we...@intel.com wrote: The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to process Intel Omni-Path Architecture MADs on devices which support them. So this series allows to process such mad when it arrives or goes beyond that and allows to send such mad too? Both send and receive is supported. My apologies for not being clear. In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These jumbo MADs increase the performance of management traffic. Can you provide 1-2 use cases where such mads will be sent and by what entity? I recall 2KB mads were mentioned over our LWG talks 8 years ago on IB routers... The Intel Omni-Path driver and SM will be the entities using these MADs. The patch series is written such that other devices could use jumbo MAD's but there is no attempt to predict how other technologies would do so. [snip] [RFC PATCH 07/16] ib/mad: create a jumbo MAD kmem_cache why not use a single kmem-cache instance with a non hard coded element size, 256B (or whatever we use today) or 2KB? I wanted to be able to adjust the element count of the caches separately to better tune overall memory usage. However, I stopped short of adding additional module parameters to adjust the 2K cache at this time. Also (nit), please change the prefix for all patches to be IB/mad: and not ib/mad: to comply with the existing habit of patch titles for the IB subsystem I will thanks. And (another nit), generate patch 0/N using $ git format-patch --cover-letter so we have the exact subject line for each patch and the over-all diffstat in the cover-letter I will. My reason for not doing this previously was that I run the git send-email command on generated patches from a different machine than the one on which my repo is located due to Intel's firewall setup. I will clone the repo there and do this in the future. -- Ira
Re: [RFC PATCH 00/16] ib_mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
On 11/13/2014 2:54 PM, ira.we...@intel.com wrote: The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to process Intel Omni-Path Architecture MADs on devices which support them. In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These jumbo MADs increase the performance of management traffic. To distinguish IBTA MADs from OPA MADs a new Base Version is introduced. The new format shares the same common header with IBTA MADs which allows us to share most of the MAD processing code when dealing with the new Base Version. Is there any intention on standardizing OPA (jumbo MADs) at the IBTA or has that ship already sailed ? I'm assuming it is the latter. In any case, OPA base version should be claimed at IBTA so it won't be used by anything else in the future at the IBTA. Also, will OPA nodes interoperate with standard IBTA nodes or does OPA assume a homogeneous OPA subnet ? The patch series is broken into 3 main areas. 1) Add the ability for devices to indicate jumbo MAD support. In addition, modify the ib_mad module to detect those devices and allocate the resources for the QPs on those devices. 2) Enhance the interface to the device agents to support larger and variable length MADs. 3) Add support for creating and processing OPA Base Version MADs including a new SMP class version specific to OPA devices. At a minimum, OPA SMP class version should also be claimed at IBTA. [RFC PATCH 01/16] ib/mad: rename is_data_mad to is_rmpp_data_mad [RFC PATCH 02/16] ib/core: add IB_DEVICE_JUMBO_MAD_SUPPORT device cap [RFC PATCH 03/16] ib/mad: Add check for jumbo MADs support on a device [RFC PATCH 04/16] ib/mad: add base version parameter to [RFC PATCH 05/16] ib/mad: Add MAD size parameters to process_mad [RFC PATCH 06/16] ib/mad: Create jumbo_mad data structures [RFC PATCH 07/16] ib/mad: create a jumbo MAD kmem_cache [RFC PATCH 08/16] ib/mad: Add Intel Omni-Path Architecture defines [RFC PATCH 09/16] ib/mad: Implement support for Intel Omni-Path [RFC PATCH 10/16] ib/mad: Add registration check for Intel Omni-Path [RFC PATCH 11/16] ib/mad: create helper function for [RFC PATCH 12/16] ib/mad: create helper function for [RFC PATCH 13/16] ib/mad: create helper function for [RFC PATCH 14/16] ib/mad: Create helper function for SMI processing [RFC PATCH 15/16] ib/mad: Implement Intel Omni-Path Architecture SMP [RFC PATCH 16/16] ib/mad: Implement Intel Omni-Path Architecture General Has regression testing yet been done with these changes in terms of IBTA MAD support in terms of agents, SMs, and diagnostics to be sure that things still work properly ? -- Hal -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 00/16] ib_mad: Add support for Intel Omni-Path Architecture (OPA) MAD processing.
On Thu, Nov 13, 2014 at 9:54 PM, ira.we...@intel.com wrote: The following patch series modifies the kernel MAD processing (ib_mad/ib_umad) and related interfaces to process Intel Omni-Path Architecture MADs on devices which support them. So this series allows to process such mad when it arrives or goes beyond that and allows to send such mad too? In addition to supporting some IBTA management classes, OPA devices use MADs with lengths up to 2K. These jumbo MADs increase the performance of management traffic. Can you provide 1-2 use cases where such mads will be sent and by what entity? I recall 2KB mads were mentioned over our LWG talks 8 years ago on IB routers... To distinguish IBTA MADs from OPA MADs a new Base Version is introduced. The new format shares the same common header with IBTA MADs which allows us to share most of the MAD processing code when dealing with the new Base Version. The patch series is broken into 3 main areas. 1) Add the ability for devices to indicate jumbo MAD support. In addition, modify the ib_mad module to detect those devices and allocate the resources for the QPs on those devices. 2) Enhance the interface to the device agents to support larger and variable length MADs. 3) Add support for creating and processing OPA Base Version MADs including a new SMP class version specific to OPA devices. [RFC PATCH 01/16] ib/mad: rename is_data_mad to is_rmpp_data_mad [RFC PATCH 02/16] ib/core: add IB_DEVICE_JUMBO_MAD_SUPPORT device cap [RFC PATCH 03/16] ib/mad: Add check for jumbo MADs support on a device [RFC PATCH 04/16] ib/mad: add base version parameter to [RFC PATCH 05/16] ib/mad: Add MAD size parameters to process_mad [RFC PATCH 06/16] ib/mad: Create jumbo_mad data structures [RFC PATCH 07/16] ib/mad: create a jumbo MAD kmem_cache why not use a single kmem-cache instance with a non hard coded element size, 256B (or whatever we use today) or 2KB? Also (nit), please change the prefix for all patches to be IB/mad: and not ib/mad: to comply with the existing habit of patch titles for the IB subsystem And (another nit), generate patch 0/N using $ git format-patch --cover-letter so we have the exact subject line for each patch and the over-all diffstat in the cover-letter [RFC PATCH 08/16] ib/mad: Add Intel Omni-Path Architecture defines [RFC PATCH 09/16] ib/mad: Implement support for Intel Omni-Path [RFC PATCH 10/16] ib/mad: Add registration check for Intel Omni-Path [RFC PATCH 11/16] ib/mad: create helper function for [RFC PATCH 12/16] ib/mad: create helper function for [RFC PATCH 13/16] ib/mad: create helper function for [RFC PATCH 14/16] ib/mad: Create helper function for SMI processing [RFC PATCH 15/16] ib/mad: Implement Intel Omni-Path Architecture SMP [RFC PATCH 16/16] ib/mad: Implement Intel Omni-Path Architecture General -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
