Re: [vpp-dev] Using prefetch and buffer_enqueue

[Damjan Marion via Lists.Fd.Io]

> On 10 Jan 2019, at 07:15, Raj  wrote:
> 
> On Wed, Jan 9, 2019 at 4:18 PM Damjan Marion  wrote:
>>> On 9 Jan 2019, at 09:53, Raj  wrote:
>>> 1.   static_always_inline void
>>>  vlib_buffer_enqueue_to_next (vlib_main_t * vm, vlib_node_runtime_t * node,
>>>   u32 * buffers, u16 * nexts, uword count)
>>> 
>> 
>> This function simply gets array of buffer indices and array of next indices,
>> and enqueues those buffers to be processed by those next nodes...
> 
> Is this function only used in the case of feature arc, where some
> packets satisfies certain criteria and moves to a different node?
> Basically from where do we get the array of next indices?

No, it can be used without feature arcs.

> 
> Also in the code certain macros CLIB_HAVE_VEC128, CLIB_HAVE_VEC256 and
> CLIB_HAVE_VEC512 are defined. I could not figure out what these do.

They don't do anything, they are just set or not set depending on availability 
of
vector instructions on the target cpu.


> 
>>> type can be either LOAD or STORE. What is the purpose of these two 
>>> operations?
>> On x86, there was only load prefetch for a long time, recently store 
>> prefetch is introduced
>> with Skylake microarchitecture. You can find explanation here:
> 
> Based on code walkthrough, one thing I have observed is where ever
> .function is defined, static binding happens. In other nodes, dynamic
> binding based on the macro VLIB_NODE_FN happens.

yep

> So If I understand
> correctly, we have to use STORE in the latter case. But we can use
> LOAD as well. There is no hard and fast rule. Please correct me if I'm
> wrong.

It is good practice to pick the one which you really need. i.e. if you want to 
prefetch for 
store, just use STORE, if store is not available compiler will do LOAD 
instead...

-- 
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11911): https://lists.fd.io/g/vpp-dev/message/11911
Mute This Topic: https://lists.fd.io/mt/28982509/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Using prefetch and buffer_enqueue

[Raj]

On Wed, Jan 9, 2019 at 4:18 PM Damjan Marion  wrote:
>> On 9 Jan 2019, at 09:53, Raj  wrote:
>> 1.   static_always_inline void
>>   vlib_buffer_enqueue_to_next (vlib_main_t * vm, vlib_node_runtime_t * node,
>>u32 * buffers, u16 * nexts, uword count)
>>
>
> This function simply gets array of buffer indices and array of next indices,
> and enqueues those buffers to be processed by those next nodes...

Is this function only used in the case of feature arc, where some
packets satisfies certain criteria and moves to a different node?
Basically from where do we get the array of next indices?

Also in the code certain macros CLIB_HAVE_VEC128, CLIB_HAVE_VEC256 and
CLIB_HAVE_VEC512 are defined. I could not figure out what these do.

>> type can be either LOAD or STORE. What is the purpose of these two 
>> operations?
> On x86, there was only load prefetch for a long time, recently store prefetch 
> is introduced
> with Skylake microarchitecture. You can find explanation here:

Based on code walkthrough, one thing I have observed is where ever
.function is defined, static binding happens. In other nodes, dynamic
binding based on the macro VLIB_NODE_FN happens. So If I understand
correctly, we have to use STORE in the latter case. But we can use
LOAD as well. There is no hard and fast rule. Please correct me if I'm
wrong.

Thanks and Regards,

Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11893): https://lists.fd.io/g/vpp-dev/message/11893
Mute This Topic: https://lists.fd.io/mt/28982509/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Using prefetch and buffer_enqueue

[Damjan Marion via Lists.Fd.Io]

> On 9 Jan 2019, at 09:53, Raj  wrote:
> 
> Hello all,
> 
> While going through the VPP code, I have not fully understood  what
> the following functions mean and when should they be used?
> 
> 1.   static_always_inline void
>   vlib_buffer_enqueue_to_next (vlib_main_t * vm, vlib_node_runtime_t * node,
>u32 * buffers, u16 * nexts, uword count)
> 
>   I could see following comments about this function
>   /*
>* Send the packets along their respective next-node graph arcs
>* Considerable locality of reference is expected, most if not all
>* packets in the inbound vector will traverse the same next-node
>* arc
>*/
> I tried reading through the code but could not figure out what this
> function does, and how this is implemented. I see that the code has
> optimizations for various CPU, but not could not figure out what those
> optimizations are.

This function simply gets array of buffer indices and array of next indices,
and enqueues those buffers to be processed by those next nodes...


> 
> 2.  #define vlib_prefetch_buffer_header(b, type )
>   CLIB_PREFETCH (b, 64, type)
> 
> This gets expanded to:
> #define _CLIB_PREFETCH(n,size,type) \
>  if ((size) > (n)*CLIB_CACHE_LINE_BYTES)   \
>__builtin_prefetch (_addr + (n)*CLIB_CACHE_LINE_BYTES,  \
>CLIB_PREFETCH_##type,   \
>/* locality */ 3);
> 
> type can be either LOAD or STORE. What is the purpose of these two operations?
> 
> I am sure I am missing some thing with respect to understanding
> prefetch. I assume LOAD will prefetch data from memory, and STORE
> performs its inverse operation?


On x86, there was only load prefetch for a long time, recently store prefetch 
is introduced
with Skylake microarchitecture. You can find explanation here:

https://www.felixcloutier.com/x86/prefetchwt1 


Please note that compiler will emit PREFETCHWT1 instruction only if your
code is using multi-versioning in VPP (use of VLIB_NODE_FN() macro + 
MULTIARCH_SOURCES in CMakeLists.txt file).
and only when skylake is detected on runtime.

Otherwise CLIB_PREFETCH (x, 64, STORE) will produce identical results 
like you did LOAD

-- 
Damjan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11882): https://lists.fd.io/g/vpp-dev/message/11882
Mute This Topic: https://lists.fd.io/mt/28982509/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Using prefetch and buffer_enqueue

[Raj]

 Hello all,

While going through the VPP code, I have not fully understood  what
the following functions mean and when should they be used?

1.   static_always_inline void
   vlib_buffer_enqueue_to_next (vlib_main_t * vm, vlib_node_runtime_t * node,
u32 * buffers, u16 * nexts, uword count)

   I could see following comments about this function
   /*
* Send the packets along their respective next-node graph arcs
* Considerable locality of reference is expected, most if not all
* packets in the inbound vector will traverse the same next-node
* arc
*/
I tried reading through the code but could not figure out what this
function does, and how this is implemented. I see that the code has
optimizations for various CPU, but not could not figure out what those
optimizations are.

2.  #define vlib_prefetch_buffer_header(b, type )
   CLIB_PREFETCH (b, 64, type)

This gets expanded to:
#define _CLIB_PREFETCH(n,size,type) \
  if ((size) > (n)*CLIB_CACHE_LINE_BYTES)   \
__builtin_prefetch (_addr + (n)*CLIB_CACHE_LINE_BYTES,  \
CLIB_PREFETCH_##type,   \
/* locality */ 3);

type can be either LOAD or STORE. What is the purpose of these two operations?

I am sure I am missing some thing with respect to understanding
prefetch. I assume LOAD will prefetch data from memory, and STORE
performs its inverse operation?

Thanks and Regards,

Raj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11878): https://lists.fd.io/g/vpp-dev/message/11878
Mute This Topic: https://lists.fd.io/mt/28982509/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-