Re: [Ntop-misc] Determining rx queues of an interface

[Amir Kaduri]

Thank you!

On Wed, Jul 11, 2018 at 5:09 PM Alfredo Cardigliano 
wrote:

> Hi Amir
> the number of rx queues is determined loading the driver or at runtime
> with ethtool ‘combined’ as you can see at:
>
>
> https://github.com/ntop/PF_RING/blob/dev/drivers/intel/i40e/i40e-2.4.6-zc/src/load_driver.sh#L39
>
> Please also make sure that you are actually unloading the old driver
> before running insmod (maybe the new insmod is failing)
>
> Alfredo
>
> On 10 Jul 2018, at 15:46, Amir Kaduri  wrote:
>
> Do the rx queues of an interface are determined only when loading the
> kernel driver, or is there another thing that affects that?
>
> My problem is that I get a number that differs than what I think should be
> (because maybe something during the machine boot doesn't work properly)
>
> For example:
> What I assume our load script do is "*insmod
> /usr/local/pfring/drivers/intel/i40e.ko RSS=12,12,12,12,12,12,12,12*"
> What I actually get when I run "*cat /proc/net/pf_ring/dev/eth4/info*"
> is "*RX Queues:14*" (from previous reboot).
>
> So maybe the "insmod" doesn't actually work as expected, or is there
> anything else that I'm not aware of?
>
> A'ii appreciate any help.
>
> Thanks
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Determining rx queues of an interface

[Amir Kaduri]

Do the rx queues of an interface are determined only when loading the
kernel driver, or is there another thing that affects that?

My problem is that I get a number that differs than what I think should be
(because maybe something during the machine boot doesn't work properly)

For example:
What I assume our load script do is "*insmod
/usr/local/pfring/drivers/intel/i40e.ko RSS=12,12,12,12,12,12,12,12*"
What I actually get when I run "*cat /proc/net/pf_ring/dev/eth4/info*"
is "*RX Queues:14*" (from previous reboot).

So maybe the "insmod" doesn't actually work as expected, or is there
anything else that I'm not aware of?

A'ii appreciate any help.

Thanks
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Same feature, new implementation in PR:
https://github.com/ntop/PF_RING/pull/343


On Sun, May 6, 2018 at 7:18 PM, Amir Kaduri  wrote:

> Good.
> There is a pull-request waiting. I hope you'll find it beneficial:
> https://github.com/ntop/PF_RING/pull/340
>
> Thanks,
> Amir
>
> On Tue, May 1, 2018 at 1:19 AM, Alfredo Cardigliano 
> wrote:
>
>>
>>
>> On 30 Apr 2018, at 17:52, Amir Kaduri  wrote:
>>
>> Thanks for the answers.
>>
>> So the only way to make handlep->timeout>=0, is by setting the
>> file-descriptor to "blocking" (nonblock=0) according to the logic in
>> function pcap_setnonblock_mmap() and this is something that we would
>> like to avoid.
>> Therefore, we do the polling (non-blocking) in the application that uses
>> pcap/pf_ring.
>> The problem we have is with low-traffic network. According to the logic
>> in function copy_data_to_ring(), as long as the queue didn't reach the
>> "poll_num_pkts_watermark" threshold (in our case 128 packets),
>> the poll() (in userspace) won't be called (since  wake_up_interruptible(..)
>> is not called), which means that we have packets that are stuck in the ring
>> till the queue reaches the watermark.
>>
>> I wonder if you see any rationale in improving the pf_ring kernel module
>> code, to call  wake_up_interruptible() (in order to flush the queue) if
>> some "timeout" passed and the queue is not empty (but still didn't reach
>> the watermark).
>>
>>
>> I think that using the watermark in combination with a timeout is a good
>> idea.
>>
>> Alfredo
>>
>> Amir
>>
>>
>> On Thu, Apr 26, 2018 at 6:00 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>>
>>> On 26 Apr 2018, at 15:34, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>> My code is based on libpcap, while pfring's userland examples use pfring
>>> APIs directly, therefore things are a bit harder for me.
>>>
>>> Short clarification about a related code-line:
>>> Please look at the following line: https://github.com/ntop/
>>> PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875
>>>
>>> (1)  If I understand it correctly, if wait_for_incoming_packet is true,
>>> then pfring_poll() should be called.
>>>   Don't you want wait_for_incoming_packet to be true in case
>>> pf_ring_active_poll is true?
>>>
>>>
>>> “active” means spinning, thus poll should not be used in that case.
>>>
>>>   Currently, its the opposite (i.e. if pf_ring_active_poll is true,
>>> wait_for_incoming_packet will be false thus pfring_poll() won't be
>>> called).
>>>
>>>
>>> This seems to be correct
>>>
>>>
>>> (2) If the code is ok, then the only way for me to make
>>> wait_for_incoming_packet true (for pfring_poll() to be called) is by
>>> making handlep->timeout >= 0.
>>>  Correct?
>>>
>>>
>>> Correct
>>>
>>> Alfredo
>>>
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>> Hi Amir
>>>> if I understand correctly, pfcount_multichannel is working, while in
>>>> your application
>>>> it seems that poll does not honor the timeout, if this is the case it
>>>> seems the problem
>>>> is not in the kernel module, I think you should look for differences
>>>> between the two applications..
>>>>
>>>> Alfredo
>>>>
>>>> On 9 Apr 2018, at 07:20, Amir Kaduri  wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> I'm back to investigate/debug this issue in my environment, and maybe
>>>> you'll manage to save me some time:
>>>>
>>>> When I use the example program "pfcount_multichannel", poll-duration
>>>> works for me as expected:
>>>> For watermark=128, poll-duration=1000, even if less than 128 packets
>>>> received, I get them in pfcount_multichannel.
>>>>
>>>> On the other hand, in my other program (which is a complex one), the
>>>> userspace application gets the packets only after 128 packets
>>>> aggregated by the ring, regardless the polling rate (which is done
>>>> always using 50ms timeou

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Good.
There is a pull-request waiting. I hope you'll find it beneficial:
https://github.com/ntop/PF_RING/pull/340

Thanks,
Amir

On Tue, May 1, 2018 at 1:19 AM, Alfredo Cardigliano 
wrote:

>
>
> On 30 Apr 2018, at 17:52, Amir Kaduri  wrote:
>
> Thanks for the answers.
>
> So the only way to make handlep->timeout>=0, is by setting the
> file-descriptor to "blocking" (nonblock=0) according to the logic in
> function pcap_setnonblock_mmap() and this is something that we would like
> to avoid.
> Therefore, we do the polling (non-blocking) in the application that uses
> pcap/pf_ring.
> The problem we have is with low-traffic network. According to the logic in
> function copy_data_to_ring(), as long as the queue didn't reach the
> "poll_num_pkts_watermark" threshold (in our case 128 packets),
> the poll() (in userspace) won't be called (since  wake_up_interruptible(..)
> is not called), which means that we have packets that are stuck in the ring
> till the queue reaches the watermark.
>
> I wonder if you see any rationale in improving the pf_ring kernel module
> code, to call  wake_up_interruptible() (in order to flush the queue) if
> some "timeout" passed and the queue is not empty (but still didn't reach
> the watermark).
>
>
> I think that using the watermark in combination with a timeout is a good
> idea.
>
> Alfredo
>
> Amir
>
>
> On Thu, Apr 26, 2018 at 6:00 PM, Alfredo Cardigliano  > wrote:
>
>>
>>
>> On 26 Apr 2018, at 15:34, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> My code is based on libpcap, while pfring's userland examples use pfring
>> APIs directly, therefore things are a bit harder for me.
>>
>> Short clarification about a related code-line:
>> Please look at the following line: https://github.com/ntop/
>> PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875
>>
>> (1)  If I understand it correctly, if wait_for_incoming_packet is true,
>> then pfring_poll() should be called.
>>   Don't you want wait_for_incoming_packet to be true in case
>> pf_ring_active_poll is true?
>>
>>
>> “active” means spinning, thus poll should not be used in that case.
>>
>>   Currently, its the opposite (i.e. if pf_ring_active_poll is true,
>> wait_for_incoming_packet will be false thus pfring_poll() won't be
>> called).
>>
>>
>> This seems to be correct
>>
>>
>> (2) If the code is ok, then the only way for me to make
>> wait_for_incoming_packet true (for pfring_poll() to be called) is by
>> making handlep->timeout >= 0.
>>  Correct?
>>
>>
>> Correct
>>
>> Alfredo
>>
>>
>> Thanks,
>> Amir
>>
>> On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Hi Amir
>>> if I understand correctly, pfcount_multichannel is working, while in
>>> your application
>>> it seems that poll does not honor the timeout, if this is the case it
>>> seems the problem
>>> is not in the kernel module, I think you should look for differences
>>> between the two applications..
>>>
>>> Alfredo
>>>
>>> On 9 Apr 2018, at 07:20, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>> I'm back to investigate/debug this issue in my environment, and maybe
>>> you'll manage to save me some time:
>>>
>>> When I use the example program "pfcount_multichannel", poll-duration
>>> works for me as expected:
>>> For watermark=128, poll-duration=1000, even if less than 128 packets
>>> received, I get them in pfcount_multichannel.
>>>
>>> On the other hand, in my other program (which is a complex one), the
>>> userspace application gets the packets only after 128 packets
>>> aggregated by the ring, regardless the polling rate (which is done
>>> always using 50ms timeout).
>>>
>>> Maybe you can figure out what can "hold" the packets in the ring and
>>> forward them to userspace only when the watermark threshold passes?
>>> Maybe something is missing during initialization?
>>> (for simplicity I'm not using rehash, and not using any filters).
>>>
>>> Thanks
>>>
>>> On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>> Hi Amir
>>>> that's correct, however for some reason it seems it is not the case in
>>&g

Re: [Ntop-misc] Question regarding implementation of API function pfring_get_ring_id()

[Amir Kaduri]

Hi Alfredo,

I didn't cache it since I didn't bare in mind that the implementation is
like that. I discovered it by chance, when debugging something else.
I can cache it of course quite easily and add a wrapping API in the
application, but I asked myself why to do it in the application if the
right thing is to do it in the original API.
The use-case BTW, is when you want the application to periodically dump
statistical information about the channels (rings). Its nice to have the
ring_id dumped as well.
Also, there is no API for reading the information of a socket (e.g.
/proc/net/pf_ring/1234-eth0.567), so to create it, you need the ring_id,
and here I thought that its better to get it directly from the API instead
of a variable that I'll hold.
Anyway, its not a big issue for me, I can cache it in my application of
course. I just raised it as a point to think about.

Thanks,
Amir


On Thu, May 3, 2018 at 4:44 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> please note I am not against caching it in the library, I was just
> wondering
> why you are not caching it in the application as it was the easiest
> solution
> and I am not aware of other use cases / applications reading it so often.
>
> Alfredo
>
> On 3 May 2018, at 15:40, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> No reason not to cache it inside the application, but why let any
> application to implement it, if its possible to implement it in pf_ring's
> userspace API (the way I mentioned)?
> BTW, if its sounds reasonable, I can push a suggested solution, and you
> can consider adopting it.
>
> Amir
>
> On Thu, May 3, 2018 at 4:11 PM, Alfredo Cardigliano 
> wrote:
>
>> Hi Amir
>> ring_id should not change, thus it can be cached in userspace,
>> any reason you cannot cache it inside the application?
>>
>> Alfredo
>>
>> > On 3 May 2018, at 15:09, Amir Kaduri  wrote:
>> >
>> > User-space API function pfring_get_ring_id() always bring the ring id
>> from the kernel. In some applications, this API call might be used within
>> debug messages, in order to be able to track a relevant ring. If the
>> application uses this API too often (for debugging purposes), there could
>> be too much calls for the kernel, for getting information that usually
>> doesn't change.
>> > So my questions are:
>> > 1. Once a ring is initialized with a ring_id (in function
>> ring_create()), will it ever be changed during the application life-time?
>> > 2. Assuming the answer is negative (i.e. once a ring id is determined,
>> it doesn't change),
>> > is there a room for caching this value in userspace and avoid future
>> calls to kernel?
>> > (e.g. by adding "ring_id" member to "struct __pfring" in pfring.h, and
>> let function pfring_mod_get_ring_id() to return it once initialized,
>> instead of the call to getsockopt(.. SO_GET_RING_ID ..))
>> >
>> > Thanks, Amir
>> >
>> >
>> > ___
>> > Ntop-misc mailing list
>> > Ntop-misc@listgateway.unipi.it
>> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Question regarding implementation of API function pfring_get_ring_id()

[Amir Kaduri]

Hi Alfredo,

No reason not to cache it inside the application, but why let any
application to implement it, if its possible to implement it in pf_ring's
userspace API (the way I mentioned)?
BTW, if its sounds reasonable, I can push a suggested solution, and you can
consider adopting it.

Amir

On Thu, May 3, 2018 at 4:11 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> ring_id should not change, thus it can be cached in userspace,
> any reason you cannot cache it inside the application?
>
> Alfredo
>
> > On 3 May 2018, at 15:09, Amir Kaduri  wrote:
> >
> > User-space API function pfring_get_ring_id() always bring the ring id
> from the kernel. In some applications, this API call might be used within
> debug messages, in order to be able to track a relevant ring. If the
> application uses this API too often (for debugging purposes), there could
> be too much calls for the kernel, for getting information that usually
> doesn't change.
> > So my questions are:
> > 1. Once a ring is initialized with a ring_id (in function
> ring_create()), will it ever be changed during the application life-time?
> > 2. Assuming the answer is negative (i.e. once a ring id is determined,
> it doesn't change),
> > is there a room for caching this value in userspace and avoid future
> calls to kernel?
> > (e.g. by adding "ring_id" member to "struct __pfring" in pfring.h, and
> let function pfring_mod_get_ring_id() to return it once initialized,
> instead of the call to getsockopt(.. SO_GET_RING_ID ..))
> >
> > Thanks, Amir
> >
> >
> > ___
> > Ntop-misc mailing list
> > Ntop-misc@listgateway.unipi.it
> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Question regarding implementation of API function pfring_get_ring_id()

[Amir Kaduri]

User-space API function pfring_get_ring_id() always bring the ring id from
the kernel. In some applications, this API call might be used within debug
messages, in order to be able to track a relevant ring. If the application
uses this API too often (for debugging purposes), there could be too much
calls for the kernel, for getting information that usually doesn't change.
So my questions are:
1. Once a ring is initialized with a ring_id (in function ring_create()),
will it ever be changed during the application life-time?
2. Assuming the answer is negative (i.e. once a ring id is determined, it
doesn't change),
is there a room for caching this value in userspace and avoid future calls
to kernel?
(e.g. by adding "ring_id" member to "struct __pfring" in pfring.h, and let
function pfring_mod_get_ring_id() to return it once initialized, instead of
the call to getsockopt(.. SO_GET_RING_ID ..))

Thanks, Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Thanks for the answers.

So the only way to make handlep->timeout>=0, is by setting the
file-descriptor to "blocking" (nonblock=0) according to the logic in
function pcap_setnonblock_mmap() and this is something that we would like
to avoid.
Therefore, we do the polling (non-blocking) in the application that uses
pcap/pf_ring.
The problem we have is with low-traffic network. According to the logic in
function copy_data_to_ring(), as long as the queue didn't reach the
"poll_num_pkts_watermark" threshold (in our case 128 packets),
the poll() (in userspace) won't be called (since  wake_up_interruptible(..)
is not called), which means that we have packets that are stuck in the ring
till the queue reaches the watermark.

I wonder if you see any rationale in improving the pf_ring kernel module
code, to call  wake_up_interruptible() (in order to flush the queue) if
some "timeout" passed and the queue is not empty (but still didn't reach
the watermark).

Amir


On Thu, Apr 26, 2018 at 6:00 PM, Alfredo Cardigliano 
wrote:

>
>
> On 26 Apr 2018, at 15:34, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> My code is based on libpcap, while pfring's userland examples use pfring
> APIs directly, therefore things are a bit harder for me.
>
> Short clarification about a related code-line:
> Please look at the following line: https://github.com/ntop/
> PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875
>
> (1)  If I understand it correctly, if wait_for_incoming_packet is true,
> then pfring_poll() should be called.
>   Don't you want wait_for_incoming_packet to be true in case
> pf_ring_active_poll is true?
>
>
> “active” means spinning, thus poll should not be used in that case.
>
>   Currently, its the opposite (i.e. if pf_ring_active_poll is true,
> wait_for_incoming_packet will be false thus pfring_poll() won't be
> called).
>
>
> This seems to be correct
>
>
> (2) If the code is ok, then the only way for me to make
> wait_for_incoming_packet true (for pfring_poll() to be called) is by
> making handlep->timeout >= 0.
>  Correct?
>
>
> Correct
>
> Alfredo
>
>
> Thanks,
> Amir
>
> On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano  > wrote:
>
>> Hi Amir
>> if I understand correctly, pfcount_multichannel is working, while in your
>> application
>> it seems that poll does not honor the timeout, if this is the case it
>> seems the problem
>> is not in the kernel module, I think you should look for differences
>> between the two applications..
>>
>> Alfredo
>>
>> On 9 Apr 2018, at 07:20, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> I'm back to investigate/debug this issue in my environment, and maybe
>> you'll manage to save me some time:
>>
>> When I use the example program "pfcount_multichannel", poll-duration
>> works for me as expected:
>> For watermark=128, poll-duration=1000, even if less than 128 packets
>> received, I get them in pfcount_multichannel.
>>
>> On the other hand, in my other program (which is a complex one), the
>> userspace application gets the packets only after 128 packets
>> aggregated by the ring, regardless the polling rate (which is done always
>> using 50ms timeout).
>>
>> Maybe you can figure out what can "hold" the packets in the ring and
>> forward them to userspace only when the watermark threshold passes?
>> Maybe something is missing during initialization?
>> (for simplicity I'm not using rehash, and not using any filters).
>>
>> Thanks
>>
>> On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Hi Amir
>>> that's correct, however for some reason it seems it is not the case in
>>> your tests.
>>>
>>> Alfredo
>>>
>>> On 31 Oct 2017, at 12:08, Amir Kaduri  wrote:
>>>
>>> Thanks. tot_insert apparently works ok.
>>>
>>> Regarding function copy_data_to_ring():
>>> At the end of it there is the statement:
>>>  if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark)
>>>  wake_up_interruptible(&pfr->ring_slots_waitqueue);
>>>
>>> Since watermark is set to 128, and I send <128 packets, this causes them
>>> to wait in kernel queue.
>>> But since poll_duration is set to 1 (1 millisecond I assume), I expect
>>> the condition to check this also (meaning, there are packets in queue but 1
>>> millisecond passed and they weren't read),
>>> the wake_up_interr

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Hi Alfredo,

My code is based on libpcap, while pfring's userland examples use pfring
APIs directly, therefore things are a bit harder for me.

Short clarification about a related code-line:
Please look at the following line:
https://github.com/ntop/PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875

(1)  If I understand it correctly, if wait_for_incoming_packet is true,
then pfring_poll() should be called.
  Don't you want wait_for_incoming_packet to be true in case
pf_ring_active_poll is true?
  Currently, its the opposite (i.e. if pf_ring_active_poll is true,
wait_for_incoming_packet will be false thus pfring_poll() won't be called).

(2) If the code is ok, then the only way for me to make
wait_for_incoming_packet true (for pfring_poll() to be called) is by making
handlep->timeout >= 0.
 Correct?

Thanks,
Amir

On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano 
wrote:

> Hi Amir
> if I understand correctly, pfcount_multichannel is working, while in your
> application
> it seems that poll does not honor the timeout, if this is the case it
> seems the problem
> is not in the kernel module, I think you should look for differences
> between the two applications..
>
> Alfredo
>
> On 9 Apr 2018, at 07:20, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> I'm back to investigate/debug this issue in my environment, and maybe
> you'll manage to save me some time:
>
> When I use the example program "pfcount_multichannel", poll-duration works
> for me as expected:
> For watermark=128, poll-duration=1000, even if less than 128 packets
> received, I get them in pfcount_multichannel.
>
> On the other hand, in my other program (which is a complex one), the
> userspace application gets the packets only after 128 packets
> aggregated by the ring, regardless the polling rate (which is done always
> using 50ms timeout).
>
> Maybe you can figure out what can "hold" the packets in the ring and
> forward them to userspace only when the watermark threshold passes?
> Maybe something is missing during initialization?
> (for simplicity I'm not using rehash, and not using any filters).
>
> Thanks
>
> On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano  > wrote:
>
>> Hi Amir
>> that's correct, however for some reason it seems it is not the case in
>> your tests.
>>
>> Alfredo
>>
>> On 31 Oct 2017, at 12:08, Amir Kaduri  wrote:
>>
>> Thanks. tot_insert apparently works ok.
>>
>> Regarding function copy_data_to_ring():
>> At the end of it there is the statement:
>>  if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark)
>>  wake_up_interruptible(&pfr->ring_slots_waitqueue);
>>
>> Since watermark is set to 128, and I send <128 packets, this causes them
>> to wait in kernel queue.
>> But since poll_duration is set to 1 (1 millisecond I assume), I expect
>> the condition to check this also (meaning, there are packets in queue but 1
>> millisecond passed and they weren't read),
>> the wake_up_interruptible should also be called. No?
>>
>> Thanks,
>> Amir
>>
>>
>> On Tue, Oct 31, 2017 at 10:20 AM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>>
>>> On 31 Oct 2017, at 08:42, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>> I'm trying to debug the issue, and I have a question about the code, to
>>> make sure that there is no problem there:
>>> Specifically, I'm referring to the function "pfring_mod_recv":
>>> In order that the line that refers to poll_duration ("pfring_poll(ring,
>>> ring->poll_duration)") will be reached, there are 2 conditions that should
>>> occur:
>>> 1. pfring_there_is_pkt_available(ring) should return false (otherwise,
>>> the function returns at the end of the condition).
>>> 2. wait_for_incoming_packet should be set to true.
>>> Currently, I'm referring to the first one:
>>> In order that the macro pfring_there_is_pkt_available(ring) will return
>>> false, ring->slots_info->tot_insert should be equal to
>>> ring->slots_info->tot_read.
>>> What I see in my tests that they don't get equal. I always see that
>>> tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
>>> but it happens inside the condition, so the "pfring_mod_recv" returns with
>>> 1.
>>>
>>>
>>> It seems to be correct. The kernel module inserts packets into the ring
>>> increasing tot_insert, the userspace

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Hi Alfredo,

I'm back to investigate/debug this issue in my environment, and maybe
you'll manage to save me some time:

When I use the example program "pfcount_multichannel", poll-duration works
for me as expected:
For watermark=128, poll-duration=1000, even if less than 128 packets
received, I get them in pfcount_multichannel.

On the other hand, in my other program (which is a complex one), the
userspace application gets the packets only after 128 packets
aggregated by the ring, regardless the polling rate (which is done always
using 50ms timeout).

Maybe you can figure out what can "hold" the packets in the ring and
forward them to userspace only when the watermark threshold passes?
Maybe something is missing during initialization?
(for simplicity I'm not using rehash, and not using any filters).

Thanks

On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> that's correct, however for some reason it seems it is not the case in
> your tests.
>
> Alfredo
>
> On 31 Oct 2017, at 12:08, Amir Kaduri  wrote:
>
> Thanks. tot_insert apparently works ok.
>
> Regarding function copy_data_to_ring():
> At the end of it there is the statement:
>  if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark)
>  wake_up_interruptible(&pfr->ring_slots_waitqueue);
>
> Since watermark is set to 128, and I send <128 packets, this causes them
> to wait in kernel queue.
> But since poll_duration is set to 1 (1 millisecond I assume), I expect the
> condition to check this also (meaning, there are packets in queue but 1
> millisecond passed and they weren't read),
> the wake_up_interruptible should also be called. No?
>
> Thanks,
> Amir
>
>
> On Tue, Oct 31, 2017 at 10:20 AM, Alfredo Cardigliano <
> cardigli...@ntop.org> wrote:
>
>>
>>
>> On 31 Oct 2017, at 08:42, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> I'm trying to debug the issue, and I have a question about the code, to
>> make sure that there is no problem there:
>> Specifically, I'm referring to the function "pfring_mod_recv":
>> In order that the line that refers to poll_duration ("pfring_poll(ring,
>> ring->poll_duration)") will be reached, there are 2 conditions that should
>> occur:
>> 1. pfring_there_is_pkt_available(ring) should return false (otherwise,
>> the function returns at the end of the condition).
>> 2. wait_for_incoming_packet should be set to true.
>> Currently, I'm referring to the first one:
>> In order that the macro pfring_there_is_pkt_available(ring) will return
>> false, ring->slots_info->tot_insert should be equal to
>> ring->slots_info->tot_read.
>> What I see in my tests that they don't get equal. I always see that
>> tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
>> but it happens inside the condition, so the "pfring_mod_recv" returns with
>> 1.
>>
>>
>> It seems to be correct. The kernel module inserts packets into the ring
>> increasing tot_insert, the userspace library reads packets from the ring
>> increasing tot_read. This means that if tot_insert == tot_read there is no
>> packet to read. If there is a bug, it should be in the kernel module that
>> is somehow not adding packets to the ring (thus not updating tot_insert).
>>
>> Alfredo
>>
>> I remind that I set the watermark to be high, in order to see the
>> poll_duration takes effect.
>>
>> Could you please approve that you don't see any problem in the code?
>>
>> Thanks,
>> Amir
>>
>>
>> On Thu, Oct 26, 2017 at 12:22 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Hi Amir
>>> yes, that’s the way it should work, if this is not the case, some
>>> debugging is needed to identify the problem
>>>
>>> Alfredo
>>>
>>> On 26 Oct 2017, at 10:14, Amir Kaduri  wrote:
>>>
>>> Basically, the functionality that I would like to have is even if less
>>> than poll-watermark-threshold (default: 128) packets arrives the socket,
>>> they will be forwarded to userland if 1 millisecond has passed.
>>> How can I gain this? Isn't it by using  pfring_set_poll_duration()?
>>>
>>> Alfredo, could you please clarify?
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe
>>>&

[Ntop-misc] Using upgraded i40e from pf_ring 7.0 (2.2.4), on pf_ring 6.6.0

[Amir Kaduri]

Hi,

I'm using pf_ring 6.6.0 (not using  ZC) on CentOS 7.0.
I would like to upgrade the i40e driver (only) to 2.2.4 (i.e. take the code
of i40e from pf_ring 7.0 and put it in pf_ring 6.6.0 and use them together).
The question is how risky it is? How pf_ring is tight up with the i40e
driver version?

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Thanks. tot_insert apparently works ok.

Regarding function copy_data_to_ring():
At the end of it there is the statement:
 if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark)
 wake_up_interruptible(&pfr->ring_slots_waitqueue);

Since watermark is set to 128, and I send <128 packets, this causes them to
wait in kernel queue.
But since poll_duration is set to 1 (1 millisecond I assume), I expect the
condition to check this also (meaning, there are packets in queue but 1
millisecond passed and they weren't read),
the wake_up_interruptible should also be called. No?

Thanks,
Amir


On Tue, Oct 31, 2017 at 10:20 AM, Alfredo Cardigliano 
wrote:

>
>
> On 31 Oct 2017, at 08:42, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> I'm trying to debug the issue, and I have a question about the code, to
> make sure that there is no problem there:
> Specifically, I'm referring to the function "pfring_mod_recv":
> In order that the line that refers to poll_duration ("pfring_poll(ring,
> ring->poll_duration)") will be reached, there are 2 conditions that should
> occur:
> 1. pfring_there_is_pkt_available(ring) should return false (otherwise,
> the function returns at the end of the condition).
> 2. wait_for_incoming_packet should be set to true.
> Currently, I'm referring to the first one:
> In order that the macro pfring_there_is_pkt_available(ring) will return
> false, ring->slots_info->tot_insert should be equal to
> ring->slots_info->tot_read.
> What I see in my tests that they don't get equal. I always see that
> tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
> but it happens inside the condition, so the "pfring_mod_recv" returns with
> 1.
>
>
> It seems to be correct. The kernel module inserts packets into the ring
> increasing tot_insert, the userspace library reads packets from the ring
> increasing tot_read. This means that if tot_insert == tot_read there is no
> packet to read. If there is a bug, it should be in the kernel module that
> is somehow not adding packets to the ring (thus not updating tot_insert).
>
> Alfredo
>
> I remind that I set the watermark to be high, in order to see the
> poll_duration takes effect.
>
> Could you please approve that you don't see any problem in the code?
>
> Thanks,
> Amir
>
>
> On Thu, Oct 26, 2017 at 12:22 PM, Alfredo Cardigliano <
> cardigli...@ntop.org> wrote:
>
>> Hi Amir
>> yes, that’s the way it should work, if this is not the case, some
>> debugging is needed to identify the problem
>>
>> Alfredo
>>
>> On 26 Oct 2017, at 10:14, Amir Kaduri  wrote:
>>
>> Basically, the functionality that I would like to have is even if less
>> than poll-watermark-threshold (default: 128) packets arrives the socket,
>> they will be forwarded to userland if 1 millisecond has passed.
>> How can I gain this? Isn't it by using  pfring_set_poll_duration()?
>>
>> Alfredo, could you please clarify?
>>
>> Thanks,
>> Amir
>>
>> On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri  wrote:
>>
>>> Hi,
>>>
>>> I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe
>>> drivers).
>>> As far as I understand the relation between poll-watermark and
>>> poll-duration, packets will be queued untill one of comes first: or passing
>>> the poll-watermark packets threshold, or a poll-duration milliseconds has
>>> passed.
>>> I set poll-watermark to the maximum (4096) (using 
>>> pfring_set_poll_watermark())
>>> and set poll-duration to the minimum (1) (using pfring_set_poll_duratio
>>> n()).
>>> I've sent 400 packets to the socket. I see that they are received by the
>>> NIC, but they didn't pass to userland. Only when passing 500 packets, a
>>> chunk of them passed to userland.
>>> I don't quite understand the behavior: since poll-duration is 1
>>> (millisecond I assume), I've expected all the packets to pass to userland
>>> immediately, even though poll-watermark is much higher.
>>>
>>> Can anyone shed some light on the above?
>>>
>>> Thanks,
>>> Amir
>>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Hi Alfredo,

I'm trying to debug the issue, and I have a question about the code, to
make sure that there is no problem there:
Specifically, I'm referring to the function "pfring_mod_recv":
In order that the line that refers to poll_duration ("pfring_poll(ring,
ring->poll_duration)") will be reached, there are 2 conditions that should
occur:
1. pfring_there_is_pkt_available(ring) should return false (otherwise, the
function returns at the end of the condition).
2. wait_for_incoming_packet should be set to true.
Currently, I'm referring to the first one:
In order that the macro pfring_there_is_pkt_available(ring) will return
false, ring->slots_info->tot_insert should be equal to
ring->slots_info->tot_read.
What I see in my tests that they don't get equal. I always see that
tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
but it happens inside the condition, so the "pfring_mod_recv" returns with
1.
I remind that I set the watermark to be high, in order to see the
poll_duration takes effect.

Could you please approve that you don't see any problem in the code?

Thanks,
Amir


On Thu, Oct 26, 2017 at 12:22 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> yes, that’s the way it should work, if this is not the case, some
> debugging is needed to identify the problem
>
> Alfredo
>
> On 26 Oct 2017, at 10:14, Amir Kaduri  wrote:
>
> Basically, the functionality that I would like to have is even if less
> than poll-watermark-threshold (default: 128) packets arrives the socket,
> they will be forwarded to userland if 1 millisecond has passed.
> How can I gain this? Isn't it by using  pfring_set_poll_duration()?
>
> Alfredo, could you please clarify?
>
> Thanks,
> Amir
>
> On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri  wrote:
>
>> Hi,
>>
>> I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe
>> drivers).
>> As far as I understand the relation between poll-watermark and
>> poll-duration, packets will be queued untill one of comes first: or passing
>> the poll-watermark packets threshold, or a poll-duration milliseconds has
>> passed.
>> I set poll-watermark to the maximum (4096) (using 
>> pfring_set_poll_watermark())
>> and set poll-duration to the minimum (1) (using pfring_set_poll_duratio
>> n()).
>> I've sent 400 packets to the socket. I see that they are received by the
>> NIC, but they didn't pass to userland. Only when passing 500 packets, a
>> chunk of them passed to userland.
>> I don't quite understand the behavior: since poll-duration is 1
>> (millisecond I assume), I've expected all the packets to pass to userland
>> immediately, even though poll-watermark is much higher.
>>
>> Can anyone shed some light on the above?
>>
>> Thanks,
>> Amir
>>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Basically, the functionality that I would like to have is even if less than
poll-watermark-threshold (default: 128) packets arrives the socket, they
will be forwarded to userland if 1 millisecond has passed.
How can I gain this? Isn't it by using  pfring_set_poll_duration()?

Alfredo, could you please clarify?

Thanks,
Amir

On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri  wrote:

> Hi,
>
> I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe
> drivers).
> As far as I understand the relation between poll-watermark and
> poll-duration, packets will be queued untill one of comes first: or passing
> the poll-watermark packets threshold, or a poll-duration milliseconds has
> passed.
> I set poll-watermark to the maximum (4096) (using pfring_set_poll_watermark())
> and set poll-duration to the minimum (1) (using pfring_set_poll_
> duration()).
> I've sent 400 packets to the socket. I see that they are received by the
> NIC, but they didn't pass to userland. Only when passing 500 packets, a
> chunk of them passed to userland.
> I don't quite understand the behavior: since poll-duration is 1
> (millisecond I assume), I've expected all the packets to pass to userland
> immediately, even though poll-watermark is much higher.
>
> Can anyone shed some light on the above?
>
> Thanks,
> Amir
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration

[Amir Kaduri]

Hi,

I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe
drivers).
As far as I understand the relation between poll-watermark and
poll-duration, packets will be queued untill one of comes first: or passing
the poll-watermark packets threshold, or a poll-duration milliseconds has
passed.
I set poll-watermark to the maximum (4096)
(using pfring_set_poll_watermark()) and set poll-duration to the minimum
(1) (using pfring_set_poll_duration()).
I've sent 400 packets to the socket. I see that they are received by the
NIC, but they didn't pass to userland. Only when passing 500 packets, a
chunk of them passed to userland.
I don't quite understand the behavior: since poll-duration is 1
(millisecond I assume), I've expected all the packets to pass to userland
immediately, even though poll-watermark is much higher.

Can anyone shed some light on the above?

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] pf_ring: bug fix (rule inactivity)

[Amir Kaduri]

Hi Alfredo,

The attached patch includes a fix of a bug that relates to updating the
last time a hash rule was active. I assume that the
pfring_purge_idle_hash_rules()
API doesn't work for this reason.
I also added an enhancement of the API pfring_get_hash_filtering_rule_stats()
to return for how long a hash filtering rule is inactive.
The patch is based on pfring 6.6.0.

Since the fix and the enhancement are pretty short, I put them together in
the same patch.

I'll appreciate getting a feedback on this fix (I already sent it over a
month ago).

Thanks,
Amir
diff -Naur PF_RING-6.6.0-clean/kernel/linux/pf_ring.h 
PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h
--- PF_RING-6.6.0-clean/kernel/linux/pf_ring.h  2017-04-24 12:53:37.0 
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h2017-09-19 
13:51:42.709946000 +0300
@@ -604,6 +604,7 @@
 typedef struct {
   u_int64_t match;
   u_int64_t miss;
+  u_int32_t rule_inactivity;
 } hash_filtering_rule_stats;
 
 /* * */
diff -Naur PF_RING-6.6.0-clean/kernel/pf_ring.c 
PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c
--- PF_RING-6.6.0-clean/kernel/pf_ring.c2017-04-24 12:53:37.0 
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c  2017-09-19 
14:04:11.680328000 +0300
@@ -2372,7 +2372,6 @@
 && (hash_bucket->rule.host4_peer_b == rule->host4_peer_a)
 && (hash_bucket->rule.port_peer_a == rule->port_peer_b)
 && (hash_bucket->rule.port_peer_b == rule->port_peer_a {
-hash_bucket->rule.internals.jiffies_last_match = jiffies;
 return(1);
   } else
 return(0);
@@ -3291,6 +3290,7 @@
   while(hash_bucket != NULL) {
 if(hash_bucket_match(hash_bucket, hdr, 0, 0)) {
   hash_bucket->match++;
+  hash_bucket->rule.internals.jiffies_last_match = jiffies;
   hash_found = 1;
   break;
 } else
@@ -7146,6 +7146,7 @@
   hash_filtering_rule_stats hfrs;
   hfrs.match = bucket->match;
   hfrs.miss = pfr->sw_filtering_hash_miss;
+  hfrs.rule_inactivity = (u_int32_t)(jiffies_to_msecs(jiffies - 
bucket->rule.internals.jiffies_last_match) / 1000);
  rc = sizeof(hash_filtering_rule_stats);
   if(copy_to_user(optval, &hfrs, rc)) {
printk("[PF_RING] copy_to_user() failure\n");
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Clarification regarding pf_ring filtering using software hash filtering rule and the purge-idle feature

[Amir Kaduri]

Following further investigation of the subject, I guess that I found a bug
and attached a patch with a suggested fix.



Following the problematic design I've described in this thread, at least
for my application purposes, I’ve decided to improve one of the APIs to
return the inactivity time of a specified sw hash filtering rule.

That is, enhance the API pfring_get_hash_filtering_rule_stats() to return
this data as well. That way the userland API will have the chance to know
for how long a specified hash rule is idle.

During my implementation, I’m pretty sure that I found a bug in the way the
timings are updated. This makes me assume that the
pfring_purge_idle_hash_rules() API doesn’t work well (but didn’t test it).


Anyway, attached a patch, based on pfring 6.6.0,  that includes both the
fix and the enhancement for pfring_get_hash_filtering_rule_stats().

Since both of them are very short, I included them in the same patch file,
rather than splitting them into 2 separate patches (fix and enhancement).



Thanks,

Amir

On Thu, Sep 14, 2017 at 9:07 AM, Amir Kaduri  wrote:

> Another question related to the same issue:
>
> I wonder why the design is that the application responsible for
> managing/allocating the rule_id, and not the infrastructure.
> I would expect the API just to ask to add a rule, and the infrastructure
> will return 2 parameters:
> 1. Success/Failure
> 2. The allocated rule_id in case of success (that will be used for
> removing the rule)
>
> Thanks
>
> On Tue, Sep 12, 2017 at 8:34 PM, Amir Kaduri  wrote:
>
>> Hi,
>>
>>
>>
>> I would like to clarify a point regarding sw hash filtering rules and the
>> purge idle feature:
>>
>> In my application, I’m using sw hash filtering rules to block sessions.
>> For that purpose, I’m using the API pfring_handle_hash_filtering_rule()
>> to add and remove rules.
>>
>> When adding a rule with an ID of a rule that that has been already added,
>> this “add rule” action fails. This means that the application’s
>> responsibility to manage the rule IDs.
>>
>> In other words, the application’s responsibility to know which IDs are in
>> use and which are not, if it wants to avoid trial-and-error every time a
>> rule should be added.
>>
>> On the other hand, there is the API pfring_purge_idle_hash_rules() that
>> can assist in removing (purging) hash filtering rules that are inactive for
>> a while, automatically.
>>
>> So the situation is that pf_ring purges a filtering rule, and the
>> application is not aware of it.
>>
>>
>> For this reason:
>> 1.   The application continues to keep the relevant rule-id,
>> assuming it’s still active.
>> 2.   The application avoids using this rule-id for filtering other
>> sessions, since it assumes it’s still occupied.
>> 3.   If the session suddenly revives after the purge period arrives,
>> the application is confused since it’s not clear to it how come packets are
>> forwarded while there is an active rule.
>>
>> The above happens since there is no notification mechanism to the
>> userspace application, that a rule has been purged (removed).
>>
>>
>> Is the description correct or maybe I miss something critical in the APIs
>> usage?
>>
>>
>>
>> Thanks,
>>
>> Amir
>>
>
>
diff -Naur PF_RING-6.6.0-clean/kernel/linux/pf_ring.h 
PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h
--- PF_RING-6.6.0-clean/kernel/linux/pf_ring.h  2017-04-24 12:53:37.0 
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h2017-09-19 
13:51:42.709946000 +0300
@@ -604,6 +604,7 @@
 typedef struct {
   u_int64_t match;
   u_int64_t miss;
+  u_int32_t rule_inactivity;
 } hash_filtering_rule_stats;
 
 /* * */
diff -Naur PF_RING-6.6.0-clean/kernel/pf_ring.c 
PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c
--- PF_RING-6.6.0-clean/kernel/pf_ring.c2017-04-24 12:53:37.0 
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c  2017-09-19 
14:04:11.680328000 +0300
@@ -2372,7 +2372,6 @@
 && (hash_bucket->rule.host4_peer_b == rule->host4_peer_a)
 && (hash_bucket->rule.port_peer_a == rule->port_peer_b)
 && (hash_bucket->rule.port_peer_b == rule->port_peer_a {
-hash_bucket->rule.internals.jiffies_last_match = jiffies;
 return(1);
   } else
 return(0);
@@ -3291,6 +3290,7 @@
   while(hash_bucket != NULL) {
 if(hash_bucket_match(hash_bucket, hdr, 0, 0)) {
   hash_bucket->match++;
+  hash_bucket->rule.internals.jiffies_last_match = jiffies;
   hash_found = 1;
   break;
 } else
@@

Re: [Ntop-misc] Clarification regarding pf_ring filtering using software hash filtering rule and the purge-idle feature

[Amir Kaduri]

Another question related to the same issue:

I wonder why the design is that the application responsible for
managing/allocating the rule_id, and not the infrastructure.
I would expect the API just to ask to add a rule, and the infrastructure
will return 2 parameters:
1. Success/Failure
2. The allocated rule_id in case of success (that will be used for removing
the rule)

Thanks

On Tue, Sep 12, 2017 at 8:34 PM, Amir Kaduri  wrote:

> Hi,
>
>
>
> I would like to clarify a point regarding sw hash filtering rules and the
> purge idle feature:
>
> In my application, I’m using sw hash filtering rules to block sessions.
> For that purpose, I’m using the API pfring_handle_hash_filtering_rule()
> to add and remove rules.
>
> When adding a rule with an ID of a rule that that has been already added,
> this “add rule” action fails. This means that the application’s
> responsibility to manage the rule IDs.
>
> In other words, the application’s responsibility to know which IDs are in
> use and which are not, if it wants to avoid trial-and-error every time a
> rule should be added.
>
> On the other hand, there is the API pfring_purge_idle_hash_rules() that
> can assist in removing (purging) hash filtering rules that are inactive for
> a while, automatically.
>
> So the situation is that pf_ring purges a filtering rule, and the
> application is not aware of it.
>
>
> For this reason:
> 1.   The application continues to keep the relevant rule-id, assuming
> it’s still active.
> 2.   The application avoids using this rule-id for filtering other
> sessions, since it assumes it’s still occupied.
> 3.   If the session suddenly revives after the purge period arrives,
> the application is confused since it’s not clear to it how come packets are
> forwarded while there is an active rule.
>
> The above happens since there is no notification mechanism to the
> userspace application, that a rule has been purged (removed).
>
>
> Is the description correct or maybe I miss something critical in the APIs
> usage?
>
>
>
> Thanks,
>
> Amir
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Clarification regarding pf_ring filtering using software hash filtering rule and the purge-idle feature

[Amir Kaduri]

Hi,



I would like to clarify a point regarding sw hash filtering rules and the
purge idle feature:

In my application, I’m using sw hash filtering rules to block sessions. For
that purpose, I’m using the API pfring_handle_hash_filtering_rule() to add
and remove rules.

When adding a rule with an ID of a rule that that has been already added,
this “add rule” action fails. This means that the application’s
responsibility to manage the rule IDs.

In other words, the application’s responsibility to know which IDs are in
use and which are not, if it wants to avoid trial-and-error every time a
rule should be added.

On the other hand, there is the API pfring_purge_idle_hash_rules() that can
assist in removing (purging) hash filtering rules that are inactive for a
while, automatically.

So the situation is that pf_ring purges a filtering rule, and the
application is not aware of it.


For this reason:
1.   The application continues to keep the relevant rule-id, assuming
it’s still active.
2.   The application avoids using this rule-id for filtering other
sessions, since it assumes it’s still occupied.
3.   If the session suddenly revives after the purge period arrives,
the application is confused since it’s not clear to it how come packets are
forwarded while there is an active rule.

The above happens since there is no notification mechanism to the userspace
application, that a rule has been purged (removed).


Is the description correct or maybe I miss something critical in the APIs
usage?



Thanks,

Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] New suggested (and implemented) feature: pf_ring sw filters packets sampling

[Amir Kaduri]

I’ve added a new feature that I would like to share.



The pf_ring feature allows packets sampling of packets that belong to a sw
filtered session.

In other words, if a sw filter is applied, you will be able not to block
100% of the packets (till filtering rule removal). Some packets will be
allowed to “pass” the filter.

The use case is keeping the advantage of improved performance by filtering
most (e.g. 90%) of the traffic, while being able to get some (e.g. 10%) of
the traffic for analysis purposes. This will allow the application not be
100% blind.



This feature is not good for applications that are sensitive to
out-of-sequence packets.



Technical code details:

1.   New pf_ring param “filtering_sampling_segment_size” is set to 100
by default.

2.   New pf_ring userspace API int
pfring_set_sw_filtering_sampling_rate(pfring *ring, u_int32_t rate /* 0 =
no sampling */) was added.

This API should be used right after adding a new SW hash filtering rule,
with a “rate” parameter.

A rate=10 means that the 10 first packets of every segment of 100 filtered
packets of the session, will be forwarded.

If the API is not called, then the default rate=0, thus no change in
current behavior (i.e. 100% of the packets are filtered, since 0 filtered
packets are forwarded).

3.   I’ve had to add a new member variable to struct pf_ring_socket
which is ‘sw_filtering_hash_filtered’ that counts the actual filtered
packets.

Current ‘sw_filtering_hash_match’ is not good enough anymore for this
purpose since it counts the matches before excluding the forwarded packets,
but it has a great help

in calculating the sampled filtered packets.



Technical build/test details:

1.   Functionality was tested based on pf_ring version 6.6.0 (on CentOS
7)

2.   Attached patch was created based on latest pf_ring code-base
(Sep-7, 2017), and tested for compilation only. Not functionality.



If this feature will ever make it to the main code, it would be great.



Any comments are welcome.



Amir
diff -Naur PF_RING-dev.20170907/kernel/linux/pf_ring.h 
PF_RING-dev.20170907.sampling/kernel/linux/pf_ring.h
--- PF_RING-dev.20170907/kernel/linux/pf_ring.h 2017-09-07 13:02:53.0 
+0300
+++ PF_RING-dev.20170907.sampling/kernel/linux/pf_ring.h2017-09-07 
13:35:43.508272000 +0300
@@ -65,6 +65,7 @@
 #define SO_SET_POLL_WATERMARK117
 #define SO_SET_VIRTUAL_FILTERING_DEVICE  118
 #define SO_REHASH_RSS_PACKET 119
+#define SO_SET_SW_FILTERING_SAMPLING_RATE 120
 #define SO_SHUTDOWN_RING 124
 #define SO_PURGE_IDLE_RULES  125 /* inactivity (sec) */
 #define SO_SET_SOCKET_MODE   126
@@ -1078,6 +1079,7 @@
   sw_filtering_hash_bucket **sw_filtering_hash;
   u_int64_t sw_filtering_hash_match;
   u_int64_t sw_filtering_hash_miss;
+  u_int64_t sw_filtering_hash_filtered;
   u_int32_t num_sw_filtering_hash;
 
   /* Sw Filtering Rules - wildcard */
@@ -1087,6 +1089,8 @@
   /* Hw Filtering Rules */
   u_int16_t num_hw_filtering_rules;
   struct list_head hw_filtering_rules;
+  /* Sw Filtering - sampling rate */
+  u_int32_t sw_filtering_sampling_rate;
 
   /* Locks */
   atomic_t num_ring_users;
diff -Naur PF_RING-dev.20170907/kernel/pf_ring.c 
PF_RING-dev.20170907.sampling/kernel/pf_ring.c
--- PF_RING-dev.20170907/kernel/pf_ring.c   2017-09-07 13:02:53.0 
+0300
+++ PF_RING-dev.20170907.sampling/kernel/pf_ring.c  2017-09-07 
13:54:45.102938000 +0300
@@ -365,6 +365,7 @@
 static unsigned int force_ring_lock = 0;
 static unsigned int enable_debug = 0;
 static unsigned int transparent_mode = 0;
+static unsigned int filtering_sampling_segment_size = 100;
 static atomic_t ring_id_serial = ATOMIC_INIT(0);
 
 #if defined(RHEL_RELEASE_CODE)
@@ -382,6 +383,7 @@
 module_param(force_ring_lock, uint, 0644);
 module_param(enable_debug, uint, 0644);
 module_param(transparent_mode, uint, 0644);
+module_param(filtering_sampling_segment_size, uint, 0644);
 
 MODULE_PARM_DESC(min_num_slots, "Min number of ring slots");
 MODULE_PARM_DESC(perfect_rules_hash_size, "Perfect rules hash size");
@@ -397,6 +399,7 @@
 MODULE_PARM_DESC(enable_debug, "Set to 1 to enable PF_RING debug tracing into 
the syslog, 2 for more verbosity");
 MODULE_PARM_DESC(transparent_mode,
 "(deprecated)");
+MODULE_PARM_DESC(filtering_sampling_segment_size, "Size of filtered packets 
segment for sampling (currently, for sw hash filters only)");
 
 /* ** */
 
@@ -1360,20 +1363,21 @@
 
   if(m->private == NULL) {
 /* /proc/net/pf_ring/info */
-seq_printf(m, "PF_RING Version  : %s (%s)\n", RING_VERSION, 
GIT_REV);
-seq_printf(m, "Total rings  : %d\n", 
atomic_read(&ring_table_size));
+seq_printf(m, "PF_RING Version : %s (%s)\n", RING_VERSION, 
GIT_REV);
+seq_printf(m, "Total rings : %d\n", 
atomic_read(&ring_table_size));
 seq_printf(m, "\nStandard (non ZC) Options\n");
-

Re: [Ntop-misc] Deprecetaed bundle API

[Amir Kaduri]

Thanks

On Mon, Aug 21, 2017 at 2:00 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> copying those function is definitely the simplest solution and it should
> work fine for you,
> as of extended_hdr.parsed_header_len please just remove the if branch and
> delete any
> occurrence as that was the length of metadata when using kernel plugins
> (deprecated).
>
> Regards
> Alfredo
>
> On 21 Aug 2017, at 12:31, Amir Kaduri  wrote:
>
> Since its about 6 functions, a possible solution could be just copying
> them to the application as static functions. Not a perfect solution, but
> possible. If there is a better one (that is simple), I'll be glad to hear.
> Anyway, another thing in the same context:
> The (deprecated) pfcount_bundle.c example application, had a check "if
> (h->extended_hdr.parsed_header_len > 0)". "parsed_header_len" has been
> also removed from struct pfring_extended_pkthdr{} since pf_ring 6.6.0
> (on Oct 19, 2016). I assume it was calculated during packet parsing. Could
> you please suggest a simple alternative to this check, based on pf_ring
> 6.6.0?
> Thanks
>
> On Thu, Aug 17, 2017 at 5:11 PM, Amir Kaduri  wrote:
>
>> Small correction: Jul-15, 2016. Sorry.
>>
>> On Thu, Aug 17, 2017 at 5:04 PM, Amir Kaduri  wrote:
>>
>>> I have an application written in the past, that heavily consists on
>>> bundle API in general, and on pfcount_bundle example application
>>> specifically. These APIs were deprecated on Jul-15, 2017 and don't exist
>>> anymore in pf_ring 6.6.0.
>>> What is the easiest way to adjust the existing code in order to be able
>>> to compile and run based on pf_ring 6.6.0?
>>>
>>> Thanks
>>>
>>
>>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Deprecetaed bundle API

[Amir Kaduri]

Since its about 6 functions, a possible solution could be just copying them
to the application as static functions. Not a perfect solution, but
possible. If there is a better one (that is simple), I'll be glad to hear.
Anyway, another thing in the same context:
The (deprecated) pfcount_bundle.c example application, had a check "if
(h->extended_hdr.parsed_header_len > 0)". "parsed_header_len" has been also
removed from struct pfring_extended_pkthdr{} since pf_ring 6.6.0
(on Oct 19, 2016). I assume it was calculated during packet parsing. Could
you please suggest a simple alternative to this check, based on pf_ring
6.6.0?
Thanks

On Thu, Aug 17, 2017 at 5:11 PM, Amir Kaduri  wrote:

> Small correction: Jul-15, 2016. Sorry.
>
> On Thu, Aug 17, 2017 at 5:04 PM, Amir Kaduri  wrote:
>
>> I have an application written in the past, that heavily consists on
>> bundle API in general, and on pfcount_bundle example application
>> specifically. These APIs were deprecated on Jul-15, 2017 and don't exist
>> anymore in pf_ring 6.6.0.
>> What is the easiest way to adjust the existing code in order to be able
>> to compile and run based on pf_ring 6.6.0?
>>
>> Thanks
>>
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Deprecetaed bundle API

[Amir Kaduri]

Small correction: Jul-15, 2016. Sorry.

On Thu, Aug 17, 2017 at 5:04 PM, Amir Kaduri  wrote:

> I have an application written in the past, that heavily consists on bundle
> API in general, and on pfcount_bundle example application specifically.
> These APIs were deprecated on Jul-15, 2017 and don't exist anymore in
> pf_ring 6.6.0.
> What is the easiest way to adjust the existing code in order to be able to
> compile and run based on pf_ring 6.6.0?
>
> Thanks
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Deprecetaed bundle API

[Amir Kaduri]

I have an application written in the past, that heavily consists on bundle
API in general, and on pfcount_bundle example application specifically.
These APIs were deprecated on Jul-15, 2017 and don't exist anymore in
pf_ring 6.6.0.
What is the easiest way to adjust the existing code in order to be able to
compile and run based on pf_ring 6.6.0?

Thanks
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Using pf_ring on VMware (installed with e1000 drivers)

[Amir Kaduri]

Thanks.
Under the same details/conditions described in my question (i.e. pf_ring SW
hash filters only), is there any specific need to replace the standard
igb.ko and ixgbe.ko drivers with the ones that are provided by pf_ring?

On Wed, Aug 9, 2017 at 11:08 AM, Alfredo Cardigliano 
wrote:

> Hi Amir
> e1000 and e1000e support different adapters (the latter is for pcie
> adapters in essence).
>
> Alfredo
>
> > On 9 Aug 2017, at 08:03, Amir Kaduri  wrote:
> >
> > Hi,
> >
> > I'm using pf_ring 6.6.0 (for SW hash filtering only - no ZC) on CentOS 7
> on VMware.
> > All drivers are e1000. The pf_ring filtering seems to work.
> > Is there any benefit to remove the e1000 driver and install the e1000e
> drivers that comes with pf_ring?
> >
> > Thanks,
> > Amir
> > ___
> > Ntop-misc mailing list
> > Ntop-misc@listgateway.unipi.it
> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Using pf_ring on VMware (installed with e1000 drivers)

[Amir Kaduri]

Hi,

I'm using pf_ring 6.6.0 (for SW hash filtering only - no ZC) on CentOS 7 on
VMware.
All drivers are e1000. The pf_ring filtering seems to work.
Is there any benefit to remove the e1000 driver and install the e1000e
drivers that comes with pf_ring?

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Fail to rmmod pf_ring

[Amir Kaduri]

I've managed to rmmov the "used" pfring.ko by removing first the ixgbe.ko
and igb.ko drivers. Not sure how its related (since no process really uses
pf_ring) but currently its good enough for me.

Thanks

On Mon, Jul 10, 2017 at 4:22 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> you are "trying to replace the igb.ko" with?
> Please send us the output from dmesg.
>
> Alfredo
>
> 2017-07-10 13:22 GMT+02:00 Amir Kaduri :
>
>> No, not using ZC. But if it could be a reason and maybe I'm doing
>> something wrong during load, how can I check about zc?
>> Anyway, the background for this issue is that I'm trying to replace the
>> igb.ko during machine installation and I'm trying to overcome the special
>> situation that its used by the management interface.
>> To debug the situation, I fiddle with the machine so I'm not sure what
>> exactly brought me to the situation described above, so I probably
>> interested in a tweak (just to remove the pf_ring module) rather than a
>> proper solution.
>>
>> Amir
>>
>> On Mon, Jul 10, 2017 at 12:42 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Hi Amir
>>> did you load a ZC driver perhaps?
>>>
>>> Alfredo
>>>
>>> 2017-07-10 11:00 GMT+02:00 Amir Kaduri :
>>>
>>>> OS CentOS 7
>>>> pf_ring version: 6.6.0
>>>>
>>>> On Mon, Jul 10, 2017 at 11:57 AM, Amir Kaduri 
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm in a situation where I try to remove the pf_ring kernel module.
>>>>> "rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use"
>>>>> "lsmod | grep pf_ring" gives "pf_ring 1234009 20"
>>>>> In this case, I try to find which process uses pf_ring, in order to
>>>>> stop/kill it.
>>>>> I run "ls -l /proc/net/pf_ring/" to hopefully get the list of
>>>>> process-ids of the relevant processes. The problem is that the list gives
>>>>> only "dev" (subdir) "info" (file) and "stats" (subdir). There are no files
>>>>> like (for example) "/proc/net/pf_ring/12345-eth0.23" that can give me
>>>>> an indication who uses the pf_ring kernel module.
>>>>> Any advice on how can I force removing it or at least know who uses it?
>>>>>
>>>>> Thanks,
>>>>> Amir
>>>>>
>>>>
>>>>
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>
>>>
>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Fail to rmmod pf_ring

[Amir Kaduri]

No, not using ZC. But if it could be a reason and maybe I'm doing something
wrong during load, how can I check about zc?
Anyway, the background for this issue is that I'm trying to replace the
igb.ko during machine installation and I'm trying to overcome the special
situation that its used by the management interface.
To debug the situation, I fiddle with the machine so I'm not sure what
exactly brought me to the situation described above, so I probably
interested in a tweak (just to remove the pf_ring module) rather than a
proper solution.

Amir

On Mon, Jul 10, 2017 at 12:42 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> did you load a ZC driver perhaps?
>
> Alfredo
>
> 2017-07-10 11:00 GMT+02:00 Amir Kaduri :
>
>> OS CentOS 7
>> pf_ring version: 6.6.0
>>
>> On Mon, Jul 10, 2017 at 11:57 AM, Amir Kaduri 
>> wrote:
>>
>>> Hi,
>>>
>>> I'm in a situation where I try to remove the pf_ring kernel module.
>>> "rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use"
>>> "lsmod | grep pf_ring" gives "pf_ring 1234009 20"
>>> In this case, I try to find which process uses pf_ring, in order to
>>> stop/kill it.
>>> I run "ls -l /proc/net/pf_ring/" to hopefully get the list of
>>> process-ids of the relevant processes. The problem is that the list gives
>>> only "dev" (subdir) "info" (file) and "stats" (subdir). There are no files
>>> like (for example) "/proc/net/pf_ring/12345-eth0.23" that can give me
>>> an indication who uses the pf_ring kernel module.
>>> Any advice on how can I force removing it or at least know who uses it?
>>>
>>> Thanks,
>>> Amir
>>>
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Fail to rmmod pf_ring

[Amir Kaduri]

OS CentOS 7
pf_ring version: 6.6.0

On Mon, Jul 10, 2017 at 11:57 AM, Amir Kaduri  wrote:

> Hi,
>
> I'm in a situation where I try to remove the pf_ring kernel module.
> "rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use"
> "lsmod | grep pf_ring" gives "pf_ring 1234009 20"
> In this case, I try to find which process uses pf_ring, in order to
> stop/kill it.
> I run "ls -l /proc/net/pf_ring/" to hopefully get the list of process-ids
> of the relevant processes. The problem is that the list gives only "dev"
> (subdir) "info" (file) and "stats" (subdir). There are no files like (for
> example) "/proc/net/pf_ring/12345-eth0.23" that can give me an indication
> who uses the pf_ring kernel module.
> Any advice on how can I force removing it or at least know who uses it?
>
> Thanks,
> Amir
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Fail to rmmod pf_ring

[Amir Kaduri]

Hi,

I'm in a situation where I try to remove the pf_ring kernel module.
"rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use"
"lsmod | grep pf_ring" gives "pf_ring 1234009 20"
In this case, I try to find which process uses pf_ring, in order to
stop/kill it.
I run "ls -l /proc/net/pf_ring/" to hopefully get the list of process-ids
of the relevant processes. The problem is that the list gives only "dev"
(subdir) "info" (file) and "stats" (subdir). There are no files like (for
example) "/proc/net/pf_ring/12345-eth0.23" that can give me an indication
who uses the pf_ring kernel module.
Any advice on how can I force removing it or at least know who uses it?

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Question regarding filtering using software hash filtering rule

[Amir Kaduri]

Hi,

Apparently, it was an MTU issue.

Thanks (and sorry)

On Sun, Jun 4, 2017 at 12:29 PM, Alfredo Cardigliano
 wrote:
> Hi Amir
> could you provide a pcap and the commands (baed on our examples) to test what 
> you are doing?
>
> Thank you
> Alfredo
>
>> On 4 Jun 2017, at 10:45, Amir Kaduri  wrote:
>>
>> Hello,
>>
>> I have a situation where not all of the received packets are counted
>> as filtered, and I would like to better understand why.
>> To better understand it, I've done a controlled experiment, where
>> after the software hash filtering rule was added on a specific
>> 5-tuple, I send exactly 5000 packets on the same 5-tuple. All packets
>> received by the same ring.
>> When I look at the ring info file, I see that "Sw Filt Hash Match"
>> increases by 4955 exactly. (Its the same number whenever I repeat the
>> experiment on the same filter). Which means 45 packets are not
>> counted. No other statistics parameter can explain the missing 45
>> packets, not in the ring info file (e.g. "Sw Filt Hash Miss") and not
>> by using "ethtool -S" on the interface (although by using ethtool -S I
>> see that all 5000 packets are definitely received to the NIC).
>> When looking deeply into the replayed pcap, I see a high correlation
>> between the number of missing packets (i.e. 45) and the number of
>> packets that are "TCP Segment of a reassembled PDU" (by wireshark).
>> My rss rehash set to 1.
>>
>> Questions:
>> 1. Any explanation for packets that are not counted by  "Sw Filt Hash
>> Match" (and not by any other parameter)?
>> 2. Does the "TCP Segment of a reassembled PDU" could explain it somehow?
>> 3. Could it be a behavioral change compared to previous pf_ring
>> versions (e.g. 6.0.3)?
>>
>> Thanks,
>> Amir
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Question regarding filtering using software hash filtering rule

[Amir Kaduri]

Hello,

I have a situation where not all of the received packets are counted
as filtered, and I would like to better understand why.
To better understand it, I've done a controlled experiment, where
after the software hash filtering rule was added on a specific
5-tuple, I send exactly 5000 packets on the same 5-tuple. All packets
received by the same ring.
When I look at the ring info file, I see that "Sw Filt Hash Match"
increases by 4955 exactly. (Its the same number whenever I repeat the
experiment on the same filter). Which means 45 packets are not
counted. No other statistics parameter can explain the missing 45
packets, not in the ring info file (e.g. "Sw Filt Hash Miss") and not
by using "ethtool -S" on the interface (although by using ethtool -S I
see that all 5000 packets are definitely received to the NIC).
When looking deeply into the replayed pcap, I see a high correlation
between the number of missing packets (i.e. 45) and the number of
packets that are "TCP Segment of a reassembled PDU" (by wireshark).
My rss rehash set to 1.

Questions:
1. Any explanation for packets that are not counted by  "Sw Filt Hash
Match" (and not by any other parameter)?
2. Does the "TCP Segment of a reassembled PDU" could explain it somehow?
3. Could it be a behavioral change compared to previous pf_ring
versions (e.g. 6.0.3)?

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()

[Amir Kaduri]

Ok, thanks.

On Thu, Jun 1, 2017 at 12:16 PM, Alfredo Cardigliano
 wrote:
> Hi Amir
> regardless of the user input, when a packet is received  ah hash is computed 
> calling parse_pkt -> parse_raw_pkt -> hash_pkt_header ->  hash_pkt
> if the packet is not an IP packet (ARP or whatever), ip_version is 0 and your 
> message is printed, for every packet.
>
> Alfredo
>
>> On 1 Jun 2017, at 11:00, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> I'm aware that this function is called per packet, but if you think of
>> it deeper, then if everything is configured correctly by the user,
>> ip_version should give 4 or 6 only, thus it will never rich the "else"
>> I suggest to add and it will never actually add any redundant check
>> nor print per packet. The only time the warning will be printed is
>> only if ip_version will not be 4 or 6 and its great. It clearly warns
>> about a problem (whether its not set or set to anything else than 4 or
>> 6).
>> Just to make sure you got me right, the code with the fix should look like 
>> that:
>> if (ip_version == 4)
>>hash += host_peer_a.v4 + host_peer_b.v4;
>> else if (ip_version == 6)
>>hash += (<... Omitted long condition >);
>> else
>>printk("Warning: ip_version (currently equals %d) should be 4 or 6
>> only\n", ip_version);
>>
>> So again, if you take a moment to think of it, this warning should
>> never cost you anything if everything is configured well, but it warns
>> if anything went wrong.
>>
>> Thanks,
>> Amir
>>
>> On Thu, Jun 1, 2017 at 11:39 AM, Alfredo Cardigliano
>>  wrote:
>>> Hi Amir
>>> hash_pkt is called per-packet, we want to a void a printk per packet to 
>>> avoid flooding dmesg and slowing doen packet processing,
>>> if ip_version comes from user input, sanity check should happen in 
>>> userspace in my opinion.
>>>
>>> Regards
>>> Alfredo
>>>
>>>> On 1 Jun 2017, at 07:38, Amir Kaduri  wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> This is the exact location of the function:
>>>> https://github.com/ntop/PF_RING/blob/dev/kernel/pf_ring.c#L1794
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Mon, May 29, 2017 at 7:28 PM, Alfredo Cardigliano
>>>>  wrote:
>>>>> Hi Amir
>>>>> what is the file location you are talking about?
>>>>>
>>>>> Alfredo
>>>>>
>>>>>> On 29 May 2017, at 18:23, Amir Kaduri  wrote:
>>>>>>
>>>>>> In function hash_pkt(), there is a if-else-if statement based on
>>>>>> ip_version. If ip_version is 0, the hash won't include the ipaddress.
>>>>>> Since the ip_version might come from the user input, I suggest adding
>>>>>> an "else" and issue a warning in case ip_version wasn't set.
>>>>>> ___
>>>>>> Ntop-misc mailing list
>>>>>> Ntop-misc@listgateway.unipi.it
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>>
>>>>>
>>>>> ___
>>>>> Ntop-misc mailing list
>>>>> Ntop-misc@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()

[Amir Kaduri]

Hi Alfredo,

I'm aware that this function is called per packet, but if you think of
it deeper, then if everything is configured correctly by the user,
ip_version should give 4 or 6 only, thus it will never rich the "else"
I suggest to add and it will never actually add any redundant check
nor print per packet. The only time the warning will be printed is
only if ip_version will not be 4 or 6 and its great. It clearly warns
about a problem (whether its not set or set to anything else than 4 or
6).
Just to make sure you got me right, the code with the fix should look like that:
if (ip_version == 4)
hash += host_peer_a.v4 + host_peer_b.v4;
else if (ip_version == 6)
hash += (<... Omitted long condition >);
else
printk("Warning: ip_version (currently equals %d) should be 4 or 6
only\n", ip_version);

So again, if you take a moment to think of it, this warning should
never cost you anything if everything is configured well, but it warns
if anything went wrong.

Thanks,
Amir

On Thu, Jun 1, 2017 at 11:39 AM, Alfredo Cardigliano
 wrote:
> Hi Amir
> hash_pkt is called per-packet, we want to a void a printk per packet to avoid 
> flooding dmesg and slowing doen packet processing,
> if ip_version comes from user input, sanity check should happen in userspace 
> in my opinion.
>
> Regards
> Alfredo
>
>> On 1 Jun 2017, at 07:38, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> This is the exact location of the function:
>> https://github.com/ntop/PF_RING/blob/dev/kernel/pf_ring.c#L1794
>>
>> Thanks,
>> Amir
>>
>> On Mon, May 29, 2017 at 7:28 PM, Alfredo Cardigliano
>>  wrote:
>>> Hi Amir
>>> what is the file location you are talking about?
>>>
>>> Alfredo
>>>
>>>> On 29 May 2017, at 18:23, Amir Kaduri  wrote:
>>>>
>>>> In function hash_pkt(), there is a if-else-if statement based on
>>>> ip_version. If ip_version is 0, the hash won't include the ipaddress.
>>>> Since the ip_version might come from the user input, I suggest adding
>>>> an "else" and issue a warning in case ip_version wasn't set.
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()

[Amir Kaduri]

Hi Alfredo,

This is the exact location of the function:
https://github.com/ntop/PF_RING/blob/dev/kernel/pf_ring.c#L1794

Thanks,
Amir

On Mon, May 29, 2017 at 7:28 PM, Alfredo Cardigliano
 wrote:
> Hi Amir
> what is the file location you are talking about?
>
> Alfredo
>
>> On 29 May 2017, at 18:23, Amir Kaduri  wrote:
>>
>> In function hash_pkt(), there is a if-else-if statement based on
>> ip_version. If ip_version is 0, the hash won't include the ipaddress.
>> Since the ip_version might come from the user input, I suggest adding
>> an "else" and issue a warning in case ip_version wasn't set.
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()

[Amir Kaduri]

In function hash_pkt(), there is a if-else-if statement based on
ip_version. If ip_version is 0, the hash won't include the ipaddress.
Since the ip_version might come from the user input, I suggest adding
an "else" and issue a warning in case ip_version wasn't set.
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Question about pf_ring installation rpm (pf_ring 6.6.0)

[Amir Kaduri]

Hi Alfredo,

You are correct, and this is exactly what I've tried to explain.
The origin of the problem is in the source-control system that I use,
and its probably not a problem in the spec file.

Since systemd is sensitive to the permissions of these file, I just
suggested maybe to copy them using "install -m 644" instead of "cp" to
make things easier for others in case they have similar problems.

If you don't think that it should be done like that in pfring.spec.in,
its a reasonable decision. No problem. Its just a suggestion.

Thanks,
Amir

On Tue, May 23, 2017 at 1:37 PM, Alfredo Cardigliano
 wrote:
> Hi Amir
> I double checked in our lab and pf_ring.service is installed without 
> execution rights,
> could you provide more details about your installation? I think you should 
> fix something in your installation process.
>
> Best Regards
> Alfredo
>
>> On 22 May 2017, at 10:23, Amir Kaduri  wrote:
>>
>> Apparently, systemd doesn't like that the pf_ring.service and
>> cluster.service files have executable bit set and keeps complaining in
>> /var/log/messages.
>> This is just a warning but it spams the file.
>> Example:
>> var/log/messages-20170521:May 21 07:28:18 localhost systemd:
>> Configuration file /etc/systemd/system/pf_ring.service is marked
>> executable. Please remove executable permission bits. Proceeding
>> anyway.
>>
>> The executable bit is set probably due to my source-control system,
>> and probably not a problem that many will suffer from.
>>
>> Still, you can consider changing command "cp" to "install -m 644" as follows:
>>
>> In file package/rpm/pfring.spec.in:
>>
>> From:
>> cp $HOME/PF_RING/package/etc/systemd/system/pf_ring.service
>> $RPM_BUILD_ROOT/etc/systemd/system
>> cp $HOME/PF_RING/package/etc/systemd/system/cluster.service
>> $RPM_BUILD_ROOT/etc/systemd/system
>>
>> To:
>> install -m 644 $HOME/PF_RING/package/etc/systemd/system/pf_ring.service
>> $RPM_BUILD_ROOT/etc/systemd/system
>> install -m 644 $HOME/PF_RING/package/etc/systemd/system/cluster.service
>> $RPM_BUILD_ROOT/etc/systemd/system
>>
>> Thanks,
>> Amir
>>
>>
>> On Wed, May 17, 2017 at 2:20 PM, Amir Kaduri  wrote:
>>> Thanks for the prompt reply.
>>> Amir
>>>
>>> On Tue, May 16, 2017 at 12:09 PM, Alfredo Cardigliano
>>>  wrote:
>>>> Hi Amir
>>>>
>>>>> On 16 May 2017, at 10:44, Amir Kaduri  wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I assume that I miss a minor detail, but I'll appreciate explaining
>>>>> how the following files are deployed in the right place on the machine
>>>>> when the pf_ring installation rpm is installed:
>>>>> 1. /usr/local/pfring/kernel/pf_ring.ko:
>>>>>   In source file ../package/rpm/pfring.spec.in line 37, the copy
>>>>> command is commented out.
>>>>
>>>> DKMS build and install the module when installing the package, I added 
>>>> some comments to the spec.
>>>> This is where the module should be installed by dkms:
>>>>
>>>> /usr/lib/modules/`uname -r`/weak-updates/pf_ring.ko
>>>>
>>>>> 2. /etc/systemd/system/pf_ring.service:
>>>>>   The file itself appears in the source code under
>>>>> ../package/etc/systemd/system/pf_ring.service,
>>>>>   but it doesn't appear in source file ../package/rpm/pfring.spec.in
>>>>
>>>> Added to the spec (it was installed for ubuntu only)
>>>>
>>>> Alfredo
>>>>
>>>>>
>>>>> Thanks,
>>>>> Amir
>>>>> ___
>>>>> Ntop-misc mailing list
>>>>> Ntop-misc@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>
>>>>
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Question about pf_ring installation rpm (pf_ring 6.6.0)

[Amir Kaduri]

Apparently, systemd doesn't like that the pf_ring.service and
cluster.service files have executable bit set and keeps complaining in
/var/log/messages.
This is just a warning but it spams the file.
Example:
var/log/messages-20170521:May 21 07:28:18 localhost systemd:
Configuration file /etc/systemd/system/pf_ring.service is marked
executable. Please remove executable permission bits. Proceeding
anyway.

The executable bit is set probably due to my source-control system,
and probably not a problem that many will suffer from.

Still, you can consider changing command "cp" to "install -m 644" as follows:

In file package/rpm/pfring.spec.in:

From:
cp $HOME/PF_RING/package/etc/systemd/system/pf_ring.service
$RPM_BUILD_ROOT/etc/systemd/system
cp $HOME/PF_RING/package/etc/systemd/system/cluster.service
$RPM_BUILD_ROOT/etc/systemd/system

To:
install -m 644 $HOME/PF_RING/package/etc/systemd/system/pf_ring.service
$RPM_BUILD_ROOT/etc/systemd/system
install -m 644 $HOME/PF_RING/package/etc/systemd/system/cluster.service
$RPM_BUILD_ROOT/etc/systemd/system

Thanks,
Amir


On Wed, May 17, 2017 at 2:20 PM, Amir Kaduri  wrote:
> Thanks for the prompt reply.
> Amir
>
> On Tue, May 16, 2017 at 12:09 PM, Alfredo Cardigliano
>  wrote:
>> Hi Amir
>>
>>> On 16 May 2017, at 10:44, Amir Kaduri  wrote:
>>>
>>> Hello,
>>>
>>> I assume that I miss a minor detail, but I'll appreciate explaining
>>> how the following files are deployed in the right place on the machine
>>> when the pf_ring installation rpm is installed:
>>> 1. /usr/local/pfring/kernel/pf_ring.ko:
>>>In source file ../package/rpm/pfring.spec.in line 37, the copy
>>> command is commented out.
>>
>> DKMS build and install the module when installing the package, I added some 
>> comments to the spec.
>> This is where the module should be installed by dkms:
>>
>> /usr/lib/modules/`uname -r`/weak-updates/pf_ring.ko
>>
>>> 2. /etc/systemd/system/pf_ring.service:
>>>The file itself appears in the source code under
>>> ../package/etc/systemd/system/pf_ring.service,
>>>but it doesn't appear in source file ../package/rpm/pfring.spec.in
>>
>> Added to the spec (it was installed for ubuntu only)
>>
>> Alfredo
>>
>>>
>>> Thanks,
>>> Amir
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Question about pf_ring installation rpm (pf_ring 6.6.0)

[Amir Kaduri]

Thanks for the prompt reply.
Amir

On Tue, May 16, 2017 at 12:09 PM, Alfredo Cardigliano
 wrote:
> Hi Amir
>
>> On 16 May 2017, at 10:44, Amir Kaduri  wrote:
>>
>> Hello,
>>
>> I assume that I miss a minor detail, but I'll appreciate explaining
>> how the following files are deployed in the right place on the machine
>> when the pf_ring installation rpm is installed:
>> 1. /usr/local/pfring/kernel/pf_ring.ko:
>>In source file ../package/rpm/pfring.spec.in line 37, the copy
>> command is commented out.
>
> DKMS build and install the module when installing the package, I added some 
> comments to the spec.
> This is where the module should be installed by dkms:
>
> /usr/lib/modules/`uname -r`/weak-updates/pf_ring.ko
>
>> 2. /etc/systemd/system/pf_ring.service:
>>The file itself appears in the source code under
>> ../package/etc/systemd/system/pf_ring.service,
>>but it doesn't appear in source file ../package/rpm/pfring.spec.in
>
> Added to the spec (it was installed for ubuntu only)
>
> Alfredo
>
>>
>> Thanks,
>> Amir
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Question about pf_ring installation rpm (pf_ring 6.6.0)

[Amir Kaduri]

Hello,

I assume that I miss a minor detail, but I'll appreciate explaining
how the following files are deployed in the right place on the machine
when the pf_ring installation rpm is installed:
1. /usr/local/pfring/kernel/pf_ring.ko:
In source file ../package/rpm/pfring.spec.in line 37, the copy
command is commented out.
2. /etc/systemd/system/pf_ring.service:
The file itself appears in the source code under
../package/etc/systemd/system/pf_ring.service,
but it doesn't appear in source file ../package/rpm/pfring.spec.in

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] ring_id is declared with 32 bit, but returned with 16 bit

[Amir Kaduri]

Excellent!

Thanks,
Amir

On Tue, Aug 16, 2016 at 11:00 AM, Alfredo Cardigliano 
wrote:

> Hi Amir
> fixed in latest code on github, thank you for reporting.
>
> Best Regards
> Alfredo
>
> On 15 Aug 2016, at 13:20, Amir Kaduri  wrote:
>
> I found a bug in my code, that reveals a potential bug in pf_ring code:
>
> In user-mode, I’m reading ring statistics by parsing files under
> /proc/net/pf_ring/ in the format -..
>
> For example, /proc/5998/net/pf_ring/2851-eth8.251768
>
> The problem is that the ring_id in these files is generated in 32 bit,
> while the API that returns them uses 16 bit type.
>
> This causes the ring_id to wraparound 2^16, thus not to find the ring file
> in user-mode.
>
> See for example function void ring_proc_add(struct pf_ring_socket *pfr),
> that pointer pfr of type struct pf_ring_socket
>
> owns member ring_id of type u_int32_t, while API function
> pfring_get_ring_id(pfring *ring), that I use in user-mode
>
> to access the same file, returns type u_int16_t.
>
> It also seen very clearly in function pfring_mod_get_ring_id(), that
> declares u_int32_t id and returns u_int16_t:
>
> u_int16_t pfring_mod_get_ring_id(pfring *ring) {
>
>   u_int32_t id;
>
>   socklen_t len = sizeof(id);
>
>   int rc = getsockopt(ring->fd, 0, SO_GET_RING_ID, &id, &len);
>
>   return((rc == 0) ? id : -1);
>
> }
>
>
> The above code is similar to latest pf_ring code in github.
>
>
> Please confirm that this is indeed not the intention (i.e. bug).
>
>
> Thanks,
>
> Amir
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] ring_id is declared with 32 bit, but returned with 16 bit

[Amir Kaduri]

Small correction about the example ring filename. It should be:
 /proc/net/pf_ring/2851-eth8.251768

Amir

On Mon, Aug 15, 2016 at 2:20 PM, Amir Kaduri  wrote:

> I found a bug in my code, that reveals a potential bug in pf_ring code:
>
> In user-mode, I’m reading ring statistics by parsing files under
> /proc/net/pf_ring/ in the format -..
>
> For example, /proc/5998/net/pf_ring/2851-eth8.251768
>
> The problem is that the ring_id in these files is generated in 32 bit,
> while the API that returns them uses 16 bit type.
>
> This causes the ring_id to wraparound 2^16, thus not to find the ring file
> in user-mode.
>
> See for example function void ring_proc_add(struct pf_ring_socket *pfr),
> that pointer pfr of type struct pf_ring_socket
>
> owns member ring_id of type u_int32_t, while API function
> pfring_get_ring_id(pfring *ring), that I use in user-mode
>
> to access the same file, returns type u_int16_t.
>
> It also seen very clearly in function pfring_mod_get_ring_id(), that
> declares u_int32_t id and returns u_int16_t:
>
> u_int16_t pfring_mod_get_ring_id(pfring *ring) {
>
>   u_int32_t id;
>
>   socklen_t len = sizeof(id);
>
>   int rc = getsockopt(ring->fd, 0, SO_GET_RING_ID, &id, &len);
>
>   return((rc == 0) ? id : -1);
>
> }
>
>
>
> The above code is similar to latest pf_ring code in github.
>
>
>
> Please confirm that this is indeed not the intention (i.e. bug).
>
>
> Thanks,
>
> Amir
>
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] ring_id is declared with 32 bit, but returned with 16 bit

[Amir Kaduri]

I found a bug in my code, that reveals a potential bug in pf_ring code:

In user-mode, I’m reading ring statistics by parsing files under
/proc/net/pf_ring/ in the format -..

For example, /proc/5998/net/pf_ring/2851-eth8.251768

The problem is that the ring_id in these files is generated in 32 bit,
while the API that returns them uses 16 bit type.

This causes the ring_id to wraparound 2^16, thus not to find the ring file
in user-mode.

See for example function void ring_proc_add(struct pf_ring_socket *pfr),
that pointer pfr of type struct pf_ring_socket

owns member ring_id of type u_int32_t, while API function
pfring_get_ring_id(pfring *ring), that I use in user-mode

to access the same file, returns type u_int16_t.

It also seen very clearly in function pfring_mod_get_ring_id(), that
declares u_int32_t id and returns u_int16_t:

u_int16_t pfring_mod_get_ring_id(pfring *ring) {

  u_int32_t id;

  socklen_t len = sizeof(id);

  int rc = getsockopt(ring->fd, 0, SO_GET_RING_ID, &id, &len);

  return((rc == 0) ? id : -1);

}



The above code is similar to latest pf_ring code in github.



Please confirm that this is indeed not the intention (i.e. bug).


Thanks,

Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Questions regarding pf_ring software hash filters

[Amir Kaduri]

Hi,

I'm using pf_ring software hash filters in my user-space process.
1. Is there a possibility that while the process is down (e.g. by "kill -9"
or crash), the filters are still active in the pf_ring driver, so that when
a new instance of the same process is up, traffic is still filtered by the
rules of its own previous instance?
2. What is the easiest way or the right way to have a list of all active sw
hash filters, not within the same process that put them. Preferably by
using some command-line script.

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Using SW hash filters without replacing the nic driver

[Amir Kaduri]

Hi,

When using SW hash filters only, without ZC or anything like that, is it a
must to replace also the nic driver (igb or ixgbe) to a "pfringaware" one?
I see that without replacing the nic driver, SW hash filters still work ok,
and wonder whether there is anything else I should be aware of.

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi Alfredo,

Even if these two issues are not going to be fixed soon, I just want to
know that you are aware of them.

Thanks,
Amir

On Tue, Dec 8, 2015 at 9:53 AM, Amir Kaduri  wrote:

> Hi,
>
> Another problem in the counters (that were added to
> function add_skb_to_ring()) is that it doesn't refer to rss-rehash:
> If rss-rehash is disabled, and there are 2 rings involved in a session
> (each for a direction), and each of them holds a sw-hash-rule,
> then the sw_filtering_hash_match counter will be incremented in both of
> them.
> For example: Assume rss-rehash is disabled, and the machine receives
> 10,000 packets of a single session (5000 packets of each direction).
> Each ring actually received 5000 packets, but each of them will report
> 10,000 packets filtered which is a total of 20,000 packets.
>
> So to summarize the problems in the fix in add_skb_to_ring():
> 1. On miss, each ring that has at least 1 rule will increment the counter
> ("Sw Filt Hash Miss" in /proc)
> 2. On match, when rss is disabled, both rings of both directions will
> increment their counter ("Sw Filt Hash Match" in /proc)
>
> Thanks,
> Amir
>
>
>
>
> On Mon, Dec 7, 2015 at 2:10 PM, Amir Kaduri  wrote:
>
>> Hi,
>>
>> In regard of the problem that missed packets counted on all rings instead
>> of on the relevant ring only:
>> We use a quite simple configuration: We use PF_RING_Aware, splitting the
>> incoming traffic to multiple queues (just like the tester
>> pfcount_multichannel does),
>> currently we have 2 ixgbe interfaces, and the packets physically arrive
>> to only one of them.
>>
>> Amir
>>
>> On Thu, Dec 3, 2015 at 3:41 PM, Alfredo Cardigliano > > wrote:
>>
>>>
>>> On 03 Dec 2015, at 13:44, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>> (1) Per your fix to update /proc/net/pf_ring with match+miss+rules
>>> statistics of sw hash rules, I've noticed the following:
>>> A received packet that doesn't match any rule, will cause an increase of
>>> the counter ("Sw Filt Hash Miss") of all rings that have at least 1
>>> rule, rather than the relevant ring only.
>>> I assume its not the intention, right?
>>>
>>>
>>> Are all those ring bound to the same interface the packet is coming
>>> from? Please provide more details about the configuration.
>>>
>>> (2) ethtool have also the "fdir_overflow" property. Is it possible to
>>> add this statistics to /proc per ring, together with the other statistics
>>> you've added?
>>>   (Note that I just want to know if pf_ring have enough info to do
>>> it or not).
>>>
>>>
>>> pf_ring does not usually use those info, they need to be retrieved
>>> somehow: do you really need the same stats in /proc?
>>>
>>> Alfredo
>>>
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>>
>>>> On 17 Nov 2015, at 15:54, Amir Kaduri  wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Thanks for that.
>>>> 1. stats per socket means stats per ring, correct?
>>>>
>>>>
>>>> Yes
>>>>
>>>> 2. I would like to know the best way to read this info from /proc.
>>>> Is there any dedicated API for that?
>>>>     I took a look at /proc on a machine running an application using
>>>> pf_ring, and found many files containing
>>>> the stats that appear in function proc_get_info(..). I would like
>>>> to better understand the convention of files
>>>>     to read, in order to have calculate accurate aggregated stats.
>>>>
>>>>
>>>> /proc/net/pf_ring/* contains one file per ring, the name contains the
>>>> process pid and ring id.
>>>>
>>>> Alfredo
>>>>
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano <
>>>> cardigli...@ntop.org> wrote:
>>>>
>>>>> Added to git code, please note stats are per socket, not per interface.
>>>>>
>>>>> Alfredo
>>>>>
>>>>> On 16 Nov 2015, at 13:48, Amir Kaduri  wrote:
>>>>>
>>>>> Hi Alfredo,
>>>>>
>>>>> Having it per interface u

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi,

Another problem in the counters (that were added to
function add_skb_to_ring()) is that it doesn't refer to rss-rehash:
If rss-rehash is disabled, and there are 2 rings involved in a session
(each for a direction), and each of them holds a sw-hash-rule,
then the sw_filtering_hash_match counter will be incremented in both of
them.
For example: Assume rss-rehash is disabled, and the machine receives 10,000
packets of a single session (5000 packets of each direction).
Each ring actually received 5000 packets, but each of them will report
10,000 packets filtered which is a total of 20,000 packets.

So to summarize the problems in the fix in add_skb_to_ring():
1. On miss, each ring that has at least 1 rule will increment the counter
("Sw Filt Hash Miss" in /proc)
2. On match, when rss is disabled, both rings of both directions will
increment their counter ("Sw Filt Hash Match" in /proc)

Thanks,
Amir




On Mon, Dec 7, 2015 at 2:10 PM, Amir Kaduri  wrote:

> Hi,
>
> In regard of the problem that missed packets counted on all rings instead
> of on the relevant ring only:
> We use a quite simple configuration: We use PF_RING_Aware, splitting the
> incoming traffic to multiple queues (just like the tester
> pfcount_multichannel does),
> currently we have 2 ixgbe interfaces, and the packets physically arrive to
> only one of them.
>
> Amir
>
> On Thu, Dec 3, 2015 at 3:41 PM, Alfredo Cardigliano 
> wrote:
>
>>
>> On 03 Dec 2015, at 13:44, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> (1) Per your fix to update /proc/net/pf_ring with match+miss+rules
>> statistics of sw hash rules, I've noticed the following:
>> A received packet that doesn't match any rule, will cause an increase of
>> the counter ("Sw Filt Hash Miss") of all rings that have at least 1
>> rule, rather than the relevant ring only.
>> I assume its not the intention, right?
>>
>>
>> Are all those ring bound to the same interface the packet is coming from?
>> Please provide more details about the configuration.
>>
>> (2) ethtool have also the "fdir_overflow" property. Is it possible to add
>> this statistics to /proc per ring, together with the other statistics
>> you've added?
>>   (Note that I just want to know if pf_ring have enough info to do it
>> or not).
>>
>>
>> pf_ring does not usually use those info, they need to be retrieved
>> somehow: do you really need the same stats in /proc?
>>
>> Alfredo
>>
>>
>> Thanks,
>> Amir
>>
>> On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>> On 17 Nov 2015, at 15:54, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Thanks for that.
>>> 1. stats per socket means stats per ring, correct?
>>>
>>>
>>> Yes
>>>
>>> 2. I would like to know the best way to read this info from /proc.
>>> Is there any dedicated API for that?
>>> I took a look at /proc on a machine running an application using
>>> pf_ring, and found many files containing
>>> the stats that appear in function proc_get_info(..). I would like to
>>> better understand the convention of files
>>> to read, in order to have calculate accurate aggregated stats.
>>>
>>>
>>> /proc/net/pf_ring/* contains one file per ring, the name contains the
>>> process pid and ring id.
>>>
>>> Alfredo
>>>
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>> Added to git code, please note stats are per socket, not per interface.
>>>>
>>>> Alfredo
>>>>
>>>> On 16 Nov 2015, at 13:48, Amir Kaduri  wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Having it per interface under /proc is excellent.
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <
>>>> cardigli...@ntop.org> wrote:
>>>>
>>>>>
>>>>> On 16 Nov 2015, at 10:53, Amir Kaduri  wrote:
>>>>>
>>>>> Hi Alfredo,
>>>>>
>>>>>
>>>>> I reviewed the implementation of the feature (great!) and I would like
>>>>> to clarify please:
>>>>>
>>>>> 1. If I understood it correctly, the number of matched packets is per

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi,

In regard of the problem that missed packets counted on all rings instead
of on the relevant ring only:
We use a quite simple configuration: We use PF_RING_Aware, splitting the
incoming traffic to multiple queues (just like the tester
pfcount_multichannel does),
currently we have 2 ixgbe interfaces, and the packets physically arrive to
only one of them.

Amir

On Thu, Dec 3, 2015 at 3:41 PM, Alfredo Cardigliano 
wrote:

>
> On 03 Dec 2015, at 13:44, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> (1) Per your fix to update /proc/net/pf_ring with match+miss+rules
> statistics of sw hash rules, I've noticed the following:
> A received packet that doesn't match any rule, will cause an increase of
> the counter ("Sw Filt Hash Miss") of all rings that have at least 1 rule,
> rather than the relevant ring only.
> I assume its not the intention, right?
>
>
> Are all those ring bound to the same interface the packet is coming from?
> Please provide more details about the configuration.
>
> (2) ethtool have also the "fdir_overflow" property. Is it possible to add
> this statistics to /proc per ring, together with the other statistics
> you've added?
>   (Note that I just want to know if pf_ring have enough info to do it
> or not).
>
>
> pf_ring does not usually use those info, they need to be retrieved
> somehow: do you really need the same stats in /proc?
>
> Alfredo
>
>
> Thanks,
> Amir
>
> On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano  > wrote:
>
>>
>> On 17 Nov 2015, at 15:54, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> Thanks for that.
>> 1. stats per socket means stats per ring, correct?
>>
>>
>> Yes
>>
>> 2. I would like to know the best way to read this info from /proc.
>> Is there any dedicated API for that?
>> I took a look at /proc on a machine running an application using
>> pf_ring, and found many files containing
>> the stats that appear in function proc_get_info(..). I would like to
>> better understand the convention of files
>> to read, in order to have calculate accurate aggregated stats.
>>
>>
>> /proc/net/pf_ring/* contains one file per ring, the name contains the
>> process pid and ring id.
>>
>> Alfredo
>>
>>
>> Thanks,
>> Amir
>>
>> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Added to git code, please note stats are per socket, not per interface.
>>>
>>> Alfredo
>>>
>>> On 16 Nov 2015, at 13:48, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Having it per interface under /proc is excellent.
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>>
>>>> On 16 Nov 2015, at 10:53, Amir Kaduri  wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>>
>>>> I reviewed the implementation of the feature (great!) and I would like
>>>> to clarify please:
>>>>
>>>> 1. If I understood it correctly, the number of matched packets is per
>>>> given rule, while the number of missed packets is per ring (and not per the
>>>> given rule). Is it correct, or am I wrong?
>>>>
>>>>  If I’m correct, can I provide a dummy rule and get the number of
>>>> missed packets on the given ring?
>>>>
>>>> Correct
>>>>
>>>> 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX
>>>> | grep fdir_match” you actually get the aggregation of missed/matched
>>>> packets on the interface (ethX).
>>>>
>>>>  Is it possible to get the pf_ring statistics aggregated as well,
>>>> per interface, or at least per ring?
>>>>
>>>> Is it ok for you if we write this stats under /proc instead of adding
>>>> another API call?
>>>>
>>>> Alfredo
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Amir
>>>>
>>>> On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano <
>>>> cardigli...@ntop.org> wrote:
>>>>
>>>>>
>>>>> On 11 Nov 2015, at 18:02, Amir Kaduri  wrote:
>>>>>
>>>>> Two following questions please:
>>>>> 1. fdir_miss (of ethtool -S) continues to increase even if my process
>>>&g

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi Alfredo,

(1) Per your fix to update /proc/net/pf_ring with match+miss+rules
statistics of sw hash rules, I've noticed the following:
A received packet that doesn't match any rule, will cause an increase of
the counter ("Sw Filt Hash Miss") of all rings that have at least 1 rule,
rather than the relevant ring only.
I assume its not the intention, right?
(2) ethtool have also the "fdir_overflow" property. Is it possible to add
this statistics to /proc per ring, together with the other statistics
you've added?
  (Note that I just want to know if pf_ring have enough info to do it
or not).

Thanks,
Amir

On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano 
wrote:

>
> On 17 Nov 2015, at 15:54, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> Thanks for that.
> 1. stats per socket means stats per ring, correct?
>
>
> Yes
>
> 2. I would like to know the best way to read this info from /proc.
> Is there any dedicated API for that?
> I took a look at /proc on a machine running an application using
> pf_ring, and found many files containing
> the stats that appear in function proc_get_info(..). I would like to
> better understand the convention of files
> to read, in order to have calculate accurate aggregated stats.
>
>
> /proc/net/pf_ring/* contains one file per ring, the name contains the
> process pid and ring id.
>
> Alfredo
>
>
> Thanks,
> Amir
>
> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano  > wrote:
>
>> Added to git code, please note stats are per socket, not per interface.
>>
>> Alfredo
>>
>> On 16 Nov 2015, at 13:48, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> Having it per interface under /proc is excellent.
>>
>> Thanks,
>> Amir
>>
>> On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>> On 16 Nov 2015, at 10:53, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>>
>>> I reviewed the implementation of the feature (great!) and I would like
>>> to clarify please:
>>>
>>> 1. If I understood it correctly, the number of matched packets is per
>>> given rule, while the number of missed packets is per ring (and not per the
>>> given rule). Is it correct, or am I wrong?
>>>
>>>  If I’m correct, can I provide a dummy rule and get the number of
>>> missed packets on the given ring?
>>>
>>> Correct
>>>
>>> 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX |
>>> grep fdir_match” you actually get the aggregation of missed/matched packets
>>> on the interface (ethX).
>>>
>>>  Is it possible to get the pf_ring statistics aggregated as well,
>>> per interface, or at least per ring?
>>>
>>> Is it ok for you if we write this stats under /proc instead of adding
>>> another API call?
>>>
>>> Alfredo
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Amir
>>>
>>> On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>>
>>>> On 11 Nov 2015, at 18:02, Amir Kaduri  wrote:
>>>>
>>>> Two following questions please:
>>>> 1. fdir_miss (of ethtool -S) continues to increase even if my process
>>>> doesn't apply the filter rules. Do you know what else causes it to increase
>>>> if its not a result of a filter?
>>>> I didn't find an answer for it in the following doc:
>>>> https://www.kernel.org/doc/Documentation/networking/ixgbe.txt
>>>>
>>>>
>>>> Please have a look at the datasheet for all cases.
>>>>
>>>> 2. Should I expect the mentioned feature request to be implemented in
>>>> pf_ring soon, or its most probably be there with many other prioritized
>>>> requests?
>>>>
>>>>
>>>> It is in queue with other feature requests, we handle them in best
>>>> effort,
>>>> however I think it will not require too much (1-2 weeks at most)
>>>>
>>>> Alfredo
>>>>
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri 
>>>> wrote:
>>>>
>>>>> Thanks for the answer. It looks to me very essential statistics for
>>>>> user-space applications.
>>>>> Github feature request: https://githu

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi Alfredo,

Thanks for that.
1. stats per socket means stats per ring, correct?
2. I would like to know the best way to read this info from /proc.
Is there any dedicated API for that?
I took a look at /proc on a machine running an application using
pf_ring, and found many files containing
the stats that appear in function proc_get_info(..). I would like to
better understand the convention of files
to read, in order to have calculate accurate aggregated stats.

Thanks,
Amir

On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano 
wrote:

> Added to git code, please note stats are per socket, not per interface.
>
> Alfredo
>
> On 16 Nov 2015, at 13:48, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> Having it per interface under /proc is excellent.
>
> Thanks,
> Amir
>
> On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <
> cardigli...@ntop.org> wrote:
>
>>
>> On 16 Nov 2015, at 10:53, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>>
>> I reviewed the implementation of the feature (great!) and I would like to
>> clarify please:
>>
>> 1. If I understood it correctly, the number of matched packets is per
>> given rule, while the number of missed packets is per ring (and not per the
>> given rule). Is it correct, or am I wrong?
>>
>>  If I’m correct, can I provide a dummy rule and get the number of
>> missed packets on the given ring?
>>
>> Correct
>>
>> 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX |
>> grep fdir_match” you actually get the aggregation of missed/matched packets
>> on the interface (ethX).
>>
>>  Is it possible to get the pf_ring statistics aggregated as well, per
>> interface, or at least per ring?
>>
>> Is it ok for you if we write this stats under /proc instead of adding
>> another API call?
>>
>> Alfredo
>>
>>
>>
>> Thanks,
>>
>> Amir
>>
>> On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>> On 11 Nov 2015, at 18:02, Amir Kaduri  wrote:
>>>
>>> Two following questions please:
>>> 1. fdir_miss (of ethtool -S) continues to increase even if my process
>>> doesn't apply the filter rules. Do you know what else causes it to increase
>>> if its not a result of a filter?
>>> I didn't find an answer for it in the following doc:
>>> https://www.kernel.org/doc/Documentation/networking/ixgbe.txt
>>>
>>>
>>> Please have a look at the datasheet for all cases.
>>>
>>> 2. Should I expect the mentioned feature request to be implemented in
>>> pf_ring soon, or its most probably be there with many other prioritized
>>> requests?
>>>
>>>
>>> It is in queue with other feature requests, we handle them in best
>>> effort,
>>> however I think it will not require too much (1-2 weeks at most)
>>>
>>> Alfredo
>>>
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri 
>>> wrote:
>>>
>>>> Thanks for the answer. It looks to me very essential statistics for
>>>> user-space applications.
>>>> Github feature request: https://github.com/ntop/PF_RING/issues/52
>>>>
>>>> Amir
>>>>
>>>>
>>>> On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano <
>>>> cardigli...@ntop.org> wrote:
>>>>
>>>>> Hi Amir
>>>>> there is no statistics by default (unless you code your own kernel
>>>>> plugin and read stats via pfring_get_hash_filtering_rule_stats, but this 
>>>>> is
>>>>> overkilling for match/miss counters),
>>>>> please open a feature request on github.
>>>>>
>>>>> Alfredo
>>>>>
>>>>> On 09 Nov 2015, at 18:28, Amir Kaduri  wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> In my application, I’m using pf_ring hash software filters (API:
>>>>> pfring_handle_hash_filtering_rule()).
>>>>>
>>>>> How can I get statistics info regarding the quantity of the filtered
>>>>> packets?
>>>>>
>>>>> I know that on ixgbe module, I can get the following parameters when
>>>>> using ethtool –s :
>>>>>
>>>>> fdir_match, fdir_miss, fdir_overflow.
>>>>>
>>>>> Are there any pf_ring user-space APIs to get these parameters? Any
>>>>> code examples?
>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Amir
>>>>> ___
>>>>> Ntop-misc mailing list
>>>>> Ntop-misc@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>>
>>>>>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi Alfredo,

Having it per interface under /proc is excellent.

Thanks,
Amir

On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano 
wrote:

>
> On 16 Nov 2015, at 10:53, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
>
> I reviewed the implementation of the feature (great!) and I would like to
> clarify please:
>
> 1. If I understood it correctly, the number of matched packets is per
> given rule, while the number of missed packets is per ring (and not per the
> given rule). Is it correct, or am I wrong?
>
>  If I’m correct, can I provide a dummy rule and get the number of
> missed packets on the given ring?
>
> Correct
>
> 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX |
> grep fdir_match” you actually get the aggregation of missed/matched packets
> on the interface (ethX).
>
>  Is it possible to get the pf_ring statistics aggregated as well, per
> interface, or at least per ring?
>
> Is it ok for you if we write this stats under /proc instead of adding
> another API call?
>
> Alfredo
>
>
>
> Thanks,
>
> Amir
>
> On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano  > wrote:
>
>>
>> On 11 Nov 2015, at 18:02, Amir Kaduri  wrote:
>>
>> Two following questions please:
>> 1. fdir_miss (of ethtool -S) continues to increase even if my process
>> doesn't apply the filter rules. Do you know what else causes it to increase
>> if its not a result of a filter?
>> I didn't find an answer for it in the following doc:
>> https://www.kernel.org/doc/Documentation/networking/ixgbe.txt
>>
>>
>> Please have a look at the datasheet for all cases.
>>
>> 2. Should I expect the mentioned feature request to be implemented in
>> pf_ring soon, or its most probably be there with many other prioritized
>> requests?
>>
>>
>> It is in queue with other feature requests, we handle them in best effort,
>> however I think it will not require too much (1-2 weeks at most)
>>
>> Alfredo
>>
>>
>> Thanks,
>> Amir
>>
>> On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri  wrote:
>>
>>> Thanks for the answer. It looks to me very essential statistics for
>>> user-space applications.
>>> Github feature request: https://github.com/ntop/PF_RING/issues/52
>>>
>>> Amir
>>>
>>>
>>> On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>> Hi Amir
>>>> there is no statistics by default (unless you code your own kernel
>>>> plugin and read stats via pfring_get_hash_filtering_rule_stats, but this is
>>>> overkilling for match/miss counters),
>>>> please open a feature request on github.
>>>>
>>>> Alfredo
>>>>
>>>> On 09 Nov 2015, at 18:28, Amir Kaduri  wrote:
>>>>
>>>> Hi,
>>>>
>>>> In my application, I’m using pf_ring hash software filters (API:
>>>> pfring_handle_hash_filtering_rule()).
>>>>
>>>> How can I get statistics info regarding the quantity of the filtered
>>>> packets?
>>>>
>>>> I know that on ixgbe module, I can get the following parameters when
>>>> using ethtool –s :
>>>>
>>>> fdir_match, fdir_miss, fdir_overflow.
>>>>
>>>> Are there any pf_ring user-space APIs to get these parameters? Any code
>>>> examples?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Amir
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>
>>>>
>>>>
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>
>>>
>>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi Alfredo,



I reviewed the implementation of the feature (great!) and I would like to
clarify please:

1. If I understood it correctly, the number of matched packets is per given
rule, while the number of missed packets is per ring (and not per the given
rule). Is it correct, or am I wrong?

 If I’m correct, can I provide a dummy rule and get the number of
missed packets on the given ring?

2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX |
grep fdir_match” you actually get the aggregation of missed/matched packets
on the interface (ethX).

 Is it possible to get the pf_ring statistics aggregated as well, per
interface, or at least per ring?



Thanks,

Amir

On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano 
wrote:

>
> On 11 Nov 2015, at 18:02, Amir Kaduri  wrote:
>
> Two following questions please:
> 1. fdir_miss (of ethtool -S) continues to increase even if my process
> doesn't apply the filter rules. Do you know what else causes it to increase
> if its not a result of a filter?
> I didn't find an answer for it in the following doc:
> https://www.kernel.org/doc/Documentation/networking/ixgbe.txt
>
>
> Please have a look at the datasheet for all cases.
>
> 2. Should I expect the mentioned feature request to be implemented in
> pf_ring soon, or its most probably be there with many other prioritized
> requests?
>
>
> It is in queue with other feature requests, we handle them in best effort,
> however I think it will not require too much (1-2 weeks at most)
>
> Alfredo
>
>
> Thanks,
> Amir
>
> On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri  wrote:
>
>> Thanks for the answer. It looks to me very essential statistics for
>> user-space applications.
>> Github feature request: https://github.com/ntop/PF_RING/issues/52
>>
>> Amir
>>
>>
>> On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano > > wrote:
>>
>>> Hi Amir
>>> there is no statistics by default (unless you code your own kernel
>>> plugin and read stats via pfring_get_hash_filtering_rule_stats, but this is
>>> overkilling for match/miss counters),
>>> please open a feature request on github.
>>>
>>> Alfredo
>>>
>>> On 09 Nov 2015, at 18:28, Amir Kaduri  wrote:
>>>
>>> Hi,
>>>
>>> In my application, I’m using pf_ring hash software filters (API:
>>> pfring_handle_hash_filtering_rule()).
>>>
>>> How can I get statistics info regarding the quantity of the filtered
>>> packets?
>>>
>>> I know that on ixgbe module, I can get the following parameters when
>>> using ethtool –s :
>>>
>>> fdir_match, fdir_miss, fdir_overflow.
>>>
>>> Are there any pf_ring user-space APIs to get these parameters? Any code
>>> examples?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Amir
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>
>>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Two following questions please:
1. fdir_miss (of ethtool -S) continues to increase even if my process
doesn't apply the filter rules. Do you know what else causes it to increase
if its not a result of a filter?
I didn't find an answer for it in the following doc:
https://www.kernel.org/doc/Documentation/networking/ixgbe.txt
2. Should I expect the mentioned feature request to be implemented in
pf_ring soon, or its most probably be there with many other prioritized
requests?

Thanks,
Amir

On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri  wrote:

> Thanks for the answer. It looks to me very essential statistics for
> user-space applications.
> Github feature request: https://github.com/ntop/PF_RING/issues/52
>
> Amir
>
>
> On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano 
> wrote:
>
>> Hi Amir
>> there is no statistics by default (unless you code your own kernel plugin
>> and read stats via pfring_get_hash_filtering_rule_stats, but this is
>> overkilling for match/miss counters),
>> please open a feature request on github.
>>
>> Alfredo
>>
>> On 09 Nov 2015, at 18:28, Amir Kaduri  wrote:
>>
>> Hi,
>>
>> In my application, I’m using pf_ring hash software filters (API:
>> pfring_handle_hash_filtering_rule()).
>>
>> How can I get statistics info regarding the quantity of the filtered
>> packets?
>>
>> I know that on ixgbe module, I can get the following parameters when
>> using ethtool –s :
>>
>> fdir_match, fdir_miss, fdir_overflow.
>>
>> Are there any pf_ring user-space APIs to get these parameters? Any code
>> examples?
>>
>>
>>
>> Thanks,
>>
>> Amir
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Thanks for the answer. It looks to me very essential statistics for
user-space applications.
Github feature request: https://github.com/ntop/PF_RING/issues/52

Amir


On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> there is no statistics by default (unless you code your own kernel plugin
> and read stats via pfring_get_hash_filtering_rule_stats, but this is
> overkilling for match/miss counters),
> please open a feature request on github.
>
> Alfredo
>
> On 09 Nov 2015, at 18:28, Amir Kaduri  wrote:
>
> Hi,
>
> In my application, I’m using pf_ring hash software filters (API:
> pfring_handle_hash_filtering_rule()).
>
> How can I get statistics info regarding the quantity of the filtered
> packets?
>
> I know that on ixgbe module, I can get the following parameters when using
> ethtool –s :
>
> fdir_match, fdir_miss, fdir_overflow.
>
> Are there any pf_ring user-space APIs to get these parameters? Any code
> examples?
>
>
>
> Thanks,
>
> Amir
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] How to get pf_ring packets filtering statistics

[Amir Kaduri]

Hi,

In my application, I’m using pf_ring hash software filters (API:
pfring_handle_hash_filtering_rule()).

How can I get statistics info regarding the quantity of the filtered
packets?

I know that on ixgbe module, I can get the following parameters when using
ethtool –s :

fdir_match, fdir_miss, fdir_overflow.

Are there any pf_ring user-space APIs to get these parameters? Any code
examples?



Thanks,

Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Possible bug on rss rehash

[Amir Kaduri]

Hi Alfredo,

I would like to add that I repeated the test with RSS=8,8,8,8,8,8,8,8 like
you, and now it works.
I guess that If you'll do it like with (with RSS 6) you'll notice the
problem as well.

Thanks,
Amir

On Thu, Jul 2, 2015 at 6:06 PM, Amir Kaduri  wrote:

> Hi Alfredo,
>
> Per your suggestion, I've downloaded today the latest (dev) pf_ring code,
> compiled and repeated the test.
> Unfortunately, the problem repeats itself: without -r the
> pfcount_multichannel receives the packets, while with -r, it doesn't.
> Any thoughts will be much appreciated.
>
> Thanks,
> Amir
>
> On Wed, Jul 1, 2015 at 7:22 AM, Amir Kaduri  wrote:
>
>> Thank you Alfredo for your response. Soon I'll test again with latest
>> code.
>>
>> Amir
>>
>> On Tue, Jun 30, 2015 at 7:54 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Hi Amir
>>> sorry I didn’t have the testbed for testing this before,
>>> I ran some test with your traffic and your configuration (actually I
>>> used 8 RSS queues) and I am able to receive all traffic with
>>> pfcount_multichannel -r. Please try with latest code.
>>>
>>> Best Regards
>>> Alfredo
>>>
>>> On 02 Jun 2015, at 08:25, Amir Kaduri  wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Any chance that you'll take a look at this issue? At least give an
>>> indication whether its a real problem or not?
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Wed, May 27, 2015 at 4:59 PM, Amir Kaduri 
>>> wrote:
>>>
>>>> I'll appreciate validating that the following is a real problem that
>>>> I've found:
>>>>
>>>> *Problem description:*
>>>>
>>>> When I use the tester “pfcount_multichannel” with the –r argument (i.e.
>>>> enable rss rehash), the application doesn’t receive the packets of some
>>>> sessions.
>>>>
>>>> When I remove the “-r” argument from the command line, all packets
>>>> received as expected.
>>>>
>>>>
>>>> *Additional information:*
>>>>
>>>> 1.   1. I’m using pf_ring dev downloaded from github on 13/05/2015.
>>>>
>>>> 2.   2. The command-line I’m using is: pfcount_multichannel -i
>>>> eth3 –r
>>>>
>>>> 3.   3. A sample pcap file of a single session, that I don’t
>>>> receive its packets, can be found in the following link:
>>>> https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazA&authuser=0
>>>>
>>>> 4.   4. Machine has 24 cores.
>>>>
>>>> 5.   5. Eth3, the receiving interface, is configured to use 6
>>>> cores.
>>>>
>>>> 6.   6. Machine details:
>>>>
>>>> Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015
>>>> x86_64 x86_64 x86_64 GNU/Linux
>>>>
>>>> 7.   7. Interface details:
>>>>
>>>> eth3   Ethernet controller [0200]: Intel Corporation 82599EB
>>>> 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>>>>
>>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>>>> [8086:000c]  MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF
>>>>
>>>> 8.   8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6
>>>>
>>>> 9.   9. cat /proc/net/pf_ring/info
>>>>
>>>> PF_RING Version  : 6.1.0 ()
>>>>
>>>> Total rings  : 6
>>>>
>>>> Standard (non DNA/ZC) Options
>>>>
>>>> Ring slots   : 4096
>>>>
>>>> Slot version : 16
>>>>
>>>> Capture TX   : Yes [RX+TX]
>>>>
>>>> IP Defragment: No
>>>>
>>>> Socket Mode  : Standard
>>>>
>>>> Total plugins: 0
>>>>
>>>> Cluster Fragment Queue   : 0
>>>>
>>>> Cluster Fragment Discard : 0
>>>>
>>>>
>>>> *Additional remarks:*
>>>>
>>>> 1.  1.  In function pfring_hash_pkt(..) in file pfring_utils.c,
>>>> the fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like
>>>> a bug.
>>>>
>>>> 2.   2. By reviewing the code, it looks that the functions
>>>> hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function
>>>> pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Amir
>>>>
>>>>
>>>>
>>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>
>>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Possible bug on rss rehash

[Amir Kaduri]

Hi Alfredo,

Per your suggestion, I've downloaded today the latest (dev) pf_ring code,
compiled and repeated the test.
Unfortunately, the problem repeats itself: without -r the
pfcount_multichannel receives the packets, while with -r, it doesn't.
Any thoughts will be much appreciated.

Thanks,
Amir

On Wed, Jul 1, 2015 at 7:22 AM, Amir Kaduri  wrote:

> Thank you Alfredo for your response. Soon I'll test again with latest code.
>
> Amir
>
> On Tue, Jun 30, 2015 at 7:54 PM, Alfredo Cardigliano  > wrote:
>
>> Hi Amir
>> sorry I didn’t have the testbed for testing this before,
>> I ran some test with your traffic and your configuration (actually I used
>> 8 RSS queues) and I am able to receive all traffic with
>> pfcount_multichannel -r. Please try with latest code.
>>
>> Best Regards
>> Alfredo
>>
>> On 02 Jun 2015, at 08:25, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> Any chance that you'll take a look at this issue? At least give an
>> indication whether its a real problem or not?
>>
>> Thanks,
>> Amir
>>
>> On Wed, May 27, 2015 at 4:59 PM, Amir Kaduri  wrote:
>>
>>> I'll appreciate validating that the following is a real problem that
>>> I've found:
>>>
>>> *Problem description:*
>>>
>>> When I use the tester “pfcount_multichannel” with the –r argument (i.e.
>>> enable rss rehash), the application doesn’t receive the packets of some
>>> sessions.
>>>
>>> When I remove the “-r” argument from the command line, all packets
>>> received as expected.
>>>
>>>
>>> *Additional information:*
>>>
>>> 1.   1. I’m using pf_ring dev downloaded from github on 13/05/2015.
>>>
>>> 2.   2. The command-line I’m using is: pfcount_multichannel -i eth3
>>> –r
>>>
>>> 3.   3. A sample pcap file of a single session, that I don’t
>>> receive its packets, can be found in the following link:
>>> https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazA&authuser=0
>>>
>>> 4.   4. Machine has 24 cores.
>>>
>>> 5.   5. Eth3, the receiving interface, is configured to use 6 cores.
>>>
>>> 6.   6. Machine details:
>>>
>>> Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015
>>> x86_64 x86_64 x86_64 GNU/Linux
>>>
>>> 7.   7. Interface details:
>>>
>>> eth3   Ethernet controller [0200]: Intel Corporation 82599EB
>>> 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>>>
>>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c]
>>> MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF
>>>
>>> 8.   8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6
>>>
>>> 9.   9. cat /proc/net/pf_ring/info
>>>
>>> PF_RING Version  : 6.1.0 ()
>>>
>>> Total rings  : 6
>>>
>>> Standard (non DNA/ZC) Options
>>>
>>> Ring slots   : 4096
>>>
>>> Slot version : 16
>>>
>>> Capture TX   : Yes [RX+TX]
>>>
>>> IP Defragment: No
>>>
>>> Socket Mode  : Standard
>>>
>>> Total plugins: 0
>>>
>>> Cluster Fragment Queue   : 0
>>>
>>> Cluster Fragment Discard : 0
>>>
>>>
>>> *Additional remarks:*
>>>
>>> 1.  1.  In function pfring_hash_pkt(..) in file pfring_utils.c, the
>>> fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a
>>> bug.
>>>
>>> 2.   2. By reviewing the code, it looks that the functions
>>> hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function
>>> pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified.
>>>
>>>
>>> Regards,
>>>
>>> Amir
>>>
>>>
>>>
>>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Possible bug on rss rehash

[Amir Kaduri]

Thank you Alfredo for your response. Soon I'll test again with latest code.

Amir

On Tue, Jun 30, 2015 at 7:54 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> sorry I didn’t have the testbed for testing this before,
> I ran some test with your traffic and your configuration (actually I used
> 8 RSS queues) and I am able to receive all traffic with
> pfcount_multichannel -r. Please try with latest code.
>
> Best Regards
> Alfredo
>
> On 02 Jun 2015, at 08:25, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> Any chance that you'll take a look at this issue? At least give an
> indication whether its a real problem or not?
>
> Thanks,
> Amir
>
> On Wed, May 27, 2015 at 4:59 PM, Amir Kaduri  wrote:
>
>> I'll appreciate validating that the following is a real problem that I've
>> found:
>>
>> *Problem description:*
>>
>> When I use the tester “pfcount_multichannel” with the –r argument (i.e.
>> enable rss rehash), the application doesn’t receive the packets of some
>> sessions.
>>
>> When I remove the “-r” argument from the command line, all packets
>> received as expected.
>>
>>
>> *Additional information:*
>>
>> 1.   1. I’m using pf_ring dev downloaded from github on 13/05/2015.
>>
>> 2.   2. The command-line I’m using is: pfcount_multichannel -i eth3
>> –r
>>
>> 3.   3. A sample pcap file of a single session, that I don’t receive
>> its packets, can be found in the following link:
>> https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazA&authuser=0
>>
>> 4.   4. Machine has 24 cores.
>>
>> 5.   5. Eth3, the receiving interface, is configured to use 6 cores.
>>
>> 6.   6. Machine details:
>>
>> Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015
>> x86_64 x86_64 x86_64 GNU/Linux
>>
>> 7.   7. Interface details:
>>
>> eth3   Ethernet controller [0200]: Intel Corporation 82599EB
>> 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>>
>> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c]
>> MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF
>>
>> 8.   8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6
>>
>> 9.   9. cat /proc/net/pf_ring/info
>>
>> PF_RING Version  : 6.1.0 ()
>>
>> Total rings  : 6
>>
>> Standard (non DNA/ZC) Options
>>
>> Ring slots   : 4096
>>
>> Slot version : 16
>>
>> Capture TX   : Yes [RX+TX]
>>
>> IP Defragment: No
>>
>> Socket Mode  : Standard
>>
>> Total plugins: 0
>>
>> Cluster Fragment Queue   : 0
>>
>> Cluster Fragment Discard : 0
>>
>>
>> *Additional remarks:*
>>
>> 1.  1.  In function pfring_hash_pkt(..) in file pfring_utils.c, the
>> fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a
>> bug.
>>
>> 2.   2. By reviewing the code, it looks that the functions
>> hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function
>> pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified.
>>
>>
>> Regards,
>>
>> Amir
>>
>>
>>
>>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Possible bug on rss rehash

[Amir Kaduri]

Hi Alfredo,

Any chance that you'll take a look at this issue? At least give an
indication whether its a real problem or not?

Thanks,
Amir

On Wed, May 27, 2015 at 4:59 PM, Amir Kaduri  wrote:

> I'll appreciate validating that the following is a real problem that I've
> found:
>
> *Problem description:*
>
> When I use the tester “pfcount_multichannel” with the –r argument (i.e.
> enable rss rehash), the application doesn’t receive the packets of some
> sessions.
>
> When I remove the “-r” argument from the command line, all packets
> received as expected.
>
>
>
> *Additional information:*
>
> 1.   1. I’m using pf_ring dev downloaded from github on 13/05/2015.
>
> 2.   2. The command-line I’m using is: pfcount_multichannel -i eth3 –r
>
> 3.   3. A sample pcap file of a single session, that I don’t receive
> its packets, can be found in the following link:
> https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazA&authuser=0
>
> 4.   4. Machine has 24 cores.
>
> 5.   5. Eth3, the receiving interface, is configured to use 6 cores.
>
> 6.   6. Machine details:
>
> Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64
> x86_64 x86_64 GNU/Linux
>
> 7.   7. Interface details:
>
> eth3   Ethernet controller [0200]: Intel Corporation 82599EB
> 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>
> Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c]
> MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF
>
> 8.   8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6
>
> 9.   9. cat /proc/net/pf_ring/info
>
> PF_RING Version  : 6.1.0 ()
>
> Total rings  : 6
>
> Standard (non DNA/ZC) Options
>
> Ring slots   : 4096
>
> Slot version : 16
>
> Capture TX   : Yes [RX+TX]
>
> IP Defragment: No
>
> Socket Mode  : Standard
>
> Total plugins: 0
>
> Cluster Fragment Queue   : 0
>
> Cluster Fragment Discard : 0
>
>
>
> *Additional remarks:*
>
> 1.  1.  In function pfring_hash_pkt(..) in file pfring_utils.c, the
> fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a
> bug.
>
> 2.   2. By reviewing the code, it looks that the functions
> hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function
> pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified.
>
>
> Regards,
>
> Amir
>
>
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Possible bug on rss rehash

[Amir Kaduri]

I'll appreciate validating that the following is a real problem that I've
found:

*Problem description:*

When I use the tester “pfcount_multichannel” with the –r argument (i.e.
enable rss rehash), the application doesn’t receive the packets of some
sessions.

When I remove the “-r” argument from the command line, all packets received
as expected.



*Additional information:*

1.   1. I’m using pf_ring dev downloaded from github on 13/05/2015.

2.   2. The command-line I’m using is: pfcount_multichannel -i eth3 –r

3.   3. A sample pcap file of a single session, that I don’t receive
its packets, can be found in the following link:
https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazA&authuser=0

4.   4. Machine has 24 cores.

5.   5. Eth3, the receiving interface, is configured to use 6 cores.

6.   6. Machine details:

Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64
x86_64 x86_64 GNU/Linux

7.   7. Interface details:

eth3   Ethernet controller [0200]: Intel Corporation 82599EB 10-Gigabit
SFI/SFP+ Network Connection [8086:10fb] (rev 01)

Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c]
MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF

8.   8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6

9.   9. cat /proc/net/pf_ring/info

PF_RING Version  : 6.1.0 ()

Total rings  : 6

Standard (non DNA/ZC) Options

Ring slots   : 4096

Slot version : 16

Capture TX   : Yes [RX+TX]

IP Defragment: No

Socket Mode  : Standard

Total plugins: 0

Cluster Fragment Queue   : 0

Cluster Fragment Discard : 0



*Additional remarks:*

1.  1.  In function pfring_hash_pkt(..) in file pfring_utils.c, the
fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a
bug.

2.   2. By reviewing the code, it looks that the functions
hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function
pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified.


Regards,

Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Possible bug in SW hash rule API

[Amir Kaduri]

After using latest code from git, the problem is indeed solved!

Thanks a lot Alfredo!

On Wed, May 13, 2015 at 1:43 AM, Alfredo Cardigliano 
wrote:

> Hi Amir
> the commit below should solve your issue, please pull latest code from git
>
> Alfredo
>
>
> https://github.com/ntop/PF_RING/commit/b67a6f46a06e68f2bb6cc53e9d452cc2cbe5f18f
>
> On 12 May 2015, at 12:15, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> The application itself is one of the example application provided by
> pf_ring itself called "pfcount".
> Here is a decription of he steps done:
> Download pf_ring 6.0.3 and compile it 64bit (including drivers'
> PF_RING_aware subdirectory).
> (need to fix a small bug in pfcount.c by adding "break;" at line 799, so
> that it will be possible to use the option "-u").
> On production env, suppose current working directory is PF_RING's source
> root.
>
> From this point, the commands are:
> rmmod ixgbe
> rmmod pf_ring
> insmod ./kernel/pf_ring.ko
> insmod ./drivers/PF_RING_aware/intel/ixgbe/ixgbe-3.22.3-zc/src/ixgbe.ko
> ./userland/examples/pfcount -i eth3 -w 1 -m -v 1 -u 1
>
> You can replay the following pcap that contains 344 packets:
>
> https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
>
>  You should see that "pfcount" stops showing packets from this pcap (since
> the hash filter is applied), but on the other hand, it shows the following
> message (especially the first one):
> 13:02:28.616480105 [RX][if_index=29][00:30:05:4B:19:2C ->
> 00:01:02:03:04:05] [vlan 12] [IPv4][10.12.150.231:2489 -> 10.61.12.31:139]
> [l3_proto=TCP][hash=340372828][tos=0][tcp_seq_num=3256921511]
> [caplen=128][len=242][parsed_header_len=0][eth_offset=0][l3_offset=18][l4_offset=38][payload_offset=58]
>
> pfring_add_hash_filtering_rule(1) failed
>
> This error message indicates that pfring_add_hash_filtering_rule()
> returned failure in userland.
> Note that despite the returned error in userland, the hash rule has been
> added successfully in the kernel.
>
> Thanks,
> Amir
>
> On Mon, May 11, 2015 at 8:02 PM, Alfredo Cardigliano  > wrote:
>
>> Hi Amir
>> could you provide a sample application, or a code snippet, we can use to
>> reproduce the issue?
>>
>> Thank you
>> Alfredo
>>
>> > On 11 May 2015, at 10:52, Amir Kaduri  wrote:
>> >
>> > Hi,
>> >
>> > I'm experiencing a possible bug in the API
>> pfring_handle_hash_filtering_rule(), when adding a rule.
>> > The API return -1 (failure) although the hash rule has been added
>> successfully.
>> > I experience it in versions 6.0.2 and 6.0.3.
>> > In version 6.0.1 it works fine.
>> > Can anyone please confirm that there is a bug?
>> > If yes, can I have a patch for it, or even a quick-and-dirty
>> explanation on how to solve it?
>> >
>> > Thanks,
>> > Amir
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] Possible bug in SW hash rule API

[Amir Kaduri]

Hi Alfredo,

The application itself is one of the example application provided by
pf_ring itself called "pfcount".
Here is a decription of he steps done:
Download pf_ring 6.0.3 and compile it 64bit (including drivers'
PF_RING_aware subdirectory).
(need to fix a small bug in pfcount.c by adding "break;" at line 799, so
that it will be possible to use the option "-u").
On production env, suppose current working directory is PF_RING's source
root.

>From this point, the commands are:
rmmod ixgbe
rmmod pf_ring
insmod ./kernel/pf_ring.ko
insmod ./drivers/PF_RING_aware/intel/ixgbe/ixgbe-3.22.3-zc/src/ixgbe.ko
./userland/examples/pfcount -i eth3 -w 1 -m -v 1 -u 1

You can replay the following pcap that contains 344 packets:
https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing

 You should see that "pfcount" stops showing packets from this pcap (since
the hash filter is applied), but on the other hand, it shows the following
message (especially the first one):
13:02:28.616480105 [RX][if_index=29][00:30:05:4B:19:2C ->
00:01:02:03:04:05] [vlan 12] [IPv4][10.12.150.231:2489 -> 10.61.12.31:139]
[l3_proto=TCP][hash=340372828][tos=0][tcp_seq_num=3256921511]
[caplen=128][len=242][parsed_header_len=0][eth_offset=0][l3_offset=18][l4_offset=38][payload_offset=58]

pfring_add_hash_filtering_rule(1) failed

This error message indicates that pfring_add_hash_filtering_rule() returned
failure in userland.
Note that despite the returned error in userland, the hash rule has been
added successfully in the kernel.

Thanks,
Amir

On Mon, May 11, 2015 at 8:02 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> could you provide a sample application, or a code snippet, we can use to
> reproduce the issue?
>
> Thank you
> Alfredo
>
> > On 11 May 2015, at 10:52, Amir Kaduri  wrote:
> >
> > Hi,
> >
> > I'm experiencing a possible bug in the API
> pfring_handle_hash_filtering_rule(), when adding a rule.
> > The API return -1 (failure) although the hash rule has been added
> successfully.
> > I experience it in versions 6.0.2 and 6.0.3.
> > In version 6.0.1 it works fine.
> > Can anyone please confirm that there is a bug?
> > If yes, can I have a patch for it, or even a quick-and-dirty explanation
> on how to solve it?
> >
> > Thanks,
> > Amir
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Possible bug in SW hash rule API

[Amir Kaduri]

Hi,

I'm experiencing a possible bug in the
API pfring_handle_hash_filtering_rule(), when adding a rule.
The API return -1 (failure) although the hash rule has been added
successfully.
I experience it in versions 6.0.2 and 6.0.3.
In version 6.0.1 it works fine.
Can anyone please confirm that there is a bug?
If yes, can I have a patch for it, or even a quick-and-dirty explanation on
how to solve it?

Thanks,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hi,

I've made progress with adding hardware filtering rules, using pf_ring's
API (pfring_add_hw_rule()). I managed to add 8190 (8K-2) perfect rules.
I know that its possible to have ~32,000 (32K-2) hardware hash rules, but I
fail to reach this limit.
When loading the ixgbe kernel module, I use "FdirPballoc=3,3,3,3".
According to Intel's 82599 10GB controller datasheet
<http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/82599-10-gbe-controller-datasheet.pdf>
page
289, I understand that its somehow related to rx packet buffer memory
allocation,
but I didn't manage to find out what exactly and how should I do the
modification in order to be able to use 32K hardware hash (signature) rules.
Any help will be appreciated.

Regards,
Amir

On Wed, Apr 29, 2015 at 11:17 AM, Amir Kaduri  wrote:

> Hi Pavel,
>
> Thanks for your response.
> I didn't try it yet, I'll consider trying it, but currently my goal is to
> evaluate pf_ring C/C++ package to see if it fits my needs, in comparison to
> other packages.
> This is since I need to develop a kind of business-logic application using
> it.
> I really want to be able to add/remove HW filtering rules using pf_ring's
> APIs.
>
> Thanks,
> Amir
>
> On Wed, Apr 29, 2015 at 10:57 AM, Pavel Odintsov  > wrote:
>
>> Hello, Amir!
>>
>> Have you tried manual configuration of hardware filters for 82599? You
>> could try do it with ethtool:
>>
>> https://github.com/FastVPSEestiOu/fastnetmon/wiki/Traffic-filtration-using-NIC-capabilities-on-wire-speed-(10GE,-14Mpps)
>>
>> From my point of view ethtool interface is more flexible for hardware
>> filter management.  But I can't find any API for C/C++ for it.
>>
>> On Wed, Apr 29, 2015 at 10:50 AM, Amir Kaduri 
>> wrote:
>> > Hi,
>> >
>> > After testing software filtering rules, I've tried hardware filtering
>> rules.
>> > Unfortunately, it doesn't work for me.
>> > First, I tested it using pfring 6.0.1 64bit, and then used pfring 5.6.1
>> > 32bit to make sure its not related to the version and the architecture.
>> > In both tests, I didn't see that the hardware filters work.
>> >
>> > The command line I used: ./pfcount_82599 -i eth2 -v -m
>> >
>> > The NIC details: 06:00.1 Ethernet controller: Intel Corporation 82599EB
>> > 10-Gigabit SFI/SFP+ Network Connection (rev 01)
>> >
>> > I used the following files:
>> > 1.
>> >
>> https://drive.google.com/file/d/0B10Ms5GOXgCxYy1uWHZ2dXJrck0/view?usp=sharing
>> >The tester program pfcount_82599.c with slight changes: enabling 3
>> > intel_82599_five_tuple_rule rules:
>> > - One that drops tcp packets
>> > - Second that drops packets with source 10.12.150.231
>> > - Third that drops packets with dest 10.12.150.231
>> > 2.
>> >
>> https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
>> > The pcap file containing 344 packets, that should be filtered with
>> the
>> > tester above.
>> >
>> > Note that all 3 rules should drop the packets in the attached tester.
>> >
>> > Any help to prove that hardware rules work, based on the above info,
>> will be
>> > much appreciated.
>> >
>> > Thanks,
>> > Amir
>>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hi Pavel,

Thanks for your response.
I didn't try it yet, I'll consider trying it, but currently my goal is to
evaluate pf_ring C/C++ package to see if it fits my needs, in comparison to
other packages.
This is since I need to develop a kind of business-logic application using
it.
I really want to be able to add/remove HW filtering rules using pf_ring's
APIs.

Thanks,
Amir

On Wed, Apr 29, 2015 at 10:57 AM, Pavel Odintsov 
wrote:

> Hello, Amir!
>
> Have you tried manual configuration of hardware filters for 82599? You
> could try do it with ethtool:
>
> https://github.com/FastVPSEestiOu/fastnetmon/wiki/Traffic-filtration-using-NIC-capabilities-on-wire-speed-(10GE,-14Mpps)
>
> From my point of view ethtool interface is more flexible for hardware
> filter management.  But I can't find any API for C/C++ for it.
>
> On Wed, Apr 29, 2015 at 10:50 AM, Amir Kaduri  wrote:
> > Hi,
> >
> > After testing software filtering rules, I've tried hardware filtering
> rules.
> > Unfortunately, it doesn't work for me.
> > First, I tested it using pfring 6.0.1 64bit, and then used pfring 5.6.1
> > 32bit to make sure its not related to the version and the architecture.
> > In both tests, I didn't see that the hardware filters work.
> >
> > The command line I used: ./pfcount_82599 -i eth2 -v -m
> >
> > The NIC details: 06:00.1 Ethernet controller: Intel Corporation 82599EB
> > 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> >
> > I used the following files:
> > 1.
> >
> https://drive.google.com/file/d/0B10Ms5GOXgCxYy1uWHZ2dXJrck0/view?usp=sharing
> >The tester program pfcount_82599.c with slight changes: enabling 3
> > intel_82599_five_tuple_rule rules:
> > - One that drops tcp packets
> > - Second that drops packets with source 10.12.150.231
> > - Third that drops packets with dest 10.12.150.231
> > 2.
> >
> https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
> > The pcap file containing 344 packets, that should be filtered with
> the
> > tester above.
> >
> > Note that all 3 rules should drop the packets in the attached tester.
> >
> > Any help to prove that hardware rules work, based on the above info,
> will be
> > much appreciated.
> >
> > Thanks,
> > Amir
> >
> >
> > On Thu, Apr 16, 2015 at 12:54 AM, Alfredo Cardigliano <
> cardigli...@ntop.org>
> > wrote:
> >>
> >>
> >> On 15 Apr 2015, at 19:10, Amir Kaduri  wrote:
> >>
> >> Hi,
> >>
> >> I would like to update that the problem in the tester program was that I
> >> probably didn't add the vlan_id to rule. Once added, the hash rule
> began to
> >> work.
> >>
> >>
> >> Ok, that was definitely the problem, vlan-id is part of the hash input
> >> (vlan, protocol, src/dst ip, src/dst port).
> >>
> >> Also, I would like to refer to the argument "There is no limit to the
> >> number of software hash rules":
> >> I assume that the meaning is that the algorithm doesn't have any
> >> limitation, though, since the rule_id field type is u_int16_t, there is
> a
> >> limit of 64K hash rules, de-facto.
> >>
> >>
> >> That’s right, we can consider changing the rule_id type if 64K rules is
> >> not enough in some use case.
> >>
> >> Alfredo
> >>
> >>
> >> Thanks,
> >> Amir
> >>
> >> On Tue, Apr 14, 2015 at 12:34 PM, Alfredo Cardigliano
> >>  wrote:
> >>>
> >>> Hi Amir
> >>> yes thank you for the files, I will look at them ASAP.
> >>>
> >>> Alfredo
> >>>
> >>> On 14 Apr 2015, at 11:17, Amir Kaduri  wrote:
> >>>
> >>> Hi Alfredo,
> >>>
> >>> I hope you saw what I've sent last week. Any chance that you will take
> a
> >>> look at it soon?
> >>>
> >>> Thanks,
> >>> Amir
> >>>
> >>> On Fri, Apr 10, 2015 at 11:08 AM, Amir Kaduri 
> >>> wrote:
> >>>>>
> >>>>> Hi Alfredo,
> >>>>>
> >>>>> Since the original pcap is huge, I sliced a smaller pcap from it,
> >>>>> holding 344 packets.
> >>>>
> >>>> Link to the tester code (based on  pfcount_82599.c):
> >>>>
> >>>>
> https://drive.google.com/file/d/0B10Ms5GOXgCxdXhtRGRoQ1FTYVFDbkFGMkw

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hi,

After testing software filtering rules, I've tried hardware filtering
rules. Unfortunately, it doesn't work for me.
First, I tested it using pfring 6.0.1 64bit, and then used pfring 5.6.1
32bit to make sure its not related to the version and the architecture.
In both tests, I didn't see that the hardware filters work.

The command line I used: *./pfcount_82599 -i eth2 -v -m*

The NIC details: 06:00.1 Ethernet controller: Intel Corporation 82599EB
10-Gigabit SFI/SFP+ Network Connection (rev 01)

I used the following files:
1.
https://drive.google.com/file/d/0B10Ms5GOXgCxYy1uWHZ2dXJrck0/view?usp=sharing
   The tester program pfcount_82599.c with slight changes: enabling
3 intel_82599_five_tuple_rule rules:
- One that drops tcp packets
- Second that drops packets with source 10.12.150.231
- Third that drops packets with dest 10.12.150.231
2.
https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
The pcap file containing 344 packets, that should be filtered with the
tester above.

Note that all 3 rules should drop the packets in the attached tester.

Any help to prove that hardware rules work, based on the above info, will
be much appreciated.

Thanks,
Amir


On Thu, Apr 16, 2015 at 12:54 AM, Alfredo Cardigliano 
wrote:

>
> On 15 Apr 2015, at 19:10, Amir Kaduri  wrote:
>
> Hi,
>
> I would like to update that the problem in the tester program was that I
> probably didn't add the vlan_id to rule. Once added, the hash rule began to
> work.
>
>
> Ok, that was definitely the problem, vlan-id is part of the hash input
> (vlan, protocol, src/dst ip, src/dst port).
>
> Also, I would like to refer to the argument "There is no limit to the
> number of software hash rules":
> I assume that the meaning is that the algorithm doesn't have any
> limitation, though, since the rule_id field type is u_int16_t, there is a
> limit of 64K hash rules, de-facto.
>
>
> That’s right, we can consider changing the rule_id type if 64K rules is
> not enough in some use case.
>
> Alfredo
>
>
> Thanks,
> Amir
>
> On Tue, Apr 14, 2015 at 12:34 PM, Alfredo Cardigliano <
> cardigli...@ntop.org> wrote:
>
>> Hi Amir
>> yes thank you for the files, I will look at them ASAP.
>>
>> Alfredo
>>
>> On 14 Apr 2015, at 11:17, Amir Kaduri  wrote:
>>
>> Hi Alfredo,
>>
>> I hope you saw what I've sent last week. Any chance that you will take a
>> look at it soon?
>>
>> Thanks,
>> Amir
>>
>> On Fri, Apr 10, 2015 at 11:08 AM, Amir Kaduri 
>> wrote:
>>
>>> Hi Alfredo,
>>>>
>>>> Since the original pcap is huge, I sliced a smaller pcap from it,
>>>> holding 344 packets.
>>>>
>>> Link to the tester code (based on  pfcount_82599.c):
>>>
>>> https://drive.google.com/file/d/0B10Ms5GOXgCxdXhtRGRoQ1FTYVFDbkFGMkwzVExNRHJXbmt3/view?usp=sharing
>>>
>>> Link to the pcap file (congtaining 344 packets):
>>>
>>> https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
>>>
>>>
>>>> Thanks a lot,
>>>> Amir
>>>>
>>>>
>>> On Wed, Apr 8, 2015 at 4:07 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>> Hi Amir
>>>> could you provide your patched pfcount and a pcap with the packets
>>>> matching your rule?
>>>>
>>>> Alfredo
>>>>
>>>> On 08 Apr 2015, at 13:36, Amir Kaduri  wrote:
>>>>
>>>> I'm currently in a situation, that no rules work for me whatsoever
>>>> (same configuration I described earlier), and they did work for me a few
>>>> day ago.
>>>> I'm using the pfcount_82599.c tester, and added a single hash rule. The
>>>> only thing that does work is the default behavior, when using the API
>>>> pfring_toggle_filtering_policy(pd, 1) (default to allow - packets
>>>> arrive normally) or pfring_toggle_filtering_policy(pd, 0); (default to
>>>> drop, no packet arrive to the tester).
>>>> This is the part of code:
>>>>
>>>> --
>>>> :
>>>>  rc = pfring_enable_ring(pd);
>>>>  if (rc<0)
>>>> printf("pfring_enable_ring() failed. rc=%d\n", rc);
>>>>
>>>>   pfring_toggle_filtering_policy(pd, 1); /* Default to allow */
>>>>
>>>>   if (1) {
>>>> hash_filtering_rule rule;

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hi,

I would like to update that the problem in the tester program was that I
probably didn't add the vlan_id to rule. Once added, the hash rule began to
work.

Also, I would like to refer to the argument "There is no limit to the
number of software hash rules":
I assume that the meaning is that the algorithm doesn't have any
limitation, though, since the rule_id field type is u_int16_t, there is a
limit of 64K hash rules, de-facto.

Thanks,
Amir

On Tue, Apr 14, 2015 at 12:34 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> yes thank you for the files, I will look at them ASAP.
>
> Alfredo
>
> On 14 Apr 2015, at 11:17, Amir Kaduri  wrote:
>
> Hi Alfredo,
>
> I hope you saw what I've sent last week. Any chance that you will take a
> look at it soon?
>
> Thanks,
> Amir
>
> On Fri, Apr 10, 2015 at 11:08 AM, Amir Kaduri  wrote:
>
>> Hi Alfredo,
>>>
>>> Since the original pcap is huge, I sliced a smaller pcap from it,
>>> holding 344 packets.
>>>
>> Link to the tester code (based on  pfcount_82599.c):
>>
>> https://drive.google.com/file/d/0B10Ms5GOXgCxdXhtRGRoQ1FTYVFDbkFGMkwzVExNRHJXbmt3/view?usp=sharing
>>
>> Link to the pcap file (congtaining 344 packets):
>>
>> https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
>>
>>
>>> Thanks a lot,
>>> Amir
>>>
>>>
>> On Wed, Apr 8, 2015 at 4:07 PM, Alfredo Cardigliano > > wrote:
>>
>>> Hi Amir
>>> could you provide your patched pfcount and a pcap with the packets
>>> matching your rule?
>>>
>>> Alfredo
>>>
>>> On 08 Apr 2015, at 13:36, Amir Kaduri  wrote:
>>>
>>> I'm currently in a situation, that no rules work for me whatsoever (same
>>> configuration I described earlier), and they did work for me a few day ago.
>>> I'm using the pfcount_82599.c tester, and added a single hash rule. The
>>> only thing that does work is the default behavior, when using the API
>>> pfring_toggle_filtering_policy(pd, 1) (default to allow - packets arrive
>>> normally) or pfring_toggle_filtering_policy(pd, 0); (default to drop, no
>>> packet arrive to the tester).
>>> This is the part of code:
>>>
>>> --
>>> :
>>>  rc = pfring_enable_ring(pd);
>>>  if (rc<0)
>>> printf("pfring_enable_ring() failed. rc=%d\n", rc);
>>>
>>>   pfring_toggle_filtering_policy(pd, 1); /* Default to allow */
>>>
>>>   if (1) {
>>> hash_filtering_rule rule;
>>> u_int16_t rule_id = 0;
>>>
>>> memset(&rule, 0, sizeof(hash_filtering_rule));
>>> rule.proto = 6;
>>> rule.rule_id = rule_id++;
>>> rule.rule_action = dont_forward_packet_and_stop_rule_evaluation;
>>> rule.host4_peer_a = ntohl(inet_addr("10.12.150.231"));
>>> rule.port_peer_a = 2489;
>>> rule.host4_peer_b = ntohl(inet_addr("10.61.12.31"));
>>> rule.port_peer_b = 139;
>>> rc = pfring_handle_hash_filtering_rule(pd, &rule, 1);
>>> if(rc<0)
>>> printf("pfring_add_hash_filtering_rule(%d) failed [errno=%d: %s]\n",
>>> rule.rule_id, errno, strerror(errno));
>>> else
>>> printf("pfring_add_hash_filtering_rule(%d) succeeded\n", rule.rule_id);
>>>   }
>>>
>>> --
>>> I don't get any errors when the following code run, but when I bombard
>>> the machine
>>> with a pcap containing more than 600,000 packets of the specified
>>> session that I've expected to be filtered out,
>>> packets of it still received at the tester.
>>>
>>> I'm pretty sure that something goes wrong in pf_ring, but I can't tell
>>> what.
>>> What is the best way to get debug information, other than reading
>>> /var/log/messages?
>>>
>>> Thanks,
>>> Amir
>>>
>>>
>>> On Wed, Apr 8, 2015 at 1:08 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>>
>>>> On 05 Apr 2015, at 16:07, Amir Kaduri  wrote:
>>>>
>>>> Hi,
>>>>
>>>> I think I've made some progress: AFAIU, the packets that I see despite
>>>> the rule that supposed to filter them, are packets the receive during the
>>>> time interval fr

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hi Alfredo,

I hope you saw what I've sent last week. Any chance that you will take a
look at it soon?

Thanks,
Amir

On Fri, Apr 10, 2015 at 11:08 AM, Amir Kaduri  wrote:

> Hi Alfredo,
>>
>> Since the original pcap is huge, I sliced a smaller pcap from it, holding
>> 344 packets.
>>
> Link to the tester code (based on  pfcount_82599.c):
>
> https://drive.google.com/file/d/0B10Ms5GOXgCxdXhtRGRoQ1FTYVFDbkFGMkwzVExNRHJXbmt3/view?usp=sharing
>
> Link to the pcap file (congtaining 344 packets):
>
> https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
>
>
>> Thanks a lot,
>> Amir
>>
>>
> On Wed, Apr 8, 2015 at 4:07 PM, Alfredo Cardigliano 
> wrote:
>
>> Hi Amir
>> could you provide your patched pfcount and a pcap with the packets
>> matching your rule?
>>
>> Alfredo
>>
>> On 08 Apr 2015, at 13:36, Amir Kaduri  wrote:
>>
>> I'm currently in a situation, that no rules work for me whatsoever (same
>> configuration I described earlier), and they did work for me a few day ago.
>> I'm using the pfcount_82599.c tester, and added a single hash rule. The
>> only thing that does work is the default behavior, when using the API
>> pfring_toggle_filtering_policy(pd, 1) (default to allow - packets arrive
>> normally) or pfring_toggle_filtering_policy(pd, 0); (default to drop, no
>> packet arrive to the tester).
>> This is the part of code:
>>
>> --
>> :
>>  rc = pfring_enable_ring(pd);
>>  if (rc<0)
>> printf("pfring_enable_ring() failed. rc=%d\n", rc);
>>
>>   pfring_toggle_filtering_policy(pd, 1); /* Default to allow */
>>
>>   if (1) {
>> hash_filtering_rule rule;
>> u_int16_t rule_id = 0;
>>
>> memset(&rule, 0, sizeof(hash_filtering_rule));
>> rule.proto = 6;
>> rule.rule_id = rule_id++;
>> rule.rule_action = dont_forward_packet_and_stop_rule_evaluation;
>> rule.host4_peer_a = ntohl(inet_addr("10.12.150.231"));
>> rule.port_peer_a = 2489;
>> rule.host4_peer_b = ntohl(inet_addr("10.61.12.31"));
>> rule.port_peer_b = 139;
>> rc = pfring_handle_hash_filtering_rule(pd, &rule, 1);
>> if(rc<0)
>> printf("pfring_add_hash_filtering_rule(%d) failed [errno=%d: %s]\n",
>> rule.rule_id, errno, strerror(errno));
>> else
>> printf("pfring_add_hash_filtering_rule(%d) succeeded\n", rule.rule_id);
>>   }
>>
>> --
>> I don't get any errors when the following code run, but when I bombard
>> the machine
>> with a pcap containing more than 600,000 packets of the specified session
>> that I've expected to be filtered out,
>> packets of it still received at the tester.
>>
>> I'm pretty sure that something goes wrong in pf_ring, but I can't tell
>> what.
>> What is the best way to get debug information, other than reading
>> /var/log/messages?
>>
>> Thanks,
>> Amir
>>
>>
>> On Wed, Apr 8, 2015 at 1:08 PM, Alfredo Cardigliano > > wrote:
>>
>>>
>>> On 05 Apr 2015, at 16:07, Amir Kaduri  wrote:
>>>
>>> Hi,
>>>
>>> I think I've made some progress: AFAIU, the packets that I see despite
>>> the rule that supposed to filter them, are packets the receive during the
>>> time interval from rule-set to rule-apply by pfring.
>>> I'll appreciate getting some answers about the following:
>>> 1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way
>>> to know which rules-ids are set and which are vacant?
>>> This is because I have to follow the rules-ids when setting them,
>>> but when I purge them, I don't know which of them were removed.
>>>
>>>
>>> No, unfortunately this is not possible with the current API.
>>>
>>> 2. Does this API also purges HW rules?
>>>
>>>
>>> No, It doesn’t.
>>>
>>> 3. According to the documentation, I know that HW rules have a limit of
>>> 32,000. What is the limit for hash rules? IS this limit includes the 32,000
>>> of the HW, or additional to it?
>>>
>>>
>>> There is no limit to the number of software hash rules.
>>>
>>> 4. I have a valid rule, but whenever I
>>> call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why?
>>&g

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

I'm currently in a situation, that no rules work for me whatsoever (same
configuration I described earlier), and they did work for me a few day ago.
I'm using the pfcount_82599.c tester, and added a single hash rule. The
only thing that does work is the default behavior, when using the API
pfring_toggle_filtering_policy(pd, 1) (default to allow - packets arrive
normally) or pfring_toggle_filtering_policy(pd, 0); (default to drop, no
packet arrive to the tester).
This is the part of code:
--
:
 rc = pfring_enable_ring(pd);
 if (rc<0)
printf("pfring_enable_ring() failed. rc=%d\n", rc);

  pfring_toggle_filtering_policy(pd, 1); /* Default to allow */

  if (1) {
hash_filtering_rule rule;
u_int16_t rule_id = 0;

memset(&rule, 0, sizeof(hash_filtering_rule));
rule.proto = 6;
rule.rule_id = rule_id++;
rule.rule_action = dont_forward_packet_and_stop_rule_evaluation;
rule.host4_peer_a = ntohl(inet_addr("10.12.150.231"));
rule.port_peer_a = 2489;
rule.host4_peer_b = ntohl(inet_addr("10.61.12.31"));
rule.port_peer_b = 139;
rc = pfring_handle_hash_filtering_rule(pd, &rule, 1);
if(rc<0)
printf("pfring_add_hash_filtering_rule(%d) failed [errno=%d: %s]\n",
rule.rule_id, errno, strerror(errno));
else
printf("pfring_add_hash_filtering_rule(%d) succeeded\n", rule.rule_id);
  }
--
I don't get any errors when the following code run, but when I bombard the
machine
with a pcap containing more than 600,000 packets of the specified session
that I've expected to be filtered out,
packets of it still received at the tester.

I'm pretty sure that something goes wrong in pf_ring, but I can't tell what.
What is the best way to get debug information, other than reading
/var/log/messages?

Thanks,
Amir


On Wed, Apr 8, 2015 at 1:08 PM, Alfredo Cardigliano 
wrote:

>
> On 05 Apr 2015, at 16:07, Amir Kaduri  wrote:
>
> Hi,
>
> I think I've made some progress: AFAIU, the packets that I see despite the
> rule that supposed to filter them, are packets the receive during the time
> interval from rule-set to rule-apply by pfring.
> I'll appreciate getting some answers about the following:
> 1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way to
> know which rules-ids are set and which are vacant?
> This is because I have to follow the rules-ids when setting them, but
> when I purge them, I don't know which of them were removed.
>
>
> No, unfortunately this is not possible with the current API.
>
> 2. Does this API also purges HW rules?
>
>
> No, It doesn’t.
>
> 3. According to the documentation, I know that HW rules have a limit of
> 32,000. What is the limit for hash rules? IS this limit includes the 32,000
> of the HW, or additional to it?
>
>
> There is no limit to the number of software hash rules.
>
> 4. I have a valid rule, but whenever I
> call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why?
>
>
> pfring_get_hash_filtering_rule_stats() should be used with sw rules to get
> stats from kernel plugins (when used), otherwise there is no stats per rule.
>
> Br
> Alfredo
>
> - I've add the stats code to the pfcount_82599 tester
> - In /var/log/messages I see the following message that is probably
> originated from ring_setsockopt(): "kernel: [PF_RING] Found rule but
> pluginId 0 is not registered"
>
> Thanks,
> Amir
>
>
> On Thu, Apr 2, 2015 at 5:06 PM, Amir Kaduri  wrote:
>
>> Hi Alfredo,
>>
>> Thanks for referring to my question.
>> I hope the following answers:
>>
>> [root@CT10K10G]# cat /etc/pf_ring/pfring.conf
>> min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1
>> enable_ip_defrag=1
>>
>> [root@CT10K10G]# cat /proc/net/pf_ring/info
>> PF_RING Version  : 6.0.1 ($Revision: exported$)
>> Total rings  : 0
>>
>> Standard (non DNA) Options
>> Ring slots   : 1024
>> Slot version : 15
>> Capture TX   : Yes [RX+TX]
>> IP Defragment: Yes
>> Socket Mode  : Standard
>> Transparent mode : No [mode 2]
>> Total plugins    : 0
>> Cluster Fragment Queue   : 0
>> Cluster Fragment Discard : 0
>>
>> Thanks,
>> Amir
>>
>>
>> On Thu, Apr 2, 2015 at 4:10 PM, Alfredo Cardigliano > > wrote:
>>
>>> Hi Amir
>>> how did you load pf_ring.ko? Can I see the command line?
>>> Please also try using latest code from svn, this helps us debugging the
>&g

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Thanks Alfredo for the information.
Appreciated.

Amir

On Wed, Apr 8, 2015 at 1:08 PM, Alfredo Cardigliano 
wrote:

>
> On 05 Apr 2015, at 16:07, Amir Kaduri  wrote:
>
> Hi,
>
> I think I've made some progress: AFAIU, the packets that I see despite the
> rule that supposed to filter them, are packets the receive during the time
> interval from rule-set to rule-apply by pfring.
> I'll appreciate getting some answers about the following:
> 1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way to
> know which rules-ids are set and which are vacant?
> This is because I have to follow the rules-ids when setting them, but
> when I purge them, I don't know which of them were removed.
>
>
> No, unfortunately this is not possible with the current API.
>
> 2. Does this API also purges HW rules?
>
>
> No, It doesn’t.
>
> 3. According to the documentation, I know that HW rules have a limit of
> 32,000. What is the limit for hash rules? IS this limit includes the 32,000
> of the HW, or additional to it?
>
>
> There is no limit to the number of software hash rules.
>
> 4. I have a valid rule, but whenever I
> call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why?
>
>
> pfring_get_hash_filtering_rule_stats() should be used with sw rules to get
> stats from kernel plugins (when used), otherwise there is no stats per rule.
>
> Br
> Alfredo
>
> - I've add the stats code to the pfcount_82599 tester
> - In /var/log/messages I see the following message that is probably
> originated from ring_setsockopt(): "kernel: [PF_RING] Found rule but
> pluginId 0 is not registered"
>
> Thanks,
> Amir
>
>
> On Thu, Apr 2, 2015 at 5:06 PM, Amir Kaduri  wrote:
>
>> Hi Alfredo,
>>
>> Thanks for referring to my question.
>> I hope the following answers:
>>
>> [root@CT10K10G]# cat /etc/pf_ring/pfring.conf
>> min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1
>> enable_ip_defrag=1
>>
>> [root@CT10K10G]# cat /proc/net/pf_ring/info
>> PF_RING Version  : 6.0.1 ($Revision: exported$)
>> Total rings  : 0
>>
>> Standard (non DNA) Options
>> Ring slots   : 1024
>> Slot version : 15
>> Capture TX   : Yes [RX+TX]
>> IP Defragment: Yes
>> Socket Mode  : Standard
>> Transparent mode : No [mode 2]
>> Total plugins: 0
>> Cluster Fragment Queue   : 0
>> Cluster Fragment Discard : 0
>>
>> Thanks,
>> Amir
>>
>>
>> On Thu, Apr 2, 2015 at 4:10 PM, Alfredo Cardigliano > > wrote:
>>
>>> Hi Amir
>>> how did you load pf_ring.ko? Can I see the command line?
>>> Please also try using latest code from svn, this helps us debugging the
>>> issue.
>>>
>>> Br
>>> Alfredo
>>>
>>> On 01 Apr 2015, at 18:22, Amir Kaduri  wrote:
>>>
>>> Hello,
>>>
>>>
>>> I’m using PF_RING-6.0.1.
>>>
>>> I’m trying to develop an application that runs some algorithm consisting
>>> on rules.
>>>
>>> I made some tests using the “pfcount” tester, and unfortunately, I don’t
>>> understand the behavior:
>>>
>>> I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r
>>> –m” which AFAIU, adds a wildcard filter for each incoming packet.
>>>
>>> If I get it correctly, once a rule was added, I should not expect other
>>> packets of the same session to receive, and this is not what I’m getting.
>>>
>>> For example:
>>>
>>> ---
>>>
>>> [root@CT10K10G examples]# ./pfcount -i eth3 -u 2 -v 1 -r -m
>>>
>>> Adding wildcard filtering rules
>>>
>>> Using PF_RING v.6.0.1
>>>
>>> Capturing from eth3 [00:E0:ED:FE:18:19][ifIndex: 11]
>>>
>>> # Device RX channels: 6
>>>
>>> # Polling threads:1
>>>
>>> Dumping statistics on /proc/net/pf_ring/stats/11993-eth3.1074
>>>
>>> 18:52:35.956295950 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
>>> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
>>> 10.70.150.108:60189]
>>> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596843063]
>>> [caplen=128][len=1522][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>>>
>>> Rule 0 added successfully...
>>>
>>

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hi,

I think I've made some progress: AFAIU, the packets that I see despite the
rule that supposed to filter them, are packets the receive during the time
interval from rule-set to rule-apply by pfring.
I'll appreciate getting some answers about the following:
1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way to
know which rules-ids are set and which are vacant?
This is because I have to follow the rules-ids when setting them, but
when I purge them, I don't know which of them were removed.
2. Does this API also purges HW rules?
3. According to the documentation, I know that HW rules have a limit of
32,000. What is the limit for hash rules? IS this limit includes the 32,000
of the HW, or additional to it?
4. I have a valid rule, but whenever I
call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why?
- I've add the stats code to the pfcount_82599 tester
- In /var/log/messages I see the following message that is probably
originated from ring_setsockopt(): "kernel: [PF_RING] Found rule but
pluginId 0 is not registered"

Thanks,
Amir


On Thu, Apr 2, 2015 at 5:06 PM, Amir Kaduri  wrote:

> Hi Alfredo,
>
> Thanks for referring to my question.
> I hope the following answers:
>
> [root@CT10K10G]# cat /etc/pf_ring/pfring.conf
> min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1
> enable_ip_defrag=1
>
> [root@CT10K10G]# cat /proc/net/pf_ring/info
> PF_RING Version  : 6.0.1 ($Revision: exported$)
> Total rings  : 0
>
> Standard (non DNA) Options
> Ring slots   : 1024
> Slot version : 15
> Capture TX   : Yes [RX+TX]
> IP Defragment: Yes
> Socket Mode  : Standard
> Transparent mode : No [mode 2]
> Total plugins: 0
> Cluster Fragment Queue   : 0
> Cluster Fragment Discard : 0
>
> Thanks,
> Amir
>
>
> On Thu, Apr 2, 2015 at 4:10 PM, Alfredo Cardigliano 
> wrote:
>
>> Hi Amir
>> how did you load pf_ring.ko? Can I see the command line?
>> Please also try using latest code from svn, this helps us debugging the
>> issue.
>>
>> Br
>> Alfredo
>>
>> On 01 Apr 2015, at 18:22, Amir Kaduri  wrote:
>>
>> Hello,
>>
>>
>> I’m using PF_RING-6.0.1.
>>
>> I’m trying to develop an application that runs some algorithm consisting
>> on rules.
>>
>> I made some tests using the “pfcount” tester, and unfortunately, I don’t
>> understand the behavior:
>>
>> I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r
>> –m” which AFAIU, adds a wildcard filter for each incoming packet.
>>
>> If I get it correctly, once a rule was added, I should not expect other
>> packets of the same session to receive, and this is not what I’m getting.
>>
>> For example:
>>
>> ---
>>
>> [root@CT10K10G examples]# ./pfcount -i eth3 -u 2 -v 1 -r -m
>>
>> Adding wildcard filtering rules
>>
>> Using PF_RING v.6.0.1
>>
>> Capturing from eth3 [00:E0:ED:FE:18:19][ifIndex: 11]
>>
>> # Device RX channels: 6
>>
>> # Polling threads:1
>>
>> Dumping statistics on /proc/net/pf_ring/stats/11993-eth3.1074
>>
>> 18:52:35.956295950 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
>> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
>> 10.70.150.108:60189]
>> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596843063]
>> [caplen=128][len=1522][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>>
>> Rule 0 added successfully...
>>
>> 18:52:35.956301616 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
>> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
>> 10.70.150.108:60189]
>> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596844523]
>> [caplen=128][len=650][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>>
>> Rule 1 added successfully...
>>
>> 18:52:35.956303262 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
>> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
>> 10.70.150.108:60189]
>> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596845111]
>> [caplen=128][len=1086][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>>
>> Rule 2 added successfully...
>>
>> :
>>
>> ---
>>
>>
>> How come, that once rule #0 was added for [10.61.10.9:52311 ->
>&g

Re: [Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hi Alfredo,

Thanks for referring to my question.
I hope the following answers:

[root@CT10K10G]# cat /etc/pf_ring/pfring.conf
min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1
enable_ip_defrag=1

[root@CT10K10G]# cat /proc/net/pf_ring/info
PF_RING Version  : 6.0.1 ($Revision: exported$)
Total rings  : 0

Standard (non DNA) Options
Ring slots   : 1024
Slot version : 15
Capture TX   : Yes [RX+TX]
IP Defragment: Yes
Socket Mode  : Standard
Transparent mode : No [mode 2]
Total plugins: 0
Cluster Fragment Queue   : 0
Cluster Fragment Discard : 0

Thanks,
Amir


On Thu, Apr 2, 2015 at 4:10 PM, Alfredo Cardigliano 
wrote:

> Hi Amir
> how did you load pf_ring.ko? Can I see the command line?
> Please also try using latest code from svn, this helps us debugging the
> issue.
>
> Br
> Alfredo
>
> On 01 Apr 2015, at 18:22, Amir Kaduri  wrote:
>
> Hello,
>
>
> I’m using PF_RING-6.0.1.
>
> I’m trying to develop an application that runs some algorithm consisting
> on rules.
>
> I made some tests using the “pfcount” tester, and unfortunately, I don’t
> understand the behavior:
>
> I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r
> –m” which AFAIU, adds a wildcard filter for each incoming packet.
>
> If I get it correctly, once a rule was added, I should not expect other
> packets of the same session to receive, and this is not what I’m getting.
>
> For example:
>
> ---
>
> [root@CT10K10G examples]# ./pfcount -i eth3 -u 2 -v 1 -r -m
>
> Adding wildcard filtering rules
>
> Using PF_RING v.6.0.1
>
> Capturing from eth3 [00:E0:ED:FE:18:19][ifIndex: 11]
>
> # Device RX channels: 6
>
> # Polling threads:1
>
> Dumping statistics on /proc/net/pf_ring/stats/11993-eth3.1074
>
> 18:52:35.956295950 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
> 10.70.150.108:60189]
> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596843063]
> [caplen=128][len=1522][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>
> Rule 0 added successfully...
>
> 18:52:35.956301616 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
> 10.70.150.108:60189]
> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596844523]
> [caplen=128][len=650][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>
> Rule 1 added successfully...
>
> 18:52:35.956303262 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
> 10.70.150.108:60189]
> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596845111]
> [caplen=128][len=1086][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>
> Rule 2 added successfully...
>
> :
>
> ---
>
>
> How come, that once rule #0 was added for [10.61.10.9:52311 ->
> 10.70.150.108:60189], I still see such packets in the next lines?
> Shouldn’t they be filtered by the rule that just as added?
>
>
> (BTW, when I use the command “./pfcount -i eth3 -u 1 -v 1 -r –m” (i.e. –u
> is 1 rather than 2), the tester uses hash filters, and in this case, I get
> errors:
>
> 18:53:19.052549112 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
> 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
> 10.70.150.108:60189]
> [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596847159]
> [caplen=128][len=1490][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]
>
> pfring_add_hash_filtering_rule(1) failed)
>
>
> Any help will be appreciated.
>
>
> Thanks,
>
> Amir
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] pfring filtering fails or not clear

[Amir Kaduri]

Hello,



I’m using PF_RING-6.0.1.

I’m trying to develop an application that runs some algorithm consisting on
rules.

I made some tests using the “pfcount” tester, and unfortunately, I don’t
understand the behavior:

I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r –m”
which AFAIU, adds a wildcard filter for each incoming packet.

If I get it correctly, once a rule was added, I should not expect other
packets of the same session to receive, and this is not what I’m getting.

For example:

---

[root@CT10K10G examples]# ./pfcount -i eth3 -u 2 -v 1 -r -m

Adding wildcard filtering rules

Using PF_RING v.6.0.1

Capturing from eth3 [00:E0:ED:FE:18:19][ifIndex: 11]

# Device RX channels: 6

# Polling threads:1

Dumping statistics on /proc/net/pf_ring/stats/11993-eth3.1074

18:52:35.956295950 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
10.70.150.108:60189]
[l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596843063]
[caplen=128][len=1522][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]

Rule 0 added successfully...

18:52:35.956301616 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
10.70.150.108:60189]
[l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596844523]
[caplen=128][len=650][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]

Rule 1 added successfully...

18:52:35.956303262 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
10.70.150.108:60189]
[l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596845111]
[caplen=128][len=1086][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]

Rule 2 added successfully...

:

---



How come, that once rule #0 was added for [10.61.10.9:52311 ->
10.70.150.108:60189], I still see such packets in the next lines? Shouldn’t
they be filtered by the rule that just as added?



(BTW, when I use the command “./pfcount -i eth3 -u 1 -v 1 -r –m” (i.e. –u
is 1 rather than 2), the tester uses hash filters, and in this case, I get
errors:

18:53:19.052549112 [RX][if_index=11][00:08:E3:FF:FC:C8 ->
00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 ->
10.70.150.108:60189]
[l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596847159]
[caplen=128][len=1490][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58]

pfring_add_hash_filtering_rule(1) failed)



Any help will be appreciated.



Thanks,

Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

[Ntop-misc] Question regarding pf_ring filtering

[Amir Kaduri]

Hello,

I recently started using pf_ring, and I'm just getting to know this library.
I need to create a filter that for each session, will start filtering out
packets after receiving the X packet (e.g. X=100).
Ideally, this could be a single filter to all sessions.
Currently, to my best understanding, there is no such "magical" filter.
Meaning, I have to parse all received packets, hold a a sessions table, do
the counting myself, and add the filter on-the-fly myself.
Before developing all this stuff, I thought asking you the experts.
Any advice on the right direction will be much appreciated.

Thanks in advance,
Amir
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc