[dpdk-dev] SEGMENTATION FAULT in kni example : kni_free_kni() call
Hi, im trying the kni example. When i hit ctrl-c in the terminal stopping
the example, the os signals with a segmentation fault.
The issue is in this block of code:
static int kni_free_kni(uint8_t port_id) {
uint8_t i;
struct kni_port_params **p = kni_port_params_array;
if (port_id >= RTE_MAX_ETHPORTS || !p[port_id])
return -1;
-- for (i = 0; i < p[i]->nb_kni; i++) {
-- rte_kni_release(p[i]->kni[i]);
-- p[i]->kni[i] = NULL;
}
rte_eth_dev_stop(port_id);
return 0;
}
The following change fix that issue:
static int kni_free_kni(uint8_t port_id) {
uint8_t i;
struct kni_port_params **p = kni_port_params_array;
if (port_id >= RTE_MAX_ETHPORTS || !p[port_id])
return -1;
++ for (i = 0; i < p[port_id]->nb_kni; i++) {
++ rte_kni_release(p[port_id]->kni[i]);
++ p[port_id]->kni[i] = NULL;
}
rte_eth_dev_stop(port_id);
return 0;
}
Sorry for the basic of my fix ... i dont know the correct style to report
this kind of issues... im just a user of the dpdk library ... but i meet
this bug ...
Regards.
Ariel Horacio Rodriguez.
[dpdk-dev] ISSUE IN rte_bitmap
Hi , i am facing a issue regarding with the maximum amount of bits that the
rte_bitmap allows.
Example :
bmp_mem_size = rte_bitmap_get_memory_footprint(4000);
OK
(bmp_mem_size==2101312 (size in bytes aligned). Around 2MB)
but if i do this:
bmp_mem_size = rte_bitmap_get_memory_footprint(4294967292); (MAX uint32_t).
(bmp_mem_size == 64).
obviously this is wrong , besides the validation problem inside the
function rte_bitmap_get_memory_footprint, the main problem is that the
rte_bitmap api uses word size (4 bytes ints) for calculations of the max
bitmap size and ,of course, make them turn around.
In the rte_sched code there is a similar issue. In theory (By doc) the qos
scheduler can handle a configuration like this:
struct rte_sched_port_hierarchy {
uint32_t queue:2;/**< Queue ID (0 .. 3) */
uint32_t traffic_class:2; /**< Traffic class ID (0 .. 3)*/
uint32_t pipe:20; /**< Pipe ID */ (Compile time configuration) 1M
subscribers
uint32_t subport:6; /**< Subport ID */ 32 subport
uint32_t color:2; /**< Color */
};
If one try to make a struct rte_sched_port* with that configuration (4095
MAX pipe profiles) , the api fails because when the function
rte_sched_port_config() tries to initialize the maximum memory use for that
configuration previously calling rte_sched_port_get_array_base() , that
function generates invalid total size, because the total memory size needed
is beyond 4Gb , bigger than unsigned int max limit. (Again here the code is
using uint32_t data for size calculation).
I know maybe the case use of qos scheduler is not intend for that huge
configuration but looking at the code there is no limitation except for the
4 bytes RSS struct rte_sched_port_hierarchy bit fields .
[dpdk-dev] ISSUE IN rte_bitmap
Hi , i am facing a issue regarding with the maximum amount of bits that the
rte_bitmap allows.
Example :
bmp_mem_size = rte_bitmap_get_memory_footprint(4000);
OK
(bmp_mem_size==2101312 (size in bytes aligned). Around 2MB)
but if i do this:
bmp_mem_size = rte_bitmap_get_memory_footprint(4294967292); (MAX uint32_t).
(bmp_mem_size == 64).
obviously this is wrong , besides the validation problem inside the
function rte_bitmap_get_memory_footprint, the main problem is that the
rte_bitmap api uses word size (4 bytes ints) for calculations of the max
bitmap size and ,of course, make them turn around.
In the rte_sched code there is a similar issue. In theory (By doc) the qos
scheduler can handle a configuration like this:
struct rte_sched_port_hierarchy {
uint32_t queue:2;/**< Queue ID (0 .. 3) */
uint32_t traffic_class:2; /**< Traffic class ID (0 .. 3)*/
uint32_t pipe:20; /**< Pipe ID */ (Compile time configuration) 1M
subscribers
uint32_t subport:6; /**< Subport ID */ 32 subport
uint32_t color:2; /**< Color */
};
If one try to make a struct rte_sched_port* with that configuration (4095
MAX pipe profiles) , the api fails because when the function
rte_sched_port_config() tries to initialize the maximum memory use for that
configuration previously calling rte_sched_port_get_array_base() , that
function generates invalid total size, because the total memory size needed
is beyond 4Gb , bigger than unsigned int max limit. (Again here the code is
using uint32_t data for size calculation).
I know maybe the case use of qos scheduler is not intend for that huge
configuration but looking at the code there is no limitation except for the
4 bytes RSS struct rte_sched_port_hierarchy bit fields .
[dpdk-dev] ISSUE IN rte_bitmap
Hi , i am facing a issue regarding with the maximum amount of bits that the
rte_bitmap allows.
Example :
bmp_mem_size = rte_bitmap_get_memory_footprint(4000);
OK
(bmp_mem_size==2101312 (size in bytes aligned). Around 2MB)
but if i do this:
bmp_mem_size = rte_bitmap_get_memory_footprint(4294967292); (MAX uint32_t).
(bmp_mem_size == 64).
obviously this is wrong , besides the validation problem inside the
function rte_bitmap_get_memory_footprint, the main problem is that the
rte_bitmap api uses word size (4 bytes ints) for calculations of the max
bitmap size and ,of course, make them turn around.
In the rte_sched code there is a similar issue. In theory (By doc) the qos
scheduler can handle a configuration like this:
struct rte_sched_port_hierarchy {
uint32_t queue:2;/**< Queue ID (0 .. 3) */
uint32_t traffic_class:2; /**< Traffic class ID (0 .. 3)*/
uint32_t pipe:20; /**< Pipe ID */ (Compile time configuration) 1M
subscribers
uint32_t subport:6; /**< Subport ID */ 32 subport
uint32_t color:2; /**< Color */
};
If one try to make a struct rte_sched_port* with that configuration (4095
MAX pipe profiles) , the api fails because when the function
rte_sched_port_config() tries to initialize the maximum memory use for that
configuration previously calling rte_sched_port_get_array_base() , that
function generates invalid total size, because the total memory size needed
is beyond 4Gb , bigger than unsigned int max limit. (Again here the code is
using uint32_t data for size calculation).
I know maybe the case use of qos scheduler is not intend for that huge
configuration but looking at the code there is no limitation except for the
4 bytes RSS struct rte_sched_port_hierarchy bit fields .
[dpdk-dev] 4 Traffic classes per Pipe limitation
Hi, im working with the qos scheduler framework and i have a few questions. Why the 4 traffic classes per pipe limitation? . Im developing a deep packet inspection solution for a telecom company and i we need more than just 4 traffic classes per pipe. Im able to recognise almost all layer 7 applications, such as youtube, p2p , netflix , google-ads , etc, etc and i really need to map this type of flows in differents traffic classes. The idea is mark each flow depending on the provisioning information and assign that flows to different subport depending on the information given and assign a pipe with the subscriber contract rate, but we really need to have more than 4 traffic clases, because we want to control the bandwidth of different layer 7 protocols flows. At most we need 32 or 64 traffic classes per subscriber. I understand that in a given interval of time a subscriber dont use more than 4 protocols simultaneously , generally speaking , or 4 traffic classes in dpdk qos terminology, but the framework doesnt allow us to configure more traffic classes. Im looking the code of qos scheduler and im not seeing why this restriction. Is a performance problem, or a waste of resource problem? , maybe when the port grinder search for the active queues for each traffic class the delay of iterating over all pipes and each traffic class is too high. Cisco have a bandwidth managment solution that claims to control a million of subscribers simoultaneosly with 64 traffic classes per subscriber (pipes) and 4 queues per traffic classes (Cisco solution calls traffic clases as "Bandwith controller per service or BWC , a subscriber can have 64 BWC simoultaneasly). Its this posible? maybe this guys implements the bandwidth managment in hardware?. Anyway i really need this feature , but if the qos scheduller cannot scale to more than 4 traffic classes per pipe i would have to implement a custom packet scheduler from scratch and i really dont want to do that. Thanks for the patience, and sorry for my rusty english, im from Argentina. Best Regards. Ariel Horacio Rodriguez, Callis Technologies.
[dpdk-dev] 4 Traffic classes per Pipe limitation
Thanks for the answer, your explanation was perfect. Unfortunally , the client requirements are those, we need at traffic control level around 64 traffic metering controlers (traffic classes) at subscriber level. Each subscriber have a global plan rate (each pipe have the same token bucket configuration), inside that plan there are different rules for the traffic (traffic classes). For Example, facebook traffic, twitter traffic, whatsapp traffic have different plan rates lower than the plan global rate but different than the others protocols. We could group those in one traffic class, but still the 4 traffic classes is a strong limitation for us, beacause each protocol mapped to a traffic class share the same configuration (facebook traffic, twitter traffic have had the same rate and more, they compete for the same traffic class rate). We have to compete against cisco bandwith control solution and at least we need to offer the same features. The cisco solution its a DPI but also a traffic control solution, its permit priorization of traffic and manage the congestion inside the network per subscriber and per application service. So apperently the dpdk qos scheduller doesnt fit for our needs. Anyway, i still doesnt understand the traffic classes limitation. Inside the dpdk code of the qos scheduler i saw this: /** Number of queues per pipe traffic class. Cannot be changed. */ #define RTE_SCHED_QUEUES_PER_TRAFFIC_CLASS4 I follow where the code use that define and except for the struct rte_sched_port_hierarchy where its mandatory a bitwise field of two (0...3) , i dont see where is the limitation here (except for performance). Its worth to change the code to support more than 4 traffic classes, well i could try to change the code more precisely jejeje. I just want to know if there are another limitation than a design desicion of that number. I dont want to make the effort for nothing maybe you guys can help me to understand why the limitation. I strongly use the dpdk solution for feed our dpi solution, i wont change that because work greats!!! but its difficult to develop a traffic control managment from scratch and integrated with the dpdk in a clean way without touching the dpdk api, you guys just done that with the qos scheduler, i dont want to reinvent the wheel. Again thank you for the patience, and for your expertise. Regards, Ariel Horacio Rodriguez. Callis Technologies. On Fri, Nov 29, 2013 at 3:45 PM, Dumitrescu, Cristian < cristian.dumitrescu at intel.com> wrote: > Hi Ariel, some comments inlined below. Regards, Cristian > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Ariel Rodriguez > Sent: Thursday, November 28, 2013 8:53 PM > To: dev at dpdk.org > Subject: [dpdk-dev] 4 Traffic classes per Pipe limitation > > Hi, im working with the qos scheduler framework and i have a few > questions. Why the 4 traffic classes per pipe limitation? . > > [Cristian] The DPDK hierarchical scheduler allows for 4 traffic classes > with 4 packet queues per traffic class for each pipe (user). Traffic > classes and scheduled in strict priority, while queues within a pipe > traffic class are scheduled using byte-level Weighted Round Robin (WRR) > with configured weights. Since we have 16 packet queues per pipe (user), we > could argue that actually 16 (sub)traffic classes are supported. When the > weight ratio between different queues of the same traffic class is high > (e.g. 4:1, 8:1, 16:1, etc), then WRR starts behaving like strict priority. > If your mapping to traffic classes is done using DSCP, you can simply map > some DSCP values to different queues within same traffic class as well. > > Im developing a deep packet inspection solution for a telecom company and i > we need more than just 4 traffic classes per pipe. Im able to recognise > almost all layer 7 applications, such as youtube, p2p , netflix , > google-ads , etc, etc and i really need to map this type of flows in > differents traffic classes. > > [Cristian] Not sure I completely agree with you here. > The traffic classes are there for the reason of providing different levels > of delay (latency), delay variation (jitter), packet loss rate, etc. So, > for example, one possible mapping of traffic types to the 4 traffic classes > might be: voice = TC0 (highest priority), interactive video = TC1, > non-interactive/cached video = TC2, best effort traffic (file downloads, > web browsing, email, etc) = TC3 (lowest priority). In my opinion, youtube > and netflix could be considered as being part of the same traffic class > (e.g. TC2), as they have very similar (if not identical) requirements, > probably same for p2p and google-ads, email, web browsing, etc (where best > effort traffic class is probably applic
[dpdk-dev] 4 Traffic classes per Pipe limitation
Ok thats give the reason i need, yes i could change the number of bits of ,for example , pipe size which is 20 bytes but we need around a million of pipe (the telecom has a million of concurent subscribers). Thank you so much, i have to think about this, for the moment i believe we will use the 4 traffic classes and group the differents protocols to a traffic class. Maybe later i will ask some questions about the traffic metering. Thank you again , best regards, Ariel Horacio Rodriguez, Callis Technologies. On Fri, Nov 29, 2013 at 6:26 PM, Stephen Hemminger < stephen at networkplumber.org> wrote: > On Fri, 29 Nov 2013 17:50:34 -0200 > Ariel Rodriguez wrote: > > > Thanks for the answer, your explanation was perfect. > Unfortunally > > , the client requirements are those, we need at traffic control level > > around 64 traffic metering controlers (traffic classes) at subscriber > level. > > I think you maybe confused by the Intel QoS naming. It is better to > think about it as 3 different classification levels and not get too hung > up about the naming. > > The way to do what you want that is with 64 different 'pipes'. > In our usage: > subport => VLAN > pipe=> subscriber matched by tuple > traffic class => mapping from DSCP to TC > > > > Each subscriber have a global plan rate (each pipe have the > same > > token bucket configuration), inside that plan there are different rules > for > > the traffic (traffic classes). For Example, facebook traffic, twitter > > traffic, whatsapp traffic have different plan rates lower than the plan > > global rate but different than the others protocols. We could group those > > in one traffic class, but still the 4 traffic classes is a strong > > limitation for us, beacause each protocol mapped to a traffic class share > > the same configuration (facebook traffic, twitter traffic have had the > same > > rate and more, they compete for the same traffic class rate). > > We have to compete against cisco bandwith control solution and > at > > least we need to offer the same features. The cisco solution its a DPI > but > > also a traffic control solution, its permit priorization of traffic and > > manage the congestion inside the network per subscriber and per > application > > service. So apperently the dpdk qos scheduller doesnt fit for our needs. > > Anyway, i still doesnt understand the traffic classes > limitation. > > Inside the dpdk code of the qos scheduler i saw this: > > > > /** Number of queues per pipe traffic class. Cannot be changed. */ > > #define RTE_SCHED_QUEUES_PER_TRAFFIC_CLASS4 > > > I follow where the code use that define and except for the > struct > > rte_sched_port_hierarchy where its mandatory a bitwise field of two > (0...3) > > , i dont see where is the limitation here (except for performance). Its > > worth to change the code to support more than 4 traffic classes, well i > > could try to change the code more precisely jejeje. I just want to know > if > > there are another limitation than a design desicion of that number. I > dont > > want to make the effort for nothing maybe you guys can help me to > > understand why the limitation. > > I strongly use the dpdk solution for feed our dpi solution, i > > wont change that because work greats!!! but its difficult to develop a > > traffic control managment from scratch and integrated with the dpdk in a > > clean way without touching the dpdk api, you guys just done that with the > > qos scheduler, i dont want to reinvent the wheel. > > Again thank you for the patience, and for your expertise. > > > The limitation on number's of TC (and pipes) comes from the number of > bits available. Since the QoS code overloads the 32 bit RSS field in > the mbuf there isn't enough bits to a lot. But then again if you add lots > of pipes or subports the memory footprint gets huge. > > Since it is open source, you could reduce the number of bits for one > field and increase the other. But having lots of priority classes > would lead to poor performance and potential starvation. >
[dpdk-dev] Qos scheduler question.
Hi , we re implementing bandwith controller per user with the dpdk qos scheduler framework. I want to know if the framework support dynamic changes in the rte_sched_port structure. For example, we want to give the possibility to change the configuration of the different bucket rates int the port. We have a managment tool thats allow to change the configuration of the subport , ports, and pipe. We group a set of subscribers in a pipe. When the subscriber login in the system, the data of the login specify the subport and pipe where the subscriber will be bounded. What we want is give the customer the ability to change that login information based on their bussiness logic. So , Can we change the rte_sched_port configuration when a user apllies new configuration to the system? . I found a solution where we just create a parallel structure from scratch , and then notifies to the logical core assigned to the bandwith managment via rte_ring, when the notification is process , that core just change the reference of the rte_sched_port and frees the old one. i imagine if the old structure has packets on his queues , this packets are lost, and probably leaked. if there a way to achieve this, or the framework just support a static configuration
[dpdk-dev] Qos scheduler question.
Hi , we re implementing bandwith controller per user with the dpdk qos scheduler framework. I want to know if the framework support dynamic changes in the rte_sched_port structure. For example, we want to give the possibility to change the configuration of the different bucket rates int the port. We have a managment tool thats allow to change the configuration of the subport , ports, and pipe. We group a set of subscribers in a pipe. When the subscriber login in the system, the data of the login specify the subport and pipe where the subscriber will be bounded. What we want is give the customer the ability to change that login information based on their bussiness logic. So , Can we change the rte_sched_port configuration when a user apllies new configuration to the system? . I found a solution where we just create a parallel structure from scratch , and then notifies to the logical core assigned to the bandwith managment via rte_ring, when the notification is process , that core just change the reference of the rte_sched_port and frees the old one. i imagine if the old structure has packets on his queues , this packets are lost, and probably leaked. if there a way to achieve this, or the framework just support a static configuration On Tue, May 20, 2014 at 3:58 PM, Ariel Rodriguez wrote: > Hi , we re implementing bandwith controller per user with the dpdk qos > scheduler framework. > > I want to know if the framework support dynamic changes in the > rte_sched_port structure. > For example, we want to give the possibility to change the configuration > of the different bucket rates int the port. We have a managment tool thats > allow to change the configuration of the subport , ports, and pipe. We > group a set of subscribers in a pipe. When the subscriber login in the > system, the data of the login specify the subport and pipe where the > subscriber will be bounded. What we want is give the customer the ability > to change that login information based on their bussiness logic. > So , Can we change the rte_sched_port configuration when a user apllies > new configuration to the system? . I found a solution where we just create > a parallel structure from scratch , and then notifies to the logical core > assigned to the bandwith managment via rte_ring, when the notification > is process , that core just change the reference of the rte_sched_port and > frees the old one. i imagine if the old structure has packets on his queues > , this packets are lost, and probably leaked. if there a way to achieve > this, or the framework just support a static configuration >
[dpdk-dev] Please any one who can help me with librte_sched
Hello , this is my third mail , the previous mails have not been answered yet. I justo need someone explains to me how the librte_sched framework behaves in a specific situation. I have a managment application , this connects with a ring with the tx core, when a user applies some configuration of the bandwith mangement , the tx core read the message in the ring parse the configuration in a rte_port_params struct , subport_params and pipe_params, then creates a new rte_sched from scratch , and then changes the pointer of the current rte_sched_port currently doing scheduling and then the code execurte rte_sched_port_free() for the unreference (reference by temporal pointer) rte_sched_port . This is the only way i found for applying dinamic configuration or changes to the qos framework. So, with this, what happens with the packets attached to the old rte_sched_port while is deleted? are those lost packets inside the rte_sched_port generates memory leaks? how can i recover this packets _ just dequeing from the port scheduler? Where the port scheduler indicates empty packets in the queu state? Is there a better way to achieve this kind of behaviour? i just need to update the rte_sched_port configuration dinamically, and i want to change the current pipe configuration and sub port configuration also. Regards .
[dpdk-dev] Please any one who can help me with librte_sched
Thank you perfect explanation, i think im going to creating a new parallel rte_sched_port and change the reference with managment core updating the tx/sched core. So, what happens with the packets on the old reference if i just do rte_port_free on it, are them leaked? Is there a why to flush the rte_sched_port or maybe gets the packet total size somewhere?. Anyway the rcu algoritm fits ok in this aproach ... but maybe there is a way to flush the old reference port, and work from there with the recently created rte_sched_port Regars, Ariel. On Tue, May 27, 2014 at 3:31 PM, Dumitrescu, Cristian < cristian.dumitrescu at intel.com> wrote: > Hi Ariel, > > What's wrong with calling rte_sched_subport_config() and > rte_sched_pipe_config() during run-time? > > This assumes that: > > 1. Port initialization is done, which includes the following: > a) the number of subports, pipes per subport are fixed > b) the queues are all created and their size is fixed > c) the pipe profiles are defined > d) Basically the maximal data structures get created (maximum number of > supports, pipes and queues) with no run-time changes allowed, apart for the > bandwidth related parameters. Queues that do not receive packets are not > used now, they will be used as soon as they get packets. The > packets-to-queues mapping logic can change over time, as well as the level > of activity for different users/queues. > > 2. The CPU core calling the subport/pipe config functions is the same as > the core doing enque/dequeue for this port (for thread safety reasons). > a) As you say, the management core can send update requests to the core > running the scheduler, with the latter sampling the request queue regularly > and performing the updates. > > Regards, > Cristian > > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Stephen Hemminger > Sent: Tuesday, May 27, 2014 5:35 PM > To: Ariel Rodriguez > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] Please any one who can help me with librte_sched > > On Tue, 27 May 2014 10:33:02 -0300 > Ariel Rodriguez wrote: > > > Hello , this is my third mail , the previous mails have not been answered > > yet. > > > > I justo need someone explains to me how the librte_sched framework > behaves > > in a specific situation. > > > > I have a managment application , this connects with a ring with the tx > > core, when a user applies some configuration of the bandwith mangement , > > the tx core read the message in the ring parse the configuration in a > > rte_port_params struct , subport_params and pipe_params, then creates a > new > > rte_sched from scratch , and then changes the pointer of the current > > rte_sched_port currently doing scheduling and then the code execurte > > rte_sched_port_free() for the unreference (reference by temporal pointer) > > rte_sched_port . This is the only way i found for applying dinamic > > configuration or changes to the qos framework. > > So, with this, what happens with the packets attached to the old > > rte_sched_port while is deleted? are those lost packets inside the > > rte_sched_port generates memory leaks? how can i recover this packets _ > > just dequeing from the port scheduler? Where the port scheduler > indicates > > empty packets in the queu state? > > > > Is there a better way to achieve this kind of behaviour? i just need to > > update the rte_sched_port configuration dinamically, and i want to > change > > the current pipe configuration and sub port configuration also. > > > > Regards . > > If you need to do dynamic changes, I would recommend using an RCU type > algorithm where you exchange in new parameters and then cleanup/free > after a grace period. See http://lttng.org/urcu > > -- > Intel Shannon Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County Kildare > Registered Number: 308263 > Business address: Dromore House, East Park, Shannon, Co. Clare > > This e-mail and any attachments may contain confidential material for the > sole use of the intended recipient(s). Any review or distribution by others > is strictly prohibited. If you are not the intended recipient, please > contact the sender and delete all copies. > > >
[dpdk-dev] Please any one who can help me with librte_sched
Ok i can do that... but still is there a way to ask to the rte_sched_port something like is_empty ... Or simply if the dequeue function return 0 packets retrieved from the old port structure running in other core, Can i assume that port is empty with that? Regards Ariel. On May 28, 2014 7:10 AM, "Dumitrescu, Cristian" < cristian.dumitrescu at intel.com> wrote: > Hi Ariel, > > > > I think you put your finger precisely on the problem associated with your > approach: you have to iterate through all the queues and free up the > packets, which takes a lot of time. Obviously this is not done by the > rte_sched API. > > > > Maybe a limited workaround for this approach would be to create and > service the parallel rte_sched using a different CPU core, while the > previous CPU core takes its time to free up all the packets and data > structures correctly. > > > > Regards, > > Cristian > > > > *From:* Ariel Rodriguez [mailto:arodriguez at callistech.com] > *Sent:* Wednesday, May 28, 2014 1:46 AM > *To:* Dumitrescu, Cristian > *Cc:* Stephen Hemminger; dev at dpdk.org > *Subject:* Re: [dpdk-dev] Please any one who can help me with librte_sched > > > > Thank you perfect explanation, i think im going to creating a new parallel > rte_sched_port and change the reference with managment core updating the > tx/sched core. So, what happens with the packets on the old reference if i > just do rte_port_free on it, are them leaked? Is there a why to flush the > rte_sched_port or maybe gets the packet total size somewhere?. > > Anyway the rcu algoritm fits ok in this aproach ... but maybe there is a > way to flush the old reference port, and work from there with the recently > created rte_sched_port > > > > Regars, > > Ariel. > > > > On Tue, May 27, 2014 at 3:31 PM, Dumitrescu, Cristian < > cristian.dumitrescu at intel.com> wrote: > > Hi Ariel, > > What's wrong with calling rte_sched_subport_config() and > rte_sched_pipe_config() during run-time? > > This assumes that: > > 1. Port initialization is done, which includes the following: > a) the number of subports, pipes per subport are fixed > b) the queues are all created and their size is fixed > c) the pipe profiles are defined > d) Basically the maximal data structures get created (maximum number of > supports, pipes and queues) with no run-time changes allowed, apart for the > bandwidth related parameters. Queues that do not receive packets are not > used now, they will be used as soon as they get packets. The > packets-to-queues mapping logic can change over time, as well as the level > of activity for different users/queues. > > 2. The CPU core calling the subport/pipe config functions is the same as > the core doing enque/dequeue for this port (for thread safety reasons). > a) As you say, the management core can send update requests to the core > running the scheduler, with the latter sampling the request queue regularly > and performing the updates. > > Regards, > Cristian > > > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Stephen Hemminger > Sent: Tuesday, May 27, 2014 5:35 PM > To: Ariel Rodriguez > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] Please any one who can help me with librte_sched > > On Tue, 27 May 2014 10:33:02 -0300 > Ariel Rodriguez wrote: > > > Hello , this is my third mail , the previous mails have not been answered > > yet. > > > > I justo need someone explains to me how the librte_sched framework > behaves > > in a specific situation. > > > > I have a managment application , this connects with a ring with the tx > > core, when a user applies some configuration of the bandwith mangement , > > the tx core read the message in the ring parse the configuration in a > > rte_port_params struct , subport_params and pipe_params, then creates a > new > > rte_sched from scratch , and then changes the pointer of the current > > rte_sched_port currently doing scheduling and then the code execurte > > rte_sched_port_free() for the unreference (reference by temporal pointer) > > rte_sched_port . This is the only way i found for applying dinamic > > configuration or changes to the qos framework. > > So, with this, what happens with the packets attached to the old > > rte_sched_port while is deleted? are those lost packets inside the > > rte_sched_port generates memory leaks? how can i recover this packets _ > > just dequeing from the port scheduler? Where the port scheduler > indicates > > empty packets in the queu state? > > &
[dpdk-dev] Fwd: RE: Please any one who can help me with librte_sched
-- Forwarded message -- From: "Ariel Rodriguez" Date: May 29, 2014 8:59 PM Subject: RE: [dpdk-dev] Please any one who can help me with librte_sched To: "Dumitrescu, Cristian" Cc: Hi christian. I am aware of that... maybe i wold have to explain better my approach. 1) The client applies a new configuration, its a requirement for our customer that they could change port, subport,pipe rate and also pipe profiles in a dynamic way. So, i think the solution on this way: First when a customer applies the config , the managment core parse the conf and then creates a new rte_sched_port from scratch with the changes in pipes profiles port subport and pipe rate etc. When the managment core is done with that , it send a command update via a rte_ring and the data in the ring is actually a structure wrapping the rte_sched_port recently created. 2) The tx core which is also the same core where its enqueue and dequeue packets from the "old" rte_sched_port, reads the rte_ring and extract the rte_sched_port that the customer applies. So i will just let the two of it lives together and the tx core enqueue in the new port only, and dequeue in a round robin manner from the two rte_sched_port until the "old one" become empty. Then this "live together" state ends, and the tx core is switch to the old state in which just enqueue and dequeue from one rte_sched_port only. Just a note we have a rx core in which the packets are read from the net port and send to severals worker ring . And we have several "workers" cores that just read from the ring and pass the packets to the ,i believe you know ,Qosmos ixEngine dpi. Based on the classification that qosmos delivers , we put that packet in a specific queue from specific pipe and send to the tx ring . We are doing qos scheduling based on the dinamic information that qosmos delivers. Then the tx core reads from that ring and behave like i just told. Thats our architecture in a few words ... i wide open to suggestions je je je. Im planning when i have a little time to read the qos scheduler code in a conscious way because im interesting in the aproach of scheduling packets in that way... because my other option is a hardware solution and if so i would have to change the perspective radically. On May 29, 2014 8:07 PM, "Dumitrescu, Cristian" < cristian.dumitrescu at intel.com> wrote: > Hi Ariel, > > > > So how are you making sure you are not getting more packets in while you > are waiting for dequeue() to return zero for some time? > > > > As stated, I don?t like this approach, firstly because it forces the > on-the-fly BW configuration changes to result in dropping packets for a > considerable amount of time in an uncontrolled way, and it should not be > this way. Secondly, because making it work with no buffer leakage and no > race conditions is probably not that simple ;) > > > > Regards, > > Cristian > > > > > > *From:* Ariel Rodriguez [mailto:arodriguez at callistech.com] > *Sent:* Wednesday, May 28, 2014 11:50 AM > *To:* Dumitrescu, Cristian > *Cc:* dev at dpdk.org; Stephen Hemminger > *Subject:* RE: [dpdk-dev] Please any one who can help me with librte_sched > > > > Ok i can do that... but still is there a way to ask to the rte_sched_port > something like is_empty > ... Or simply if the dequeue function return 0 packets retrieved from the > old port structure running in other core, > Can i assume that port is empty with that? > > Regards > > Ariel. > > On May 28, 2014 7:10 AM, "Dumitrescu, Cristian" < > cristian.dumitrescu at intel.com> wrote: > > Hi Ariel, > > > > I think you put your finger precisely on the problem associated with your > approach: you have to iterate through all the queues and free up the > packets, which takes a lot of time. Obviously this is not done by the > rte_sched API. > > > > Maybe a limited workaround for this approach would be to create and > service the parallel rte_sched using a different CPU core, while the > previous CPU core takes its time to free up all the packets and data > structures correctly. > > > > Regards, > > Cristian > > > > *From:* Ariel Rodriguez [mailto:arodriguez at callistech.com] > *Sent:* Wednesday, May 28, 2014 1:46 AM > *To:* Dumitrescu, Cristian > *Cc:* Stephen Hemminger; dev at dpdk.org > *Subject:* Re: [dpdk-dev] Please any one who can help me with librte_sched > > > > Thank you perfect explanation, i think im going to creating a new parallel > rte_sched_port and change the reference with managment core updating the > tx/sched core. So, what happens with the packets on the old reference if i > just do rte_port_free on it, are them leaked? Is there a why to
[dpdk-dev] socket programming with DPDK?
Hi , im succesfully integrate boost asio c++ code with dpdk libraries. I do packet processing with dpdk collecting statistics of protocol usage and then use a extern function to send the statistics (custom struct) through a boost socket in c++ code. Boost libraries use netinet/in.h internally . Maybe the workaround is separete the compiling units and not mix the header files. Maybe this can help you. Regards. On Nov 15, 2014 10:28 AM, "Neil Horman" wrote: > On Sat, Nov 15, 2014 at 04:08:47PM +0900, Choonho Son wrote: > > Hi, > > > > I am making netflow collector with DPDK. > > I need to export result to another server with socket programming. > > But I can not include which defines struct sockaddr_in. > > > > How can I make application with traditional socket programming and DPDK? > > > > Thanks. > > > > Choonho Son > > > Why can't you add netinet/in.h? Are you getting an error when doing so? I > vaguely recall someone posting a fix for an include error for that file, > but I > don't recall its disposition > Neil > >
[dpdk-dev] "virtual" C++ keyword used in rte_devargs.h and causes compilation error in C++
"virtual" is a reserved word in c++. When the c++ compiler "g++" use that
header in a "extern way (just generate standard typo for function
identifiers)", there`s not way that the compiler posible "turn off" the
"virtual" reserved word. If, for example, you guys use the "new" word ...
its just the same as virtual.
Regards.
On Sun, Feb 8, 2015 at 3:33 AM, Ming Zhao
wrote:
> In fact the current rte_devargs.h header is enclosed inside extern C {}
> block already. But it looks like it's not sufficient. Also there is also
> the case that rte_devargs.virtual field could be accessed inside a cpp
> file.
>
> On 02/07/2015 12:23 PM, Neil Horman wrote:
> > On Fri, Feb 06, 2015 at 11:24:15PM -0800, Ming Zhao wrote:
> >> The code is in rte_devargs.h:
> >>
> >> rte_devargs.h:
> >>
> >> /** Used if type is RTE_DEVTYPE_VIRTUAL. */
> >> struct {
> >> /** Driver name. */
> >> char drv_name[32];
> >> } virtual;
> >> };
> >>
> >> Which caused clang compiler to report error when this file is included
> >> by a cpp file, the error message is:
> >>
> >> In file included from net/dpdk/testing/base-test.cc:3:
> >> In file included from net/dpdk/testing/base-test.h:8:
> >> third-party/dpdk/lib/librte_eal/common/include/rte_devargs.h:89:5:
> >> error: 'virtual' can only appear on non-static
> >> member functions
> >> } virtual;
> >> ^
> >>
> >> I think we should try to pick another name for this field. I would
> >> suggest calling it "vdev" instead, or I'll be happy to take another name
> >> if someone comes with a different idea.
> >>
> >> Thanks!
> >> Ming
> >>
> > You could do that, but it seems like it shouldn't be necessecary.
> Shouldn't the
> > solution just be to encapsulate either the header file or the #include
> directive
> > from the C++ file with extern C { }?
> > Neil
> >
>
[dpdk-dev] Problem after hours of running rte_eth_rx_burst
After several hours (6 hour average) of running a dpdk application , rte_eth_rx_burst suddenly fills only one mbuf with no data, thats is an mbuf with mbuf->pool == NULL && m->buf_physaddr == 0 && m->buf_addr == NULL. Obviosly that breaks our application. (rte_mbuf_sanity_check abort the program) How can we track the source of this kind of mis- behavoir? We are using dpdk 1.6.0r2 and we also use the qos framework api. The nic is 82599ES 10-Gigabit SFI/SFP+ with tapped traffic. The use case is simply, our client is using a traffic tap to divert a copy of around 10gbps of traffic to our appliance. We use a rxtx code similar to the load_balancer example. We read in a pair of rx queue and the use a hash function over the source ip field to deliver the packet in a worker core. Then , when the worker core finishes to process that packet and it is delivery to the tx core. The tx core enqueue the packet to the qos framework, and just a few lines code later dequeue several packet from the qos scheduler. Because we are using a tap to divert a copy of the traffic , we disable the tx code to the phisycal nic, so when we dequeue packets from the qos scheduler wi just drop all of that packets. Of course there is a reason why we use the qos scheduler code without physically transmiting a packet, and is because we just want a few stats about the qos framework behaviour. Any ideas?
[dpdk-dev] URGENT please help. Issue on ixgbe_tx_free_bufs version 2.0.0
Dear dpdk experts.
Im having a recurrent segmentation fault under the
function ixgbe_tx_free_bufs (ixgbe_rxtx.c:150) (i enable -g3 -O0).
Surfing the core dump i find out this:
txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]);
txq->tx_next_dd = 31
txq->txq->tx_rs_thresh=32
Obviosly txep points out to the first element but
*(txep).mbuf == INVALID MBUF ADDRESS
The same applies to
*(txep+1).mbuf ; *(txep +2).mbuf;*(txep+3).mbuf
from *(txep+4) .mbuf to *(txep+31).mbuf seems to be valid because im able
to derefence the mbuf's
Note:
I disable CONFIG_RTE_IXGBE_INC_VECTOR because i gets similiar behavior , I
thought the problem would disappear disabling that feature.
the program always runs well up to 4 or 5 hours and then crash ... always
in the same line.
this is the backtrace of the program:
#0 0x00677a64 in rte_atomic16_read (v=0x47dc14c18b14) at
/opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/generic/rte_atomic.h:151
#1 0x00677c1d in rte_mbuf_refcnt_read (m=0x47dc14c18b00) at
/opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:411
#2 0x0067a13c in __rte_pktmbuf_prefree_seg (m=0x47dc14c18b00) at
/opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:778
#3 rte_pktmbuf_free_seg (m=0x47dc14c18b00) at
/opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:810
#4 ixgbe_tx_free_bufs (txq=0x7ffb40ae52c0) at
/opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:150
#5 tx_xmit_pkts (tx_queue=0x7ffb40ae52c0, tx_pkts=0x64534770 ,
nb_pkts=32) at /opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:256
#6 0x0067c6f3 in ixgbe_xmit_pkts_simple (tx_queue=0x7ffb40ae52c0,
tx_pkts=0x64534570 , nb_pkts=80) at
/opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:343
#7 0x004ec93d in rte_eth_tx_burst (port_id=1 '\001', queue_id=0,
tx_pkts=0x64534570 , nb_pkts=144) at
/opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_ethdev.h:2572
frame 5 tx queue:
print *txq
{tx_ring = 0x7ffd9d3f1880, tx_ring_phys_addr = 79947569280, sw_ring =
0x7ffb40ae1280, tdt_reg_addr = 0x7fff0002a018, nb_tx_desc = 1024, tx_tail =
1008, tx_free_thresh = 32, tx_rs_thresh = 32, nb_tx_used = 0,
last_desc_cleaned = 1023, nb_tx_free = 15, tx_next_dd = 31, tx_next_rs =
1023, queue_id = 0, reg_idx = 0, port_id = 1 '\001', pthresh = 32 ' ',
hthresh = 0 '\000', wthresh = 0 '\000', txq_flags = 3841, ctx_curr = 0,
ctx_cache = {{
flags = 0, tx_offload = {data = 0, {l2_len = 0, l3_len = 0, l4_len =
0, tso_segsz = 0, vlan_tci = 0}}, tx_offload_mask = {data = 0, {l2_len = 0,
l3_len = 0, l4_len = 0, tso_segsz = 0, vlan_tci = 0}}}, {flags = 0,
tx_offload = {
data = 0, {l2_len = 0, l3_len = 0, l4_len = 0, tso_segsz = 0,
vlan_tci = 0}}, tx_offload_mask = {data = 0, {l2_len = 0, l3_len = 0,
l4_len = 0, tso_segsz = 0, vlan_tci = 0, ops = 0x7616d0 ,
tx_deferred_start = 0 '\000'}
Please help me !!!
Regards.
Ariel Rodriguez.
[dpdk-dev] URGENT please help. Issue on ixgbe_tx_free_bufs version 2.0.0
Thank you very much for your rapid response. 1) IO part is the same as load balancer. The worker part is different. The tx part use qos scheduler framework also. I will try to run the example and see what happends. 2) yes i can. I will do that too. The nic is 82599ES 10-Gigabit SFI/SFP+ with tapped traffic (is a hardware bypass device silicom vendor). I develop a similar app without the tx part. It just received a copy of the traffic (around 6gbps and 40 concurrent flows) and then free the mbufs. It works like a charm. Is strange this issue ... If i disabled the qos scheduler code and the tx code dropping all packets instead of rte_eth_tx_burst ( is like disabling tx core) the issue is happening in rte_eth_rx_burst returning corrupted mbuf (rx core) Could the nic behave anormally? I will try the 2 things you comment before. Regards . Ariel Horacio Rodriguez On Tue, Nov 10, 2015 at 01:35:21AM -0300, Ariel Rodriguez wrote: > Dear dpdk experts. > > Im having a recurrent segmentation fault under the > function ixgbe_tx_free_bufs (ixgbe_rxtx.c:150) (i enable -g3 -O0). > > Surfing the core dump i find out this: > > txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]); > > txq->tx_next_dd = 31 > txq->txq->tx_rs_thresh=32 > > Obviosly txep points out to the first element but > > *(txep).mbuf == INVALID MBUF ADDRESS > > The same applies to > > *(txep+1).mbuf ; *(txep +2).mbuf;*(txep+3).mbuf > > from *(txep+4) .mbuf to *(txep+31).mbuf seems to be valid because im able > to derefence the mbuf's > > > Note: > > I disable CONFIG_RTE_IXGBE_INC_VECTOR because i gets similiar behavior , I > thought the problem would disappear disabling that feature. > > > the program always runs well up to 4 or 5 hours and then crash ... always > in the same line. > > this is the backtrace of the program: > > #0 0x00677a64 in rte_atomic16_read (v=0x47dc14c18b14) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/generic/rte_atomic.h:151 > #1 0x00677c1d in rte_mbuf_refcnt_read (m=0x47dc14c18b00) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:411 > #2 0x0067a13c in __rte_pktmbuf_prefree_seg (m=0x47dc14c18b00) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:778 > #3 rte_pktmbuf_free_seg (m=0x47dc14c18b00) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:810 > #4 ixgbe_tx_free_bufs (txq=0x7ffb40ae52c0) at > /opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:150 > #5 tx_xmit_pkts (tx_queue=0x7ffb40ae52c0, tx_pkts=0x64534770 , > nb_pkts=32) at /opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:256 > #6 0x0067c6f3 in ixgbe_xmit_pkts_simple (tx_queue=0x7ffb40ae52c0, > tx_pkts=0x64534570 , nb_pkts=80) at > /opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:343 > #7 0x004ec93d in rte_eth_tx_burst (port_id=1 '\001', queue_id=0, > tx_pkts=0x64534570 , nb_pkts=144) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_ethdev.h:2572 > Hi, I'd like a bit more information to help debug your problem: * what application are you running when you see this crash? If it's an app of your own making, can you reproduce the crash using one of the standard DPDK apps, or example apps, e.g. testpmd, l2fwd, etc. * Can you also try to verify if the crash occurs with the latest DPDK code available in git from dpdk.org? Regards, /Bruce
[dpdk-dev] URGENT please help. Issue on ixgbe_tx_free_bufs version 2.0.0
Hi Bruce, im going to list the results after the test?s. I will start with the second hint you proposed: 2) I upgrade our custom dpdk application with the latest dpdk code (2.1.0) and the issue still there. 1) I test the load balancer app with the latest dpdk code (2.1.0) with the nic 82599ES 10-Gigabit SFI/SFP+ with tapped traffic and the results are: a) Work fine after 6 hours of running. (For timing issues i cant wait longer but the issue always happend before 5 hours of running so i supposed we are fine in this test). b) I made a change to load balancer code to behave as our dpdk application in the workers code. This change is just for giving the workers code enough load (load in terms of core frecuency) that made the rx core drop several packet because ring between workers and rx core is full. (Our application drop several packets because the workers code are not fast enough). In the last test, the segmentation fault arise , just in the same line that i previously report. Debugging and reading the code in the ixgbe_rxtx.c i see some weird things. - The core dump of the issue always is around line 260 in the ixgbe_rxtx.c code. - Looking at the function "ixgbe_tx_free_bufs" at line 132 , i understand there is a test for looking at the rs bit write back mechanism. The IXGBE_ADVTXD_STAT_DD is set and then the code type cast to ixgbe_tx_entry from the sw_ring in the tx queue (variable name txep). - The txep->mbuf entry is totally corrupted beacause has a invalid memory address, obviously i compared that memory address with the mbuf mempool and is not even close to be valid. But the address of ixgbe_tx_entry is valid and in the range of the zmalloc sotware ring structure constructed at initialization. - The txep pointer is the first one in the sw_ring. That because txq->tx_next_dd is 31 and txq->tx_rs_thresh is 32. txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]); - txq->tx_rs_thresh is 32. I use the default values just setting null in the corresponding *_queue_setup functions. - The weirdess thing is that the next entry on the software ring (next ixgbe_tx_entry) is valid and has a valid mbuf memory address. I dont know how to continue , because im tryng to find out where i could corrupt the mbuf associated with the ixgbe_tx_entry. I debug and test all part of the worker core code , finding out a bad mbuf or a mbuf corruption before enqueue on the tx ring. The tx core and the rx core is just the same as the one in the load balancer core (This apply in our application). Not issue there. If there is a corruption of the mbuf in the workers code the segmentation fault has to be before tx queue ring enqueue. (I test several field of the mbuf before enqueuing it, ->port field , ->data_len ... etc) In the second test of the load balancer core i could not see a relationship between the packets drop in the rx core and the mbuf corruption in the ixgbe_tx_entry. Waiting for some advices... Regards Ariel Horacio Rodriguez. On Tue, Nov 10, 2015 at 8:50 AM, Ariel Rodriguez wrote: > Thank you very much for your rapid response. > > 1) IO part is the same as load balancer. The worker part is different. The > tx part use qos scheduler framework also. I will try to run the example and > see what happends. > > 2) yes i can. I will do that too. > > The nic is 82599ES 10-Gigabit SFI/SFP+ with tapped traffic (is a hardware > bypass device silicom vendor). > > I develop a similar app without the tx part. It just received a copy of > the traffic (around 6gbps and 40 concurrent flows) and then free the > mbufs. It works like a charm. > > Is strange this issue ... If i disabled the qos scheduler code and the tx > code dropping all packets instead of rte_eth_tx_burst ( is like disabling > tx core) the issue is happening in rte_eth_rx_burst returning corrupted > mbuf (rx core) > > Could the nic behave anormally? > > I will try the 2 things you comment before. > > Regards . > > Ariel Horacio Rodriguez > On Tue, Nov 10, 2015 at 01:35:21AM -0300, Ariel Rodriguez wrote: > > Dear dpdk experts. > > > > Im having a recurrent segmentation fault under the > > function ixgbe_tx_free_bufs (ixgbe_rxtx.c:150) (i enable -g3 -O0). > > > > Surfing the core dump i find out this: > > > > txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]); > > > > txq->tx_next_dd = 31 > > txq->txq->tx_rs_thresh=32 > > > > Obviosly txep points out to the first element but > > > > *(txep).mbuf == INVALID MBUF ADDRESS > > > > The same applies to > > > > *(txep+1).mbuf ; *(txep +2).mbuf;*(txep+3).mbuf > > > > from *(txep+4) .mbuf to *(txep+31).mbuf seems to be valid because im able > > to derefence the mbuf
[dpdk-dev] Issue on rte_sched.c
Hello, viewing the new code of librte_sched/ i found this line strange ... #if defined(__SSE4__) if instead i use : #if defined(__SSE4_2__) || defined(__SSE4_1__) works like a charm ... i never see in any directive like __SSE4__ Regards
[dpdk-dev] Issue on rte_sched.c
Yes i am refereing on that commit. About RTE_MACHINE_CPUFLAG_* , i agreed. I never prepare a patch ... maybe is time to put my hands on that. 2016-04-13 20:35, Ariel Rodriguez: > Hello, viewing the new code of librte_sched/ i found this line strange ... > > #if defined(__SSE4__) Are you refering to http://dpdk.org/commit/90f455f ? > if instead i use : > > #if defined(__SSE4_2__) || defined(__SSE4_1__) > > works like a charm ... > > i never see in any directive like __SSE4__ Indeed, it is strange. By the way, it is recommended to use RTE_MACHINE_CPUFLAG_*.
