Re: [vpp-dev] api msg deadlock

2022-06-06 Thread NUAA无痕
Hi ,florin

if i add many message once in a while, svm_queue_add will deadlock due to
q->cursize == q->maxsize?
Maybe this reason cause deadlock

int
svm_queue_add (svm_queue_t * q, u8 * elem, int nowait)
{
  i8 *tailp;
  int need_broadcast = 0;

  if (nowait)
{
  /* zero on success */
  if (svm_queue_trylock (q))
{
 return (-1);
}
}
  else
svm_queue_lock (q);

  if (PREDICT_FALSE (q->cursize == q->maxsize))
{
  if (nowait)
{
 svm_queue_unlock (q);
 return (-2);
}
  while (q->cursize == q->maxsize)
svm_queue_wait_inline (q);
}

  tailp = (i8 *) (&q->data[0] + q->elsize * q->tail);
  clib_memcpy_fast (tailp, elem, q->elsize);

  q->tail++;
  q->cursize++;

  need_broadcast = (q->cursize == 1);

  if (q->tail == q->maxsize)
q->tail = 0;

  if (need_broadcast)
svm_queue_send_signal_inline (q, 1);

  svm_queue_unlock (q);

  return 0;
}

NUAA无痕 via lists.fd.io  于2022年6月6日周一
14:52写道:

> Hi, florin
>
> i have study jvpp code and found that maybe jvpp is ok, error occurs in
> vpp svm_queue
>
> jvpp connect vpp by vl_client_connect_to_vlib function
> (vpp/src/vlibmemory/memory_client.c)
>
> int
> vl_client_connect_to_vlib (const char *svm_name,
>   const char *client_name, int rx_queue_size)
> {
>   return connect_to_vlib_internal (svm_name, client_name, rx_queue_size,
>   rx_thread_fn, 0 /* thread fn arg */ ,
>   1 /* do map */ );
> }
>
> jvpp example is
>  jvpp-registry/jvpp_registry.c:if
> (vl_client_connect_to_vlib(shm_prefix, name, 32) < 0)
>
> i change 32 to 4096, now vpp has run three days and not deadlock
>
> so i think that whether rx_queue_size parameter is too small, resulting in
> function error handling
>
> I also compare vpp-2101 with vpp-2206 and found that binary api code(
> svm_queue in queue.c ) is no change
>
> If i send a large message that size over rx_queue_size will cause problem
> ?
> Looking forward to your opinion
>
> Best regards
> wanghe
>
> Florin Coras  于2022年6月3日周五 23:28写道:
>
>> Hi Wanghe,
>>
>> The only api bindings supported today are c, python and golang. Maybe
>> somebody familiar with the jvpp code can help you out but otherwise I’d
>> recommend to switch if possible.
>>
>> Regards,
>> Florin
>>
>> > On Jun 3, 2022, at 7:55 AM, NUAA无痕  wrote:
>> >
>> > Hi, florin
>> >
>> > About this question, i compare c++ code with jvpp code, then i found
>> that jvpp maybe have a bug and even if update vpp also cannot resolve it
>> >
>> > jvpp code according to vpp version 1901, that has jvpp example
>> >
>> vpp-1901/extras/japi/java/jvpp-core/io/fd/vpp/jvpp/core/examples/CreateSubInterfaceExample.java
>> has jvpp example and our code according to it
>> >
>> > now vpp version is 2101
>> > then when java code connected to vpp and then use "close“ function it
>> will hint "peer unresponsive, give up"
>> > this error from src/vlibmemory/memory_client.c vl_client_disconnect
>> function
>> >
>> > why this error is that svm_queue_sub always return -2 until timeout
>> >
>> > this is code , the reason is that "vl_input_queue->cursize == 0 " and
>> vl_input_queue->head == vl_input_queue->tail
>> >
>> > int
>> > vl_client_disconnect (void)
>> > {
>> >   vl_api_memclnt_delete_reply_t *rp;
>> >   svm_queue_t *vl_input_queue;
>> >   api_main_t *am = vlibapi_get_main ();
>> >   time_t begin;
>> >
>> >   vl_input_queue = am->vl_input_queue;
>> >   vl_client_send_disconnect (0 /* wait for reply */ );
>> >
>> >   /*
>> >* Have to be careful here, in case the client is disconnecting
>> >* because e.g. the vlib process died, or is unresponsive.
>> >*/
>> >   begin = time (0);
>> >   while (1)
>> > {
>> >   time_t now;
>> >
>> >   now = time (0);
>> >
>> >   if (now >= (begin + 2))
>> > {
>> >  clib_warning ("peer unresponsive, give up");
>> >  am->my_client_index = ~0;
>> >  am->my_registration = 0;
>> >  am->shmem_hdr = 0;
>> >  return -1;
>> > }
>> >
>> > /* this error because vl_input_queue->cursize == 0  */
>> >   if (svm_queue_sub (vl_input_queue, (u8 *) & rp, SVM_Q_NOWAIT, 0)
>> < 0)
>> > continue;
>> >
>> >   VL_MSG_API_UNPOISON (rp);
>> >
>> >   /* drain the queue */
>> >   if (ntohs (rp->_vl_msg_id) != VL_API_MEMCLNT_DELETE_REPLY)
>> > {
>> >  clib_warning ("queue drain: %d", ntohs (rp->_vl_msg_id));
>> >  vl_msg_api_handler ((void *) rp);
>> >  continue;
>> > }
>> >   vl_msg_api_handler ((void *) rp);
>> >   break;
>> > }
>> >
>> >   vl_api_name_and_crc_free ();
>> >   return 0;
>> > }
>> >
>> > when i use c++ for vpp binary api,  vl_input_queue->cursize == 1 and
>> vl_input_queue->head != vl_input_queue->tail
>> >
>> > so c++ use binary api is correct that about svm_queue_* series functions
>> >
>> > Although JVpp is no longer supported, but this is important for me!
>> >
>> > Can you give a patch for jvpp? Thanks
>> >
>> > Best regards
>> >
>> > Wanghe
>> >
>> >
>>
>>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21508): https://lists.fd.io

Re: [vpp-dev] #ipsec #dpdk VPP 22.02 no DPDK crypto devs available on AWS VM

2022-06-06 Thread gv . florian
Hi Matthew,

Thank you very much for your detailed response!

I saw the aesni drivers exclusion from the dpdk.mk file, but I feel it that was 
done in purpose and in the same time I was confused by the "dpdk/cryptodev     
[warn  ]: dpdk_cryptodev_init: Not enough cryptodev resources" warning.

I switched between different  crypto engines by disabling  the rest of the 
plugins from the startup.conf file to have a single one available on a time.

Sure, I am using multiples threads (workers) and multiples queues. One of the 
best achieved performance was to go with 8 workers and 4 queues (I have 2 
interfaces) so the worker distribution is looking like this:

vpp# show interface rx-placement

Thread 1 (vpp_wk_0):

node dpdk-input:

lan queue 0 (polling)

Thread 2 (vpp_wk_1):

lan queue 1 (polling)

Thread 3 (vpp_wk_2):

lan queue 2 (polling)

Thread 4 (vpp_wk_3):

lan queue 3 (polling)

Thread 5 (vpp_wk_4):

wan queue 0 (polling)

Thread 6 (vpp_wk_5):

wan queue 1 (polling)

Thread 7 (vpp_wk_6):

wan queue 2 (polling)

Thread 8 (vpp_wk_7):

wan queue 3 (polling)

I was using a single IPSec tunnel, after reading this:
https://lfnetworking.org/how-fd-io-20-01-release-improves-multicore-ipsec/
which one, btw, reading now one more time, after the ipsec experience and your 
explanation, it makes more sense to me.

By creating multiples IPSec tunnels and using async mode I was able  to 
increase the throughput for large packet sizes, going close to 20Gbps for 8900 
bytes per packet.

I am using routed IPSec and, for a single IPSec tunnle

I am avoiding fragmentation by setting the right MTU on the sender VM 
interface. I will do a test for fragmentation case in the future, thank you for 
the tip.

Best regards,

Gabi

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21509): https://lists.fd.io/g/vpp-dev/message/21509
Mute This Topic: https://lists.fd.io/mt/91520137/21656
Mute #dpdk:https://lists.fd.io/g/vpp-dev/mutehashtag/dpdk
Mute #ipsec:https://lists.fd.io/g/vpp-dev/mutehashtag/ipsec
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-