Re: [vpp-dev] fails to startup vpp on master branch.

[Zhiyong Yang]

Hi Damjan,

This issue is caused by this patch.

54e8e39a5284917efab8fd6d7d92e589d4a911a4 is the first bad commit
commit 54e8e39a5284917efab8fd6d7d92e589d4a911a4
Author: Damjan Marion mailto:damar...@cisco.com>>
Date:   Wed Nov 7 17:55:26 2018 +0100

pmalloc: fix shared mappings

Change-Id: I6782544d5ee0a66b1a027874b23574416093ca92
Signed-off-by: Damjan Marion damar...@cisco.com


diff --git a/src/vppinfra/pmalloc.c b/src/vppinfra/pmalloc.c
index a881a65..46ccd7f 100644
--- a/src/vppinfra/pmalloc.c
+++ b/src/vppinfra/pmalloc.c
@@ -290,7 +290,7 @@ pmalloc_map_pages (clib_pmalloc_main_t * pm, 
clib_pmalloc_arena_t * a,
   return 0;
 }

-  mmap_flags = MAP_FIXED | MAP_ANONYMOUS;
+  mmap_flags = MAP_FIXED;

   if ((pm->flags & CLIB_PMALLOC_F_NO_PAGEMAP) == 0)
 mmap_flags |= MAP_LOCKED;
@@ -307,10 +307,12 @@ pmalloc_map_pages (clib_pmalloc_main_t * pm, 
clib_pmalloc_arena_t * a,
pm->error = clib_mem_create_fd ((char *) a->name, >fd);
   if (a->fd == -1)
goto error;
+  if ((ftruncate (a->fd, size)) == -1)
+   goto error;
 }
   else
 {
-  mmap_flags |= MAP_PRIVATE;
+  mmap_flags |= MAP_PRIVATE | MAP_ANONYMOUS;
   a->fd = -1;
 }

I try to restore the first line changed. It works well on CentOS.


Thanks
Zhiyong

From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Zhiyong Yang
Sent: Monday, November 19, 2018 6:11 PM
To: Damjan Marion 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] fails to startup vpp on master branch.

BOOT_IMAGE=/vmlinuz-3.10.0-862.14.4.el7.x86_64 root=/dev/mapper/centos-root ro 
crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet 
default_hugepagesz=1G hugepagesz=1G hugepages=8 intel_iommu=on iommu=pt 
isolcpus=1-20 LANG=en_US.UTF-8 nohz_full=1-20 rcu_nocbs=1-20

From: Damjan Marion [mailto:dmar...@me.com]
Sent: Monday, November 19, 2018 5:59 PM
To: Yang, Zhiyong mailto:zhiyong.y...@intel.com>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] fails to startup vpp on master branch.

kernel parameters?

--
Damjan

On 19 Nov 2018, at 10:57, Yang, Zhiyong 
mailto:zhiyong.y...@intel.com>> wrote:

Damjan,

I’m working on CentOS 7.5 and on the latest master for VPP.
Uname –r
3.10.0-862.14.4.el7.x86_64

[root@localhost vpp]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)

Thanks
Zhiyong
From: vpp-dev@lists.fd.io 
[mailto:vpp-dev@lists.fd.io] On Behalf Of Damjan Marion via Lists.Fd.Io
Sent: Monday, November 19, 2018 5:42 PM
To: Yang, Zhiyong mailto:zhiyong.y...@intel.com>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] fails to startup vpp on master branch.



Your kernel is refusing to do mmap().
We need a bit more info, kernel version, kernel parameters,

Are you on the latest master?

--
Damjan


On 19 Nov 2018, at 10:25, Zhiyong Yang 
mailto:zhiyong.y...@intel.com>> wrote:

Hi VPP dev guys,

Does Anybody see the failures about vpp on master branch as 
following?  The failures must be reproduced on my 3 servers/boxes.  How can I 
fix it ?

vlib_physmem_shared_map_create: pmalloc_map_pages: failed to mmap 1 pages at 
0x7fd94000 fd 25 numa 0 flags 0x42011: Invalid argument

dpdk_buffer_pool_create: failed to allocate mempool on socket 0

Thanks
Zhiyong


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11303): https://lists.fd.io/g/vpp-dev/message/11303
Mute This Topic: https://lists.fd.io/mt/28239923/675642
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dmar...@me.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11401): https://lists.fd.io/g/vpp-dev/message/11401
Mute This Topic: https://lists.fd.io/mt/28239923/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Build failing on AArch64

[Sirshak Das]

Hi all,

I am currently facing these build failures in master on AArch64.

[38/1160] Building C object vat/CMakeFiles/vpp_api_test.dir/types.c.o
FAILED: vat/CMakeFiles/vpp_api_test.dir/types.c.o 
ccache /usr/lib/ccache/cc -DHAVE_MEMFD_CREATE -Dvpp_api_test_EXPORTS 
-I/home/sirdas/code/commita/vpp/src -I. -Iinclude -march=armv8-a+crc -g -O2 
-DFORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror   
-Wno-address-of-packed-member -pthread -MD -MT 
vat/CMakeFiles/vpp_api_test.dir/types.c.o -MF 
vat/CMakeFiles/vpp_api_test.dir/types.c.o.d -o 
vat/CMakeFiles/vpp_api_test.dir/types.c.o   -c 
/home/sirdas/code/commita/vpp/src/vat/types.c
In file included from 
/home/sirdas/code/commita/vpp/src/vpp/api/vpe_all_api_h.h:25,
 from /home/sirdas/code/commita/vpp/src/vpp/api/types.h:20,
 from /home/sirdas/code/commita/vpp/src/vat/types.c:19:
/home/sirdas/code/commita/vpp/src/vnet/vnet_all_api_h.h:33:10: fatal error: 
vnet/devices/af_packet/af_packet.api.h: No such file or directory
 #include 
  ^~~~
compilation terminated.
[85/1160] Building C object vnet/CMakeFiles/vnet_cortexa72.dir/ethernet/node.c.o
ninja: build stopped: subcommand failed.
Makefile:691: recipe for target 'vpp-build' failed
make[1]: *** [vpp-build] Error 1
make[1]: Leaving directory '/home/sirdas/code/commita/vpp/build-root'
Makefile:366: recipe for target 'build-release' failed
make: *** [build-release] Error 2

[114/1310] Building C object vat/CMakeFiles/vpp_api_test.dir/types.c.o
FAILED: vat/CMakeFiles/vpp_api_test.dir/types.c.o 
ccache /usr/lib/ccache/cc -DHAVE_MEMFD_CREATE -Dvpp_api_test_EXPORTS 
-I/home/sirdas/code/commitb/vpp/src -I. -Iinclude -march=armv8-a+crc -g -O2 
-DFORTIFY_SOURCE=2 -fstack-protector -fPIC -Werror   
-Wno-address-of-packed-member -pthread -MD -MT 
vat/CMakeFiles/vpp_api_test.dir/types.c.o -MF 
vat/CMakeFiles/vpp_api_test.dir/types.c.o.d -o 
vat/CMakeFiles/vpp_api_test.dir/types.c.o   -c 
/home/sirdas/code/commitb/vpp/src/vat/types.c
In file included from 
/home/sirdas/code/commitb/vpp/src/vpp/api/vpe_all_api_h.h:25,
 from /home/sirdas/code/commitb/vpp/src/vpp/api/types.h:20,
 from /home/sirdas/code/commitb/vpp/src/vat/types.c:19:
/home/sirdas/code/commitb/vpp/src/vnet/vnet_all_api_h.h:32:10: fatal error: 
vnet/bonding/bond.api.h: No such file or directory
 #include 
  ^
compilation terminated.
[161/1310] Building C object 
vnet/CMakeFiles/vnet_thunderx2t99.dir/ethernet/node.c.o
ninja: build stopped: subcommand failed.
Makefile:691: recipe for target 'vpp-build' failed
make[1]: *** [vpp-build] Error 1
make[1]: Leaving directory '/home/sirdas/code/commitb/vpp/build-root'
Makefile:366: recipe for target 'build-release' failed
make: *** [build-release] Error 2


Its all someway or the other related to *.api files and genereated
header files.

I am not able to isolate any particular commit that did this.

Does anybody know if anything changed off the top of their head ?

Thank you
Sirshak Das
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11400): https://lists.fd.io/g/vpp-dev/message/11400
Mute This Topic: https://lists.fd.io/mt/28318534/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] PPPOE

[Ni, Hongjun]

Hi Xiangyun,

VPP only supports PPPoE plugin, which can integrate with Control Plane to act 
as PPPoE server.
Please refer to below link for some slides:
https://schd.ws/hosted_files/onsna18/cf/Accelerated%20Open%20Source%20vBRAS%20Solution%20Based%20on%20OpenBRAS%20and%20VPP%26DPDK.PPTX

There is no PPPoE client at present.

Thanks,
Hongjun

From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of xulang
Sent: Monday, November 26, 2018 9:23 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] PPPOE

Hi all,
I would like to use pppoe server and pppoe client, is there any material  about 
this?


Regards,
Xlangyun



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11399): https://lists.fd.io/g/vpp-dev/message/11399
Mute This Topic: https://lists.fd.io/mt/28316755/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] PPPOE

[xulang]

Hi all,
I would like to use pppoe server and pppoe client, is there any material  about 
this?




Regards,
Xlangyun-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11398): https://lists.fd.io/g/vpp-dev/message/11398
Mute This Topic: https://lists.fd.io/mt/28316755/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Request: please add "real" pcap ability #vpp

[Brian Dickson]

Hi, Jerome (and everyone),

Thanks for this!

Using packet-capture + span, did indeed accomplish what I was looking for.

One useful data point: I was able to capture about 10 seconds of line-rate
10G into a pcap file, and it looks like I didn't lose any packets, on a VPP
host that was not forwarding packets.

Thanks again,
Brian

On Fri, Nov 23, 2018 at 9:06 AM Jerome Tollet (jtollet) 
wrote:

> Hi Brian,
>
> I tried what I told you and I confirm that worked fine on my setup.
>
>
>
> *create packet-generator interface pg0*
>
> *packet-generator capture pg0 pcap /tmp/mycap.pcap*
>
> *set interface span SOURCE_INTF destination pg0*
>
> *set interface state pg0 up*
>
>
>
> Jerome
>
> *De : * au nom de Brian Dickson <
> brian.peter.dick...@gmail.com>
> *Date : *vendredi 23 novembre 2018 à 08:03
> *À : *"d...@barachs.net" 
> *Cc : *"vpp-dev@lists.fd.io" 
> *Objet : *Re: [vpp-dev] Request: please add "real" pcap ability #vpp
>
>
>
>
>
> On Thu, Nov 22, 2018 at 5:30 AM  wrote:
>
> Laying aside comments about folks who aren’t regular community
> contributors introducing themselves in random ways, here are a few thoughts:
>
>
>
> We have a plan to unify pcap tracepoints when Damjan finishes reworking
> the ethernet input node.
>
>
>
> That is very welcome news.
>
>
>
> Is there a rough timeline for Damjan's reworking, and the unification? I
> just want to factor that into my own plans, if possible.
>
>
>
>
>
> No matter what, pcap capture involves a bunch of data copying. The
> forwarding rate will clearly suffer. Full stop.
>
>
>
> Yes, I fully understand that. There's no such thing as a free lunch.
>
>
>
> In the environment in question, there's VPP hosts (doing BGP with the
> netlink and router sandbox plugins to get the routing table into VPP), and
> adjacent to them (physically upstream/downstream) we are using passive
> optical splitters.
>
>
>
> Those optical splitters feed copies of traffic to capture hosts,
> specifically dedicated to packet capture and/or other integrated analysis
> code to be developed.
>
>
>
> Our packet capture would only be using VPP without any packet forwarding,
> i.e. as a convenient way of integrating kernel offload with packet capture,
> and possibly chained with other added custom nodes.
>
>
>
> (DPDK by itself is not really friendly for doing any kind of from-scratch
> integration, and I haven't found many/any other currently maintained open
> source packages/frameworks that offer pcap. E.g. netmap-libpcap seems
> abandoned.)
>
>
>
> Having the ability to add other nodes in the graph, that do other stuff,
> possibly with zero copy, is another major reason we're looking at VPP.
>
>
>
> So, pcap is the starting point, and future work might keep the pcap
> capability (assuming the ability to control whether capture is done, and
> the ability to specify pcap filter rules), and add other custom
> functionality.
>
>
>
> To give you an idea, this is not consumer-grade stuff we are using; 12 or
> 24 core Intel boxes (with HT, appears as 24 or 48 cores), and 128GB or
> 256GB of memory, just for packet capture, onto RAIDed SSDs.
>
>
>
> Thanks for the info, and I'll definitely look at that extras/wireshark
> thing.
>
>
>
> Brian
>
>
>
>
>
> In master/latest, I’ve added pcap tracing – and a wireshark dissector – to
> the graph dispatch engine. See .../extras/wireshark/readme.md for more
> detail. The wireshark dissector isn’t finished by any means, nor do we have
> a blessed encap type number from tcpdump-workers, nor is the work
> upstreamed into wireshark.
>
>
>
> [image: cid:image001.png@01D4823D.97D176D0]
>
>
>
>
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *
> brian.peter.dick...@gmail.com
> *Sent:* Wednesday, November 21, 2018 6:59 PM
> *To:* vpp-dev@lists.fd.io
> *Subject:* [vpp-dev] Request: please add "real" pcap ability #vpp
>
>
>
> Hi, dev folks,
>
> Apologies for my first message being kind of demanding.
>
> However, I think this is a reasonable request.
>
> What I am interested in, and I think this is likely to be a fairly
> universal desire, is the ability to properly integrate some kind of pcap
> packet capture to the full VPP graph.
>
> The current available mechanisms (pcap drop trace and pcap tx trace) do
> not apply to packets that are only "handled" by the host in question, i.e.
> neither originate or terminate on the local host.
>
> In particular, I'm interested in something that can run on a bare metal
> host and, presuming sufficient resources can be given to it (cores, memory,
> etc), do packet capture at line rate.
>
> Thus, any restriction ("run it on a VM") is not adequate.
>
> Given that there is already stuff for handling the pcap file already (in
> vnet/unix IIRC), this should not be a lot of work.
>
> There are two use cases I have:
>
> · debugging data plane stuff on a vpp-based router (i.e. using the vppsb
> netlink and router projects)
>
> · packet capture at line rate (a vpp host that only listens/drops
> traffic, incidental to the packet 

答复: 答复: 答复: [vpp-dev] about sctp

[刘道林]

Hi Marco，

1. If you indeed use crc32c, it's ok. But from my view, it should not include 
ip header.

2. the definition is below:

#define SHA1_OUTPUT_LENGTH 20
/*
 * 0   1   2   3
 * 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 * |  Parameter Type   |   Parameter Length|
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 * \   \
 * /   Parameter Value /
 * \   \
 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 */
typedef struct
{
  sctp_opt_params_hdr_t param_hdr;

  unsigned char mac[SHA1_OUTPUT_LENGTH];/* RFC 2104 */
  u64 creation_time;
  u32 cookie_lifespan;

} sctp_state_cookie_param_t;

The macro definition SHA1_OUTPUT_LENGTH 20 seems not enough.




Best regards
刘道林 (Daolin Liu)
T大连市共进科技有限公司
DALIAN GONGJIN TECHNOLOGY CO.,LTD
中国大连市高新园区软件园路1A-4-24层
Floor 24th, 1A-4 Software Park Road, Hi-tech Zone, Dalian, Liaoning, China
直线(TEL)：(86-411)39996705   分机(EXT)：76824
手机(Mobile)：(86)13704090959


-邮件原件-
发件人: Marco Varlese [mailto:mvarl...@suse.de]
发送时间: 2018年11月23日 17:07
收件人: Liu Daolin (刘道林); vpp-dev@lists.fd.io
主题: Re: 答复: 答复: [vpp-dev] about sctp


On Fri, 2018-11-23 at 00:57 +, Liu Daolin (刘道林) wrote:
> Hi Macro,
>
> OK, I will try it when I have time.
Well, as you wish.

>
> I'm reading the sctp source code and found some issues. The most critical one
> is the checksum. The sctp checksum should not include the ip header, just only
> sctp data, and also it should use crc32c, not the current code shown.
I am not sure what you refer to with "code shown".
However the crc calculated is indeed the one you mention and it's using SSE4
instructions which are exactly using the polynomial depicted by RFC 4960.

> Another important issue is the max cookie length is 20 from the macro
> definition. Actually it should not be a fixed value (usually more than 200).
So I cannot find a definition for the actual length of the cookie.
It would help if you could point to the actual filename / definition when
reporting issues.

>
>
> Best regards
> 刘道林 (Daolin Liu)
> T大连市共进科技有限公司
> DALIAN GONGJIN TECHNOLOGY CO.,LTD
> 中国大连市高新园区软件园路1A-4-24层
> Floor 24th, 1A-4 Software Park Road, Hi-tech Zone, Dalian, Liaoning, China
> 直线(TEL)：(86-411)39996705   分机(EXT)：76824
> 手机(Mobile)：(86)13704090959
>
> 发件人: Marco Varlese [mailto:mvarl...@suse.de]
> 发送时间: 2018年11月22日 20:52
> 收件人: Liu Daolin (刘道林); vpp-dev@lists.fd.io
> 主题: Re: 答复: [vpp-dev] about sctp
>
> Dear Liu,
>
> The reason why I asked you to take a look at the "test_sctp.py" is to see how
> things are setup and execute the exact steps in the CLI. You'll see things
> work.
>
> You cannot remove the MTU code because that's used to initialize the PMTU and
> calculate initially the MSS to be sent to the other peer.
>
>
> Cheers,
> Marco
>
> On Wed, 2018-11-21 at 01:50 +, Liu Daolin (刘道林) wrote:
> > Hi Marco,
> >
> > Thanks for your reply.
> >
> > Actually I'm trying to read the sctp code recently to want to know how it
> > works. My requirement is very simply. I need to run sctp server based on vpp
> > on one VM, and run sctp client without vpp (I have this now) on the other
> > VM. As my below email, there will be aborted (core dumped), and I found the
> > place yesterday, but I don’t know the root reason. You can see below
> > picture:
> >
> >
> >
> > The sw_if_index is -1, so crashed when get mtu. Maybe I miss something? When
> > I run actually is CLI command from echo_server.c. As you mentioned, if I run
> > test_sctp.py, it will work fine? My requirement is only C language as it
> > will merged some other code, so I don’t like to run it with python.
> >
> > Yesterday, I also try to remove this mtu code, and the crash was disappeared
> > although the sw_if_index is still -1, and then I start sctp client on the
> > other VM to send INIT, but the server with vpp seemed be dead (no crash but
> > cannot enter any command, seemed dead loop).
> >
> > Anyway, I think it's better to try CLI "test echo server" in your side and
> > find and fix all issues visually.
> >
> >
> >
> >
> >
> > Best regards
> > 刘道林 (Daolin Liu)
> > T大连市共进科技有限公司
> > DALIAN GONGJIN TECHNOLOGY CO.,LTD
> > 中国大连市高新园区软件园路1A-4-24层
> > Floor 24th, 1A-4 Software Park Road, Hi-tech Zone, Dalian, Liaoning, China
> > 直线(TEL)：(86-411)39996705   分机(EXT)：76824
> > 手机(Mobile)：(86)13704090959
> >
> > -邮件原件-
> > 发件人: Marco Varlese [mailto:mvarl...@suse.de]
> > 发送时间: 2018年11月21日 1:06
> > 收件人: Liu Daolin (刘道林); vpp-dev@lists.fd.io
> > 主题: Re: [vpp-dev] about sctp
> >
> > Hi,
> >
> > On Tue, 2018-11-20 at 02:50 +, Liu Daolin (刘道林) wrote:
> > > Hi,
> > >
> > > I encountered below Aborted (core dumped) issue:
> > >
> > >
> > >
> > 

Re: [vpp-dev] dpdk socket-mem and dpdk_pool_create

[Damjan Marion via Lists.Fd.Io]

> On 25 Nov 2018, at 05:23, Kingwel Xie  wrote:
> 
> Hi Damjan and vPPers,
> 
> I got a question about the physical mem management in vPP.
> 
> As I understand, we specify socket-mem in startup.conf for DPDK to allocate 
> mem for itself, but we are using vPP phymem allocator for mbuf and crypto 
> PMDs if vdev is specified. The latter consist of crypto op , session, and 
> driver pools, and they are quite small in general. However, at least one 
> hugepage will be allocated for each pool, which could be 1GB. Big waste in a 
> way.
> DPDK instead handles it in a better way using a mem segment based mechanism.
> 
> Question: why use a customzied dpdk_pool_create instea of asking DPDK to 
> manage mempool just like rte_pktbuf_create_by_ops?
> 


We care only for buffer mempools to be allocated by vpp physmem allocator, no 
need for others. They can be allocated by dpdk or
alternatively other mempools can share single physmem region with dpdk buffer 
mempools 


-- 
Damjan-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11395): https://lists.fd.io/g/vpp-dev/message/11395
Mute This Topic: https://lists.fd.io/mt/28308134/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-