from:"George Zhao"

Re: [vpp-dev] [tsc] Project Proposal for Sweetcomb

2018-11-12 Thread George Zhao

Congratulations

From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Edward 
Warnicke
Sent: Saturday, November 10, 2018 12:37 AM
To: Ni, Hongjun 
Cc: sweetcomb-...@lists.fd.io; t...@lists.fd.io; vpp-dev ; 
Wang, Drenfong ; ??? ; 
chen...@huachentel.com; lizhuo...@cmhi.chinamobile.com; ??? 
; zhijl@chinatelecom.cn; changlin...@nxp.com; Wang 
Tianyi ; davidfgao(?? ; 
lixin...@huachentel.com; jingqing@alibaba-inc.com; gu.ji...@zte.com.cn; 
Jerome Tollet (jtollet) ; Maciek Konstantynowicz (mkonstan) 
; Rastislav Szabo -X (raszabo - PANTHEON TECHNOLOGIES at 
Cisco) 
Subject: Re: [vpp-dev] [tsc] Project Proposal for Sweetcomb

Congratulations!

Ed

On Thu, Nov 8, 2018 at 9:32 PM Ni, Hongjun 
mailto:hongjun...@intel.com>> wrote:
Hi all,

Sweetcomb project has been approved by FD.io TSC.

And project creation has been completed.

Thank you all for your great help to make this happen!

Below is the project resource:

Committers group:  fdio-gerrit-sweetcomb-committers Invites sent to the initial 
committers:  
https://wiki.fd.io/view/Project_Proposals/Sweetcomb#Initial_Committers

JIRA:  https://jira.fd.io/projects/SWEET/issues

Jenkins:  https://jenkins.fd.io/view/sweetcomb/

Gerrit:  https://gerrit.fd.io/r/#/admin/projects/sweetcomb

Groups.io:  https://lists.fd.io/g/sweetcomb-dev

GitHub Replication:  https://github.com/FDio/sweetcomb

We also have submitted initial code for IPR Review.
https://gerrit.fd.io/r/#/c/15835/ Initial code checkin for Sweetcomb project

The initial code is contributed by HuachenTel, and we help to refactor it.
If you have any concern, please raise it in project mail list: 
sweetcomb-...@lists.fd.io

Thank you again,
Hongjun

From: t...@lists.fd.io 
[mailto:t...@lists.fd.io] On Behalf Of Ni, Hongjun
Sent: Thursday, November 8, 2018 10:57 PM
To: Ni, Hongjun mailto:hongjun...@intel.com>>; Edward 
Warnicke mailto:hagb...@gmail.com>>
Cc: t...@lists.fd.io; 
vpp-dev@lists.fd.io; Wang, Drenfong 
mailto:drenfong.w...@intel.com>>; ??? 
mailto:wangchuan...@huachentel.com>>; 
chen...@huachentel.com; 
lizhuo...@cmhi.chinamobile.com; ??? 
mailto:lihf...@chinaunicom.cn>>; 
zhijl@chinatelecom.cn; 
changlin...@nxp.com; Wang Tianyi 
mailto:tianyi.w...@tieto.com>>; davidfgao(?? 
mailto:davidf...@tencent.com>>; 
lixin...@huachentel.com; 
jingqing@alibaba-inc.com; 
gu.ji...@zte.com.cn; Jerome Tollet (jtollet) 
mailto:jtol...@cisco.com>>
Subject: Re: [vpp-dev] [tsc] Project Proposal for Sweetcomb

Hi TSC members,

Our Initial code has been refactored and  gone through internal legal review.

How can we submit the code to the fd.io Foundation for IPR Review?

Thanks,
Hongjun

From: vpp-dev@lists.fd.io 
[mailto:vpp-dev@lists.fd.io] On Behalf Of Ni, Hongjun
Sent: Thursday, November 8, 2018 12:04 PM
To: Ni, Hongjun mailto:hongjun...@intel.com>>; Edward 
Warnicke mailto:hagb...@gmail.com>>
Cc: t...@lists.fd.io; 
vpp-dev@lists.fd.io; Wang, Drenfong 
mailto:drenfong.w...@intel.com>>; ??? 
mailto:wangchuan...@huachentel.com>>; 
chen...@huachentel.com; 
lizhuo...@cmhi.chinamobile.com; ??? 
mailto:lihf...@chinaunicom.cn>>; 
zhijl@chinatelecom.cn; 
changlin...@nxp.com; Wang Tianyi 
mailto:tianyi.w...@tieto.com>>; davidfgao(?? 
mailto:davidf...@tencent.com>>; 
lixin...@huachentel.com; 
jingqing@alibaba-inc.com; 
gu.ji...@zte.com.cn; Jerome Tollet (jtollet) 
mailto:jtol...@cisco.com>>
Subject: Re: [vpp-dev] [tsc] Project Proposal for Sweetcomb

Welcome Cisco to join Sweetcomb project!

https://wiki.fd.io/view/Project_Proposals/Sweetcomb

Thanks a lot,
Hongjun

From: t...@lists.fd.io [mailto:t...@lists.fd.io] On 
Behalf Of Ni, Hongjun
Sent: Thursday, November 1, 2018 8:40 AM
To: Ni, Hongjun mailto:hongjun...@intel.com>>; Edward 
Warnicke mailto:hagb...@gmail.com>>
Cc: t...@lists.fd.io; 
vpp-dev@lists.fd.io; Wang, Drenfong 
mailto:drenfong.w...@intel.com>>; ??? 
mailto:wangchuan...@huachentel.com>>; 
chen...@huachentel.com; 
lizhuo...@cmhi.chinamobile.com; ??? 
mailto:lihf...@chinaunicom.cn>>; 
zhijl@chinatelecom.cn; 
changlin...@nxp.com; Wang Tianyi 
mailto:tianyi.w...@tieto.com>>;

Re: [vpp-dev] [csit-dev] VPP Release 18.07 is out!

2018-07-31 Thread George Zhao

Bravo!!!

From: csit-...@lists.fd.io [mailto:csit-...@lists.fd.io] On Behalf Of Edward 
Warnicke
Sent: Monday, July 30, 2018 7:57 PM
To: vpp-dev ; csit-...@lists.fd.io
Subject: [csit-dev] VPP Release 18.07 is out!

VPP Release 18.07 is out.  Packages are in the usual places.

Ed
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9994): https://lists.fd.io/g/vpp-dev/message/9994
Mute This Topic: https://lists.fd.io/mt/23924679/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Sad News from Intel - Venky Venkatesan

2018-04-05 Thread George Zhao

I am so sad to hear this news about Venky, but his legacy will live on through 
all the great work he did.

BR,
George

From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of St Leger, 
Jim
Sent: Thursday, April 5, 2018 11:37 AM
To: vpp-dev@lists.fd.io
Cc: Gadiyar, Rajesh ; Mukherjee, Udayan 

Subject: [vpp-dev] Sad News from Intel - Venky Venkatesan

Several of you have already heard the very sad news of the passing of our dear 
friend and colleague Venky.  He was "Mr. DPDK" to many, a nod to his invention 
and creation of DPDK about ten years ago (before it was called DPDK.)  Of 
course his footprints across the industry and literally across the globe are 
everywhere. If you know what ATCA is, well, Venky had his hands and inventions 
deeply into the creation of that PICMG spec and many of the first revision 
products. He touched on and gave guidance to projects spanning Linux/netdev, 
DPDK, Intel QuickAssist Technology, BSD stacks, OVS, OPNFV, FD.io, 
all-things-virtualization (KVM, Xen, VMware, Hyper-V, etc.), ETSI, IETF, 
containers, and so many other arenas that my brain can't remember at this dark 
moment.

If you knew him and engaged him, especially on deep technical discussions, you 
knew you were talking to someone with the rare combination of skills that went 
incredibly deep both on the hardware, silicon side of platforms but also on the 
software, networking side of the platforms.  Rarely did I find a topic that 
Venky wasn't well versed on. And by "topic" I mean everything from football to 
photography to history to politics and many other subjects well beyond the 
technical realm our day jobs find us in. Conversations with Venky were a treat. 
Traveling with him was an adventure. Debating with him was a lesson. I will so 
miss all of him.

Below are some details on a memorial service this Sunday in Oregon and also a 
GoFundMe page. If you're in Oregon or close by please join the service. Venky 
left behind a wife, Priya, and two young girls, Adhiti and Namrata. While I'm 
sure they are overwhelmed with sadness and grief at the moment they can use our 
support. Please consider supporting the GoFundMe memorial fund for his family.

Lastly, my network extends through this distribution list.  But Venky's network 
was one hundred times broader. Please share this sad news through your own 
networks.

Thanks,
Jim

From: A message from Sandra Rivera
Sent: Thursday, April 5, 2018 10:58 AM
Subject: Follow-up: A loss for our NPG organization

All,
We have an update on the memorial service plans in celebration of Venky 
Venkatesan's life.

The service will be held on Sunday, April 8th at our Jones Farm Conference 
Center (JFCC) in Hillsboro, OR from 2-4PM, and is open to all who would like to 
join.

Venky's family will plan to attend, as well as friends and others who have 
worked closely with him over the years. Please note that this is not an 
Intel-sponsored event and therefore any travel or expenses related to attending 
need to be managed personally.

Many of you have asked about donations, and we have helped setup a GoFundMe 
account for his family which will be used to fund scholarships for his children.
https://www.gofundme.com/venky-venkatesan-memorial-fund

If you have any questions about the service, please contact Udayan Mukherjee.

Thank you all very much for your support during this difficult time.
Sandra

From: A message from Sandra Rivera
Sent: Tuesday, April 3, 2018 9:34 PM
Subject: A loss for our NPG organization

To All,
Today is a very sad day for our NPG team and Intel.  Our dear friend and 
colleague, Venky Venkatesan, passed away early morning today after a tough 
battle with cancer.

Aside from his deep contributions to the organization, Venky provided technical 
mentorship to many employees and greatly impacted our technical leadership 
pipeline.

On a personal level, Venky was very dedicated to his family and also cared 
deeply about mentoring the next generation of engineers and scientists through 
efforts like coaching robotics teams and providing guidance to many interns and 
Intel employees. His passion, dedication and commitment to this organization, 
Intel, and the external community will be greatly missed.

We will let you know as soon as more information is available for Venky's 
memorial/celebration service.
Sandra

Re: [vpp-dev] [csit-dev] fd.io Community Goals and Objectives for reporting to the LFN board

2018-03-07 Thread George Zhao

For DMM project:

- Commit initial DMM framework
- Commit documentations for APIs, developer guides, etc.
- Plug into CSIT
- Integrate VPP host stack/TLDK
- DMM data-plane EAL on VPP L3
- Performance optimization

Thanks
George



From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Edward 
Warnicke
Sent: Wednesday, March 7, 2018 9:31 PM
To: vpp-dev@lists.fd.io
Cc: vpp-dev@lists.fd.io; Trishan de Lanerolle 
<tdelanero...@linuxfoundation.org>; Dave Barach (dbarach) <dbar...@cisco.com>; 
Maciek Konstantynowicz (mkonstan) <mkons...@cisco.com>; lmuscarie...@cisco.com; 
Ananyev, Konstantin <konstantin.anan...@intel.com>; George Zhao 
<george.y.z...@huawei.com>; francois.o...@linaro.org; Lu, Patrick 
<patrick...@intel.com>
Subject: Re: [vpp-dev] [csit-dev] fd.io Community Goals and Objectives for 
reporting to the LFN board

Ray,

Thanks for calling attention to this.  I've taken the feedback I've received 
thus far and added it to:

https://docs.google.com/presentation/d/1jUoqWt9tbMaiUsLE2IaUSSbTcjKN6c3cQ-PXID5nz0k/edit?usp=sharing

It's a rough cut, let's work it together as a community overnight and at the 
TSC meeting tomorrow.  Feel free to edit to improve.

Ed

On Thu, Mar 1, 2018 at 8:51 AM Ray Kinsella 
<ray.kinse...@intel.com<mailto:ray.kinse...@intel.com>> wrote:
Hi folks,

So the LFN TAC is looking to close on the "Community goals and priorities" 
rollup by 9th March.
This exercise is part of the process of obtaining a fair share of resources for 
FD.io in 2018.

So if you haven't already, please canvass your committers and contributors for 
their 2018 project plans.
What new features do they anticipate developing? Are there plans for 
validation, new platform or hardware support etc ?
You can either reply to this email directly, or send a note to myself or Ed W.

Much appreciated, thanks,

Ray K

From: csit-dev-boun...@lists.fd.io<mailto:csit-dev-boun...@lists.fd.io> 
[mailto:csit-dev-boun...@lists.fd.io<mailto:csit-dev-boun...@lists.fd.io>] On 
Behalf Of Ed Warnicke
Sent: Thursday 1 February 2018 16:43
To: t...@lists.fd.io<mailto:t...@lists.fd.io>; 
disc...@lists.fd.io<mailto:disc...@lists.fd.io>; vpp-dev 
<vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>>; 
csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; 
cicn-...@lists.fd.io<mailto:cicn-...@lists.fd.io>; 
tldk-...@lists.fd.io<mailto:tldk-...@lists.fd.io>; 
dmm-...@lists.fd.io<mailto:dmm-...@lists.fd.io>; 
odp4vpp-...@lists.fd.io<mailto:odp4vpp-...@lists.fd.io>; 
pma_tools-...@lists.fd.io<mailto:pma_tools-...@lists.fd.io>
Subject: [csit-dev] fd.io<http://fd.io> Community Goals and Objectives for 
reporting to the LFN board

FD.io has recently transitioned from being its own independent foundation, to 
being one of several projects (ONAP, OpNFV, ODL, etc) under the Linux 
Foundation Networking (LFN) umbrella.

As part of this transition, the LFN board has asked to be educated about what 
Goals and Objectives the community intends to undertake in the next year.   
Since an Open Source community intrinsically is a harmonious collection of 
folks working on different things, the only way to ascertain what its is really 
planning to do is to *ask* what folks are planning to do.

What are the plans of various folks participating in the community for 2018?  
What are the themes of the problems you are planning to address, the kinds of 
features you are looking to implement etc?

Ed

Re: [vpp-dev] [csit-dev] VPP 18.01 Release artifacts are now available on nexus.fd.io

2018-01-25 Thread George Zhao

Congratulations, way to go.


From: csit-dev-boun...@lists.fd.io [mailto:csit-dev-boun...@lists.fd.io] On 
Behalf Of Dave Wallace
Sent: Wednesday, January 24, 2018 9:23 PM
To: vpp-dev@lists.fd.io; csit-...@lists.fd.io
Subject: [csit-dev] VPP 18.01 Release artifacts are now available on nexus.fd.io

Folks,

The VPP 18.01 Release artifacts are now available on nexus.fd.io

The ubuntu.xenial and centos packages can be installed following the recipe on 
the wiki: https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages

Thank you to all of the VPP community who have contributed to the 18.01 VPP 
Release.


Elvis has left the building!
-daw-
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [tsc] [nsh_sfc-dev] Stepping down as NSH_SFC PTL

2018-01-04 Thread George Zhao

Hi Danny,
Thanks for your contributions.

I will let Ed correct me if I am wrong, I don’t think PTL need TSC approval,  
but an email thread from project committers voted +1 for Honghui Ni as new PTL.

BR,
George

From: tsc-boun...@lists.fd.io [mailto:tsc-boun...@lists.fd.io] On Behalf Of 
a...@yunify.org
Sent: Friday, January 05, 2018 9:06 AM
To: Zhou, Danny; t...@lists.fd.io; nsh_sfc-...@lists.fd.io
Cc: vpp-dev@lists.fd.io
Subject: Re: [tsc] [nsh_sfc-dev] Stepping down as NSH_SFC PTL

+1

Thanks for all your hard work!

Ash

Ash Young
M: 408-628-7223


On Wed, Jan 3, 2018 at 8:32 PM -0600, "Zhou, Danny" 
> wrote:
Hi TSC,
I've been taking the NSH_SFC PTL role 1 year ago. It has been a pleasure, but 
also a burden as there are some personal reasons make me cannot focus on 
NSH_SFC anymore, so I am stepping down as NSH_SFC PTL with immediate effect. 
Additionally, I would like to nominate Hongjun Ni, who is the most active 
NSH_SFC maintainer in 2017, as new NSH_SFC PTL for his excellent contribution 
as below:

•  Upstreamed 42 patches totally cover Ingress Classifier, SFF, Egress 
Classifier, NSH-Proxy, NSH-aware NAT, MD-Type2, NSH over Ethernet and NSH TTL.

•  Published four NSH_SFC Releases: 17.01, 17.04, 17.07 and 17.10.

NSH_SFC maintainers,

Pls vote for Hongjun Ni by directly replying this email.

-Danny
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] vpp continuous integration on armv8

2017-11-16 Thread George Zhao

If I understand this correctly,  Tina wants these Arm machines to be used as 
build/HW validation. I am cc-ing our folks in Huawei, who are going to work in 
the community on CSIT to see if they can provide any help, contact person would 
be Appana Prasad and Khemendra Kumar.

Thanks,
George

From: csit-dev-boun...@lists.fd.io [mailto:csit-dev-boun...@lists.fd.io] On 
Behalf Of Luke, Chris
Sent: Thursday, November 16, 2017 4:45 PM
To: Ed Warnicke ; Gabriel Ganne 
Cc: csit-...@lists.fd.io; Dave Barach (dbarach) ; Tina Tsou 
; vpp-dev ; Nicolas Bouthors 

Subject: Re: [csit-dev] [vpp-dev] vpp continuous integration on armv8

Ed,

Based on Tina saying the ones delivered are OD1000’s, Overdrive machines seem 
to come from the factory with Suse on them according to their website.

Chris.

From: csit-dev-boun...@lists.fd.io 
[mailto:csit-dev-boun...@lists.fd.io] On Behalf Of Ed Warnicke
Sent: Thursday, November 16, 2017 18:32
To: Gabriel Ganne >
Cc: csit-...@lists.fd.io; Dave Barach (dbarach) 
>; Tina Tsou 
>; vpp-dev 
>; Nicolas Bouthors 
>
Subject: Re: [csit-dev] [vpp-dev] vpp continuous integration on armv8

Gabriel,

We do need to have multiple workers to service the verify queue... having the 
queue back up behind a single server wouldn't be good.  Do you know if these 
boxes are installed with Ubuntu 16.04 already?

Ed

On Thu, Nov 16, 2017 at 9:03 AM Gabriel Ganne 
> wrote:

Yes, they are.

I estimate it should take about 40 minutes to build and test with 8 cores in 
release mode, therefore I do not think that using all 3 ThunderX is necessary.

I do not have access to a ThunderX platform on my side (I ran my tests on NXP 
and Hierofalcon platforms), if needed please see with Tina Tsou (in CC) for the 
specifics of ThunderX.

I was thinking about duplicating the *vpp-verify-master-ubuntu1604* target as a 
first step.

Ubuntu 16.04 is a LTS for armv8 and is one of the distributions already used 
for vpp x86 continuous testing.

If all goes well, I was thinking about using fedora-26 (or above) as rhel-based 
distribution, since it also has aarch64 official support.

But since it's not one of the distros already used in ci, let's keep it for 
later.

Regards,

--

Gabriel Ganne

From: Ed Warnicke >
Sent: Thursday, November 16, 2017 4:47:14 PM
To: Gabriel Ganne
Cc: vpp-dev; Dave Barach (dbarach); Nicolas Bouthors; 
csit-...@lists.fd.io; Vanessa Valderrama
Subject: Re: [vpp-dev] vpp continuous integration on armv8

Gabriel,

This is awesome news!

Do I understand correctly that these ThunderX boards are intended to be used 
for build/make test/packaging?  If so, I've cced in Vanessa, our fearless 
sysadmin to help in figuring these things out :)

Do you have opinions about which Linux distributions you want to package for 
(Ubuntu?, Centos?) and versions?

Ed

On Thu, Nov 16, 2017 at 5:27 AM Gabriel Ganne 
> wrote:

Hi,

We've been working on preparing vpp for armv8 continuous testing (See 
https://wiki.fd.io/view/VPP/AArch64),

and we think that we now have everything which is required to add arm as ci 
platform in the vpp review process (build, test and packaging). All the test 
have been done on "native".

Also 3 thunderX platforms sent by ARM should have arrived yesterday in the fdio 
lab.

I *think* that all that is required should be to set the platforms with VMs on 
it, and add them as build slaves.

How should we proceed from here ?

--
Gabriel Ganne

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io

Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

2017-08-23 Thread George Zhao

Agree should added ARMv8 in the makefile, does the workaround work if you 
manually put those parameters into the file?

-Original Message-
From: Brian Brooks [mailto:brian.bro...@arm.com] 
Sent: Tuesday, August 22, 2017 9:31 PM
To: George Zhao
Cc: Damjan Marion (damarion); discuss; csit-dev; vpp-dev
Subject: Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

Hi Damjan, George,

I just pulled lastest source and tried native build (platforms/vpp.mk) on ARMv8:

  cat: '/sys/bus/pci/devices/:00:01.0/uevent': No such file or directory

From dpdk/Makefile,

  ##
  # Intel x86
  ##
  ifeq ($(MACHINE),$(filter $(MACHINE),x86_64 i686))
  DPDK_TARGET   ?= $(MACHINE)-native-linuxapp-$(DPDK_CC)
  DPDK_MACHINE  ?= nhm
  DPDK_TUNE ?= core-avx2
  ##
  # Cavium ThunderX
  ##
  else ifneq (,$(findstring thunder,$(shell cat 
/sys/bus/pci/devices/:00:01.0/uevent | grep cavium)))
  export CROSS=""
  DPDK_TARGET   ?= arm64-thunderx-linuxapp-$(DPDK_CC)
  DPDK_MACHINE  ?= thunderx
  DPDK_TUNE ?= generic

So, I am thinking we need to modify this to support MACHINE=aarch64 and possibly
rework thunder detection to not fail hard on non-thunder machines.

Regards,
Brian

On 08/22 17:55:20, George Zhao wrote:
> Thanks Demjan,
> 
> Confirmed that your patches worked on our system as well.
> 
> George
> 
> From: Damjan Marion (damarion) [mailto:damar...@cisco.com]
> Sent: Tuesday, August 22, 2017 5:03 AM
> To: George Zhao
> Cc: Dave Barach (dbarach); discuss; csit-dev; vpp-dev
> Subject: Re: [vpp-dev] [discuss] Question about VPP support for ARM 64
> 
> Dear George,
> 
> I tried on my Cavium ThunderX system with latest Ubuntu and after fixing few 
> minor issues (all patches submitted to master) I got VPP running.
> I use latest Ubuntu devel (17.10, mainly as I upgraded to new kernel in my 
> attempts to get system working)
> 
> For me it is hard to help you with your particular system, as I don’t have 
> access to similar one, but my guess is that it shouldn’’t be too hard to get 
> it working.
> 
> Thanks,
> 
> Damjan
> 
> On 20 Aug 2017, at 23:12, George Zhao 
> <george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>> wrote:
> 
> Hi Damian,
> 
> IT is Applied Micro overdrive 1000, here are the uname -a output:
> 
> $>> uname -a
> Linux OD1K 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:10:33 UTC 2017 
> aarch64 aarch64 aarch64 GNU/Linux
> 
> thanks
> George
> 发件人：Damjan Marion (damarion)
> 收件人：George Zhao
> 抄送：dbarach,discuss,csit-dev,vpp-dev
> 时间：2017-08-20 10:03:27
> 主题:Re: [vpp-dev] [discuss] Question about VPP support for ARM 64
> 
> 
> 
> George, are you using ThunderX platform?
> 
> I spent few hours today trying to install latest ubuntu on my ThunderX system 
> but no luck, kernel hangs at some point, both ubuntu provided and manually 
> compiled.
> 
> Can you share about more details about your system?
> 
> Thanks,
> 
> Damjan
> 
> 
> 
> > On 19 Aug 2017, at 22:48, George Zhao 
> > <george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>> wrote:
> >
> > If a bug is filed, may I have the bug number, I would be love to trace this 
> > patch.
> >
> > BTW, how do I file a bug for VPP, I did a quick wiki search with no luck.
> >
> > Thanks,
> > George
> >
> > -Original Message-
> > From: Dave Barach (dbarach) [mailto:dbar...@cisco.com]
> > Sent: Saturday, August 19, 2017 7:42 AM
> > To: George Zhao <george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>>
> > Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; 
> > disc...@lists.fd.io<mailto:disc...@lists.fd.io>; 
> > csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; Damjan Marion (damarion) 
> > <damar...@cisco.com<mailto:damar...@cisco.com>>
> > Subject: RE: [discuss] Question about VPP support for ARM 64
> >
> > +1, pls add the typedef...
> >
> > Thanks… Dave
> >
> > -Original Message-
> > From: Damjan Marion (damarion)
> > Sent: Saturday, August 19, 2017 9:09 AM
> > To: Dave Barach (dbarach) <dbar...@cisco.com<mailto:dbar...@cisco.com>>
> > Cc: George Zhao 
> > <george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>>; 
> > vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; 
> &

Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

2017-08-22 Thread George Zhao

Thanks Demjan,

Confirmed that your patches worked on our system as well.

George

From: Damjan Marion (damarion) [mailto:damar...@cisco.com]
Sent: Tuesday, August 22, 2017 5:03 AM
To: George Zhao
Cc: Dave Barach (dbarach); discuss; csit-dev; vpp-dev
Subject: Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

Dear George,

I tried on my Cavium ThunderX system with latest Ubuntu and after fixing few 
minor issues (all patches submitted to master) I got VPP running.
I use latest Ubuntu devel (17.10, mainly as I upgraded to new kernel in my 
attempts to get system working)

For me it is hard to help you with your particular system, as I don’t have 
access to similar one, but my guess is that it shouldn’’t be too hard to get it 
working.

Thanks,

Damjan

On 20 Aug 2017, at 23:12, George Zhao 
<george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>> wrote:

Hi Damian,

IT is Applied Micro overdrive 1000, here are the uname -a output:

$>> uname -a
Linux OD1K 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:10:33 UTC 2017 
aarch64 aarch64 aarch64 GNU/Linux

thanks
George
发件人：Damjan Marion (damarion)
收件人：George Zhao
抄送：dbarach,discuss,csit-dev,vpp-dev
时间：2017-08-20 10:03:27
主题:Re: [vpp-dev] [discuss] Question about VPP support for ARM 64



George, are you using ThunderX platform?

I spent few hours today trying to install latest ubuntu on my ThunderX system 
but no luck, kernel hangs at some point, both ubuntu provided and manually 
compiled.

Can you share about more details about your system?

Thanks,

Damjan



> On 19 Aug 2017, at 22:48, George Zhao 
> <george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>> wrote:
>
> If a bug is filed, may I have the bug number, I would be love to trace this 
> patch.
>
> BTW, how do I file a bug for VPP, I did a quick wiki search with no luck.
>
> Thanks,
> George
>
> -Original Message-
> From: Dave Barach (dbarach) [mailto:dbar...@cisco.com]
> Sent: Saturday, August 19, 2017 7:42 AM
> To: George Zhao <george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>>
> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; 
> disc...@lists.fd.io<mailto:disc...@lists.fd.io>; 
> csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; Damjan Marion (damarion) 
> <damar...@cisco.com<mailto:damar...@cisco.com>>
> Subject: RE: [discuss] Question about VPP support for ARM 64
>
> +1, pls add the typedef...
>
> Thanks… Dave
>
> -Original Message-
> From: Damjan Marion (damarion)
> Sent: Saturday, August 19, 2017 9:09 AM
> To: Dave Barach (dbarach) <dbar...@cisco.com<mailto:dbar...@cisco.com>>
> Cc: George Zhao <george.y.z...@huawei.com<mailto:george.y.z...@huawei.com>>; 
> vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; 
> disc...@lists.fd.io<mailto:disc...@lists.fd.io>; 
> csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>
> Subject: Re: [discuss] Question about VPP support for ARM 64
>
>
> GCC is able to compile ARM64 code with 256-bit vectors even if target 
> platform have only 128-bit registers.
>
> I.e. for the u8x32 version of that function it generates:
>
> ARM64:
> dpdk_buffer_init_from_template(void*, void*, void*, void*, void*):
>ld1 {v0.16b - v1.16b}, [x4], 32
>st1 {v0.16b - v1.16b}, [x3], 32
>st1 {v0.16b - v1.16b}, [x2], 32
>st1 {v0.16b - v1.16b}, [x1], 32
>st1 {v0.16b - v1.16b}, [x0], 32
>ld1 {v0.16b - v1.16b}, [x4]
>st1 {v0.16b - v1.16b}, [x3]
>st1 {v0.16b - v1.16b}, [x2]
>st1 {v0.16b - v1.16b}, [x1]
>st1 {v0.16b - v1.16b}, [x0]
>ret
>
> intel x86-64 without AVX2:
>
> dpdk_buffer_init_from_template(void*, void*, void*, void*, void*):
> push   %rbp
> mov%rsp,%rbp
> and$0xffe0,%rsp
> lea0x10(%rsp),%rsp
> movdqa (%r8),%xmm1
> movdqa 0x10(%r8),%xmm0
> movdqa %xmm0,0x10(%rcx)
> movdqa %xmm1,(%rcx)
> movdqa %xmm1,(%rdx)
> movdqa %xmm0,0x10(%rdx)
> movdqa %xmm1,(%rsi)
> movdqa %xmm0,0x10(%rsi)
> movdqa %xmm1,(%rdi)
> movdqa %xmm0,0x10(%rdi)
> movdqa 0x20(%r8),%xmm1
> movdqa 0x30(%r8),%xmm0
> movdqa %xmm0,0x30(%rcx)
> movdqa %xmm1,0x20(%rcx)
> movdqa %xmm1,0x20(%rdx)
> movdqa %xmm0,0x30(%rdx)
> movdqa %xmm1,0x20(%rsi)
> movdqa %xmm0,0x30(%rsi)
> movdqa %xmm1,0x20(%rdi)
> movdqa %xmm0,0x30(%rdi)
> leaveq
> retq
>
>
> So i think here it is only about missing typedef….
>
>
>> On 19 Aug 2017, at 14:51, Dave Barach (dbarach) 
>> <dbar...@cisco.com<mailto:dbar...@cisco.com>> wrote:
>>
>> Dear George,
>>
>> This specific issue isn’t anywhere near as bad as you might think. A

[vpp-dev] 答复: [discuss] Question about VPP support for ARM 64

2017-08-20 Thread George Zhao

Hi Damian,

IT is Applied Micro overdrive 1000, here are the uname -a output:

$>> uname -a
Linux OD1K 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:10:33 UTC 2017 
aarch64 aarch64 aarch64 GNU/Linux

thanks
George
发件人：Damjan Marion (damarion)
收件人：George Zhao
抄送：dbarach,discuss,csit-dev,vpp-dev
时间：2017-08-20 10:03:27
主题:Re: [vpp-dev] [discuss] Question about VPP support for ARM 64



George, are you using ThunderX platform?

I spent few hours today trying to install latest ubuntu on my ThunderX system 
but no luck, kernel hangs at some point, both ubuntu provided and manually 
compiled.

Can you share about more details about your system?

Thanks,

Damjan



> On 19 Aug 2017, at 22:48, George Zhao <george.y.z...@huawei.com> wrote:
>
> If a bug is filed, may I have the bug number, I would be love to trace this 
> patch.
>
> BTW, how do I file a bug for VPP, I did a quick wiki search with no luck.
>
> Thanks,
> George
>
> -Original Message-
> From: Dave Barach (dbarach) [mailto:dbar...@cisco.com]
> Sent: Saturday, August 19, 2017 7:42 AM
> To: George Zhao <george.y.z...@huawei.com>
> Cc: vpp-dev@lists.fd.io; disc...@lists.fd.io; csit-...@lists.fd.io; Damjan 
> Marion (damarion) <damar...@cisco.com>
> Subject: RE: [discuss] Question about VPP support for ARM 64
>
> +1, pls add the typedef...
>
> Thanks… Dave
>
> -Original Message-
> From: Damjan Marion (damarion)
> Sent: Saturday, August 19, 2017 9:09 AM
> To: Dave Barach (dbarach) <dbar...@cisco.com>
> Cc: George Zhao <george.y.z...@huawei.com>; vpp-dev@lists.fd.io; 
> disc...@lists.fd.io; csit-...@lists.fd.io
> Subject: Re: [discuss] Question about VPP support for ARM 64
>
>
> GCC is able to compile ARM64 code with 256-bit vectors even if target 
> platform have only 128-bit registers.
>
> I.e. for the u8x32 version of that function it generates:
>
> ARM64:
> dpdk_buffer_init_from_template(void*, void*, void*, void*, void*):
>ld1 {v0.16b - v1.16b}, [x4], 32
>st1 {v0.16b - v1.16b}, [x3], 32
>st1 {v0.16b - v1.16b}, [x2], 32
>st1 {v0.16b - v1.16b}, [x1], 32
>st1 {v0.16b - v1.16b}, [x0], 32
>ld1 {v0.16b - v1.16b}, [x4]
>st1 {v0.16b - v1.16b}, [x3]
>st1 {v0.16b - v1.16b}, [x2]
>st1 {v0.16b - v1.16b}, [x1]
>st1 {v0.16b - v1.16b}, [x0]
>ret
>
> intel x86-64 without AVX2:
>
> dpdk_buffer_init_from_template(void*, void*, void*, void*, void*):
> push   %rbp
> mov%rsp,%rbp
> and$0xffe0,%rsp
> lea0x10(%rsp),%rsp
> movdqa (%r8),%xmm1
> movdqa 0x10(%r8),%xmm0
> movdqa %xmm0,0x10(%rcx)
> movdqa %xmm1,(%rcx)
> movdqa %xmm1,(%rdx)
> movdqa %xmm0,0x10(%rdx)
> movdqa %xmm1,(%rsi)
> movdqa %xmm0,0x10(%rsi)
> movdqa %xmm1,(%rdi)
> movdqa %xmm0,0x10(%rdi)
> movdqa 0x20(%r8),%xmm1
> movdqa 0x30(%r8),%xmm0
> movdqa %xmm0,0x30(%rcx)
> movdqa %xmm1,0x20(%rcx)
> movdqa %xmm1,0x20(%rdx)
> movdqa %xmm0,0x30(%rdx)
> movdqa %xmm1,0x20(%rsi)
> movdqa %xmm0,0x30(%rsi)
> movdqa %xmm1,0x20(%rdi)
> movdqa %xmm0,0x30(%rdi)
> leaveq
> retq
>
>
> So i think here it is only about missing typedef….
>
>
>> On 19 Aug 2017, at 14:51, Dave Barach (dbarach) <dbar...@cisco.com> wrote:
>>
>> Dear George,
>>
>> This specific issue isn’t anywhere near as bad as you might think. As given, 
>> the code confuses 128-bit vectors with 256-bit vectors, and 64-bit vectors 
>> with 128-bit vectors.
>>
>> Question: does the hardware involved support 256-bit vectors? Probably 
>> not... It almost certainly does support 128-bit vectors.
>>
>> To make progress, use the known-good u8x16 / 128-bit vector code:
>>
>> static_always_inline void
>> dpdk_buffer_init_from_template (void *d0, void *d1, void *d2, void *d3,
>>  void *s)
>> {
>> #if defined(CLIB_HAVE_VEC128)
>>  int i;
>>  for (i = 0; i < 4; i++)
>>{
>>  *(u8x16 *) (((u8 *) d0) + i * 16) =
>> *(u8x16 *) (((u8 *) d1) + i * 16) =
>> *(u8x16 *) (((u8 *) d2) + i * 16) =
>> *(u8x16 *) (((u8 *) d3) + i * 16) = *(u8x16 *) (((u8 *) s) + i * 16);
>>}
>> #else
>> #error "CLIB_HAVE_VEC128 has to be defined"
>> #endif
>> }
>>
>> Responsible parties - they know who they are - will be back from PTO 
>> shortly. We need to clean up / create CLIB_HAVE_VEC_256 and move the 256-bit 
>> vector engine code...
>>
>> You could also try adding “typedef u8 u8x32 _vector_size(32)” but I somehow 
>> doubt that will produce anything other than a compiler err

Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

2017-08-19 Thread George Zhao

If a bug is filed, may I have the bug number, I would be love to trace this 
patch.

BTW, how do I file a bug for VPP, I did a quick wiki search with no luck.

Thanks,
George

-Original Message-
From: Dave Barach (dbarach) [mailto:dbar...@cisco.com] 
Sent: Saturday, August 19, 2017 7:42 AM
To: George Zhao <george.y.z...@huawei.com>
Cc: vpp-dev@lists.fd.io; disc...@lists.fd.io; csit-...@lists.fd.io; Damjan 
Marion (damarion) <damar...@cisco.com>
Subject: RE: [discuss] Question about VPP support for ARM 64

+1, pls add the typedef...

Thanks… Dave

-Original Message-
From: Damjan Marion (damarion) 
Sent: Saturday, August 19, 2017 9:09 AM
To: Dave Barach (dbarach) <dbar...@cisco.com>
Cc: George Zhao <george.y.z...@huawei.com>; vpp-dev@lists.fd.io; 
disc...@lists.fd.io; csit-...@lists.fd.io
Subject: Re: [discuss] Question about VPP support for ARM 64


GCC is able to compile ARM64 code with 256-bit vectors even if target platform 
have only 128-bit registers.

I.e. for the u8x32 version of that function it generates:

ARM64:
dpdk_buffer_init_from_template(void*, void*, void*, void*, void*):
ld1 {v0.16b - v1.16b}, [x4], 32
st1 {v0.16b - v1.16b}, [x3], 32
st1 {v0.16b - v1.16b}, [x2], 32
st1 {v0.16b - v1.16b}, [x1], 32
st1 {v0.16b - v1.16b}, [x0], 32
ld1 {v0.16b - v1.16b}, [x4]
st1 {v0.16b - v1.16b}, [x3]
st1 {v0.16b - v1.16b}, [x2]
st1 {v0.16b - v1.16b}, [x1]
st1 {v0.16b - v1.16b}, [x0]
ret

intel x86-64 without AVX2:

dpdk_buffer_init_from_template(void*, void*, void*, void*, void*):
 push   %rbp
 mov%rsp,%rbp
 and$0xffe0,%rsp
 lea0x10(%rsp),%rsp
 movdqa (%r8),%xmm1
 movdqa 0x10(%r8),%xmm0
 movdqa %xmm0,0x10(%rcx)
 movdqa %xmm1,(%rcx)
 movdqa %xmm1,(%rdx)
 movdqa %xmm0,0x10(%rdx)
 movdqa %xmm1,(%rsi)
 movdqa %xmm0,0x10(%rsi)
 movdqa %xmm1,(%rdi)
 movdqa %xmm0,0x10(%rdi)
 movdqa 0x20(%r8),%xmm1
 movdqa 0x30(%r8),%xmm0
 movdqa %xmm0,0x30(%rcx)
 movdqa %xmm1,0x20(%rcx)
 movdqa %xmm1,0x20(%rdx)
 movdqa %xmm0,0x30(%rdx)
 movdqa %xmm1,0x20(%rsi)
 movdqa %xmm0,0x30(%rsi)
 movdqa %xmm1,0x20(%rdi)
 movdqa %xmm0,0x30(%rdi)
 leaveq 
 retq   


So i think here it is only about missing typedef….


> On 19 Aug 2017, at 14:51, Dave Barach (dbarach) <dbar...@cisco.com> wrote:
> 
> Dear George,
>  
> This specific issue isn’t anywhere near as bad as you might think. As given, 
> the code confuses 128-bit vectors with 256-bit vectors, and 64-bit vectors 
> with 128-bit vectors.
>  
> Question: does the hardware involved support 256-bit vectors? Probably not... 
> It almost certainly does support 128-bit vectors.
>  
> To make progress, use the known-good u8x16 / 128-bit vector code:   
>  
> static_always_inline void
> dpdk_buffer_init_from_template (void *d0, void *d1, void *d2, void *d3,
>   void *s)
> {
> #if defined(CLIB_HAVE_VEC128)
>   int i;
>   for (i = 0; i < 4; i++)
> {
>   *(u8x16 *) (((u8 *) d0) + i * 16) =
>  *(u8x16 *) (((u8 *) d1) + i * 16) =
>  *(u8x16 *) (((u8 *) d2) + i * 16) =
>  *(u8x16 *) (((u8 *) d3) + i * 16) = *(u8x16 *) (((u8 *) s) + i * 16);
> }
> #else
> #error "CLIB_HAVE_VEC128 has to be defined"
> #endif
> }
>  
> Responsible parties - they know who they are - will be back from PTO shortly. 
> We need to clean up / create CLIB_HAVE_VEC_256 and move the 256-bit vector 
> engine code...
>  
> You could also try adding “typedef u8 u8x32 _vector_size(32)” but I somehow 
> doubt that will produce anything other than a compiler error.
>  
> HTH… Dave
>  
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On 
> Behalf Of George Zhao
> Sent: Friday, August 18, 2017 7:32 PM
> To: 'vpp-dev@lists.fd.io' <vpp-dev@lists.fd.io>; 'disc...@lists.fd.io' 
> <disc...@lists.fd.io>; 'csit-...@lists.fd.io' <csit-...@lists.fd.io>
> Subject: [vpp-dev] Question about VPP support for ARM 64
>  
> We encounter following issues while trying to build VPP over ARM 64. It seems 
> right now only ARM32 are supported in the code. I list the steps we tried and 
> hope VPP folks can help us work around this issue.
>  
> Steps: 
> 1. install Ubuntu 16.04 on OD1K  
> $>> uname -a
> Linux OD1K 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:10:33 UTC 2017 
> aarch64 aarch64 aarch64 GNU/Linux
>  
> 2. git clone VPP 17.04 and build VPP
> ## Error:
> make[2]: Entering directory '/home/huawei/GIT/vpp.1704/dpdk'
> cat: '/sys/bus/pci/devices/:00:01.0/uevent': No such file or directory
>  
> **Work around to bypass MakeFile:
> ##
> # Cavium ThunderX
> #

[vpp-dev] Question about VPP support for ARM 64

2017-08-18 Thread George Zhao

We encounter following issues while trying to build VPP over ARM 64. It seems 
right now only ARM32 are supported in the code. I list the steps we tried and 
hope VPP folks can help us work around this issue.

Steps:
1. install Ubuntu 16.04 on OD1K
$>> uname -a
Linux OD1K 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:10:33 UTC 2017 
aarch64 aarch64 aarch64 GNU/Linux

2. git clone VPP 17.04 and build VPP
## Error:
make[2]: Entering directory '/home/huawei/GIT/vpp.1704/dpdk'
cat: '/sys/bus/pci/devices/:00:01.0/uevent': No such file or directory

**Work around to bypass MakeFile:
##
# Cavium ThunderX
##
#else ifneq (,$(findstring thunder,$(shell cat 
/sys/bus/pci/devices/:00:01.0/uevent | grep cavium)))
else
export CROSS=""
DPDK_TARGET   ?= arm64-thunderx-linuxapp-$(DPDK_CC)
DPDK_MACHINE  ?= thunderx
DPDK_TUNE ?= generic

3. Then,  make build and failed following:
/home/huawei/GIT/vpp.1704/build-data/../src/plugins/dpdk/device/node.c:276:9: 
error: `u8x32' undeclared (first use in this function)
   *(u8x32 *) (((u8 *) d0) + i * 32) =

** Check vppinfra/vppinfra/vector.h   and don't find u8x32 with "aarch64"
#if defined (__aarch64__) || defined (__arm__)
typedef unsigned int u32x4 _vector_size (16);
typedef u8 u8x16 _vector_size (16);
typedef u16 u16x8 _vector_size (16);
typedef u32 u32x4 _vector_size (16);
typedef u64 u64x2 _vector_size (16);
#endif

4. According  https://wiki.fd.io/view/VPP/Alternative_builds
The VPP seems to support arm32 only .
export PLATFORM=arm32


*Questions:
Did I miss some steps or should include other header files that defines u8x32?


Thanks,
George

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] 17.07 Release

2017-07-20 Thread George Zhao

Bravo!!!

-Original Message-
From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On 
Behalf Of Dave Wallace
Sent: Thursday, July 20, 2017 7:25 PM
To: Neale Ranns (nranns); vpp-dev@lists.fd.io; csit-...@lists.fd.io
Subject: Re: [vpp-dev] [csit-dev] 17.07 Release

Most Excellent!

Thanks Neale and everyone who contributed to the VPP 17.07 release.
-daw-

On 07/20/2017 07:56 PM, Neale Ranns (nranns) wrote:
> Dear VPP community,
>
> The VPP 17.07 release is complete. The release artefacts are now available on 
> the nexus server.
>
> I’d like to take this opportunity to thank you all for your continued support 
> for VPP.
>
> Best regards,
> Neale
>
>
> ___
> csit-dev mailing list
> csit-...@lists.fd.io
> https://lists.fd.io/mailman/listinfo/csit-dev

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [tsc] Project Proposal for Sweetcomb

Re: [vpp-dev] [csit-dev] VPP Release 18.07 is out!

Re: [vpp-dev] Sad News from Intel - Venky Venkatesan

Re: [vpp-dev] [csit-dev] fd.io Community Goals and Objectives for reporting to the LFN board

Re: [vpp-dev] [csit-dev] VPP 18.01 Release artifacts are now available on nexus.fd.io

Re: [vpp-dev] [tsc] [nsh_sfc-dev] Stepping down as NSH_SFC PTL

Re: [vpp-dev] [csit-dev] vpp continuous integration on armv8

Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

[vpp-dev] 答复: [discuss] Question about VPP support for ARM 64

Re: [vpp-dev] [discuss] Question about VPP support for ARM 64

[vpp-dev] Question about VPP support for ARM 64

Re: [vpp-dev] [csit-dev] 17.07 Release

13 matches

Site Navigation

Mail list logo

Footer information