Re: [vpp-dev] unformat %s eats newlines

[Dave Barach (dbarach)]

Folks who need even slightly bulletproof configuration methods should use 
binary APIs: directly from C or through one of several language bindings.

Debug CLI is a developer’s tool, subject to change without notice, and 
supported at the implementer’s discretion.

Extra and/or unparsed input should not go unnoticed: the next function up the 
parse stack will complain.

IIWY I’d leave unformat(…) alone.

HTH… D.

From: Andreas Schultz [mailto:andreas.schu...@travelping.com]
Sent: Friday, February 2, 2018 3:26 PM
To: Dave Barach (dbarach) 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] unformat %s eats newlines

Dave Barach (dbarach) > schrieb am 
Fr., 2. Feb. 2018 um 19:22 Uhr:
Why not simply:

while (…)
  {
if (unformat(input, “name %s”, ))
  ;
else if (…)
  ;
else
 break;
  }

if ()
return clib_error_return (0, "parse error: '%U'",
  format_unformat_error, input);

That would mean that malformated optional and random additional stuff would get 
unnoticed. CLI verification is already not that strong (the usual while loop 
parsing permits random argument order even when the help strings suggest 
strongly ordered arguments).

Is there a reason that unformat eats the newline or is just to hard to change?

Andreas

D.

From: vpp-dev-boun...@lists.fd.io 
[mailto:vpp-dev-boun...@lists.fd.io] On 
Behalf Of Andreas Schultz
Sent: Friday, February 2, 2018 12:47 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] unformat %s eats newlines

A typical construct to parse arguments is to use unformat in a while loop that 
checks for UNFORMAT_END_OF_INPUT.
For multiline input that relies on the detection of "\n" in the input stream.

The problem is that a construct like:

unformat (input, "name %_%v%_", )

eats the newline when it is the only characted following the string to be 
parsed.

This even break reading a multi line config with exec.

Regards
Andreas
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

There are even more filtering options, allowing you to specify a class or a 
single test function within a test case if needed.

Please consult `make test-help`.

Regards,
Klement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of John Lo (loj)
> Sent: Friday, February 2, 2018 10:40 PM
> To: Jon Loeliger 
> Cc: vpp-dev 
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
> Hi Jon,
> 
> 
> 
> You can easily run just vxlan tests by “make test TEST=vxlan”, or better yet
> “make test-debug TEST=vxlan”, to run just test_vxlan.py script in VPP normal
> or debug mode. Similarly for other test cases.
> 
> 
> 
> Regards,
> 
> John
> 
> 
> 
> From: Jon Loeliger [mailto:j...@netgate.com]
> Sent: Friday, February 02, 2018 4:18 PM
> To: John Lo (loj) 
> Cc: vpp-dev 
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
> 
> 
> On Fri, Feb 2, 2018 at 3:08 PM, Jon Loeliger   > wrote:
> 
> 
> 
>   I have submitted a patch to Gerrit.  When this passes "verify", I'll add
> you to be
> 
>   a Reviewer!
> 
> 
> 
> John,
> 
> 
> 
> How do I run just these tests locally?  I need to determine if
> 
> these failures are due to my patch or not.
> 
> 
> 
> Specifically, I am not able to succesfully "make test" on a CentOS system
> 
> due to Python having a memory corruption failure and dumping core on me.
> 
> As many tests until that point were passing though.
> 
> 
> 
> I'll add you as a patch reviewer anyway -- maybe you can spot something
> 
> brain-dead that I did in the patch...?
> 
> 
> 
> Thanks,
> 
> jdl
> 
> 
> 
> 
> 
> 20:46:28 20:33:41
> ===
> ===
> ==
> 20:46:28 20:33:41 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func ::
> *Ipv6 Multipath routing test cases*
> 20:46:28 20:33:41
> ===
> ===
> ==
> 20:46:28 20:34:06 TC01: IPv6 Equal-cost multipath routing :: [Top] TG=DUT
> | PASS |
> 20:46:28 20:34:06 
> ---
> -
> 20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func ::
> *Ipv6 Multipath routing test cases* | 
> PASS |
> 20:46:28 20:34:06 1 critical test, 1 passed, 0 failed
> 20:46:28 20:34:06 1 test total, 1 passed, 0 failed
> 20:46:28 20:34:06
> ===
> ===
> ==
> 20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func ::
> *IPv6 Router Advertisement test cases*
> 20:46:28 20:34:06
> ===
> ===
> ==
> 20:46:28 20:34:20 TC01: DUT transmits RA on IPv6 enabled interface :: [Top]
> TG-DUT1-DUT2-TG.  | PASS |
> 20:46:28 20:34:20 
> ---
> -
> 20:46:28 20:34:36 TC02: DUT retransmits RA on IPv6 enabled interface after a
> set interval :: [Top] TG-DUT1-DUT2-TG.   | PASS |
> 20:46:28 20:34:36 
> ---
> -
> 20:46:28 20:34:49 TC03: DUT responds to Router Solicitation request :: [Top]
> TG-DUT1-DUT2-TG. | FAIL |
> 20:46:28 20:34:49 Traffic script execution failed
> 20:46:28 20:34:49 
> ---
> -
> 20:46:28 20:35:02 TC04: DUT responds to Router Solicitation request sent
> from link local address :: [Top] TG-DUT1-DUT2-TG.| 
> FAIL |
> 20:46:28 20:35:02 Traffic script execution failed
> 20:46:28 20:35:02 
> ---
> -
> 20:46:28 20:35:02 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func ::
> *IPv6 Router Advertisement test cases*| 
> PASS |
> 20:46:28 20:35:02 0 critical tests, 0 passed, 0 failed
> 20:46:28 20:35:02 4 tests total, 2 passed, 2 failed
> 20:46:28 20:35:02
> ===
> ===
> ==
> 
> 

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

Hi Jon,

You can easily run just vxlan tests by “make test TEST=vxlan”, or better yet 
“make test-debug TEST=vxlan”, to run just test_vxlan.py script in VPP normal or 
debug mode. Similarly for other test cases.

Regards,
John

From: Jon Loeliger [mailto:j...@netgate.com]
Sent: Friday, February 02, 2018 4:18 PM
To: John Lo (loj) 
Cc: vpp-dev 
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Fri, Feb 2, 2018 at 3:08 PM, Jon Loeliger 
> wrote:

I have submitted a patch to Gerrit.  When this passes "verify", I'll add you to 
be
a Reviewer!

John,

How do I run just these tests locally?  I need to determine if
these failures are due to my patch or not.

Specifically, I am not able to succesfully "make test" on a CentOS system
due to Python having a memory corruption failure and dumping core on me.
As many tests until that point were passing though.

I'll add you as a patch reviewer anyway -- maybe you can spot something
brain-dead that I did in the patch...?

Thanks,
jdl



20:46:28 20:33:41 


20:46:28 20:33:41 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6 
Multipath routing test cases*

20:46:28 20:33:41 


20:46:28 20:34:06 TC01: IPv6 Equal-cost multipath routing :: [Top] TG=DUT   
  | PASS |

20:46:28 20:34:06 


20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6 
Multipath routing test cases* | PASS |

20:46:28 20:34:06 1 critical test, 1 passed, 0 failed

20:46:28 20:34:06 1 test total, 1 passed, 0 failed

20:46:28 20:34:06 


20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6 
Router Advertisement test cases*

20:46:28 20:34:06 


20:46:28 20:34:20 TC01: DUT transmits RA on IPv6 enabled interface :: [Top] 
TG-DUT1-DUT2-TG.  | PASS |

20:46:28 20:34:20 


20:46:28 20:34:36 TC02: DUT retransmits RA on IPv6 enabled interface after a 
set interval :: [Top] TG-DUT1-DUT2-TG.   | PASS |

20:46:28 20:34:36 


20:46:28 20:34:49 TC03: DUT responds to Router Solicitation request :: [Top] 
TG-DUT1-DUT2-TG. | FAIL |

20:46:28 20:34:49 Traffic script execution failed

20:46:28 20:34:49 


20:46:28 20:35:02 TC04: DUT responds to Router Solicitation request sent from 
link local address :: [Top] TG-DUT1-DUT2-TG.| FAIL |

20:46:28 20:35:02 Traffic script execution failed

20:46:28 20:35:02 


20:46:28 20:35:02 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6 
Router Advertisement test cases*| PASS |

20:46:28 20:35:02 0 critical tests, 0 passed, 0 failed

20:46:28 20:35:02 4 tests total, 2 passed, 2 failed

20:46:28 20:35:02 


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Fri, Feb 2, 2018 at 3:08 PM, Jon Loeliger  wrote:

>
> I have submitted a patch to Gerrit.  When this passes "verify", I'll add
> you to be
> a Reviewer!
>

John,

How do I run just these tests locally?  I need to determine if
these failures are due to my patch or not.

Specifically, I am not able to succesfully "make test" on a CentOS system
due to Python having a memory corruption failure and dumping core on me.
As many tests until that point were passing though.

I'll add you as a patch reviewer anyway -- maybe you can spot something
brain-dead that I did in the patch...?

Thanks,
jdl


*20:46:28* 20:33:41
*20:46:28*
20:33:41 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6
Multipath routing test cases*
   *20:46:28* 20:33:41
*20:46:28*
20:34:06 TC01: IPv6 Equal-cost multipath routing :: [Top] TG=DUT
  |
PASS |*20:46:28* 20:34:06
*20:46:28*
20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6
Multipath routing test cases* |
PASS |*20:46:28* 20:34:06 1 critical test, 1 passed, 0
failed*20:46:28* 20:34:06 1 test total, 1 passed, 0 failed*20:46:28*
20:34:06 
*20:46:28*
20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6
Router Advertisement test cases*
 *20:46:28* 20:34:06
*20:46:28*
20:34:20 TC01: DUT transmits RA on IPv6 enabled interface :: [Top]
TG-DUT1-DUT2-TG.
| PASS |*20:46:28* 20:34:20
*20:46:28*
20:34:36 TC02: DUT retransmits RA on IPv6 enabled interface after a
set interval :: [Top] TG-DUT1-DUT2-TG.   |
PASS |*20:46:28* 20:34:36
*20:46:28*
20:34:49 TC03: DUT responds to Router Solicitation request :: [Top]
TG-DUT1-DUT2-TG. |
FAIL |*20:46:28* 20:34:49 Traffic script execution failed*20:46:28*
20:34:49 
*20:46:28*
20:35:02 TC04: DUT responds to Router Solicitation request sent from
link local address :: [Top] TG-DUT1-DUT2-TG.|
FAIL |*20:46:28* 20:35:02 Traffic script execution failed*20:46:28*
20:35:02 
*20:46:28*
20:35:02 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6
Router Advertisement test cases*|
PASS |*20:46:28* 20:35:02 0 critical tests, 0 passed, 0
failed*20:46:28* 20:35:02 4 tests total, 2 passed, 2 failed*20:46:28*
20:35:02 

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj)  wrote:

> Hi Jon,
>

Hi John,

Thanks for taking the time to get back to me on this!

All VPP tunnel creation uses the mechanism of returning a sw_if_index of
> the created tunnel.
>

I know.  I get that.  It works.


> The name of the tunnel is then followed by a number being the instance or
> index to the tunnel struct vector. Thus, the first VXLAN tunnel created is
> called vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is
> incrementing unless tunnels are deleted and created again where a newly
> created tunnel will reuse the last deleted tunnel, hence its name. If all
> previously deleted tunnels are used up, then the tunnel name number will
> start incrementing again.
>

Right.


> I am not sure if it is feasible to follow this behavior to “predict”
> tunnel name.
>

It is not.

The problem is not just "predict it", but the user has to be
able to know the associated SW IF name at the time that
the (vxlan tunnel) create API call happens.

The reason for that is because the very next thing the
API call user is going to want to do is configure that interface.
Short of interrogating the system, there is no way to know
the IF name.  (I understand that the name can be obtained
from the sw_if_index.  That's not what I mean.)

Consider this series of PEUDO API calls that are being
issued by some client front-end.  Think of this as a batch
of CLI commands:

vxlan MyVxlan23
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface 
do stuff to interface like apply an ACL or whatever
exit

There is no way to batch-run those as there is a step
needed to determine what  is.  It could be "vxlan_tunnel..."
anything.

That  has to match a VPP interface by name.  But what name?
The entire transactional  history of VXLANs must be known in
order to guess the next name.

BTW, GRE tunnels are created and named the same way. You added TEB
> (transparent ethernet bridging) support to GRE. Have you not been using it
> and bothered by how to predict name of created GRE tunnels?
>

I've not touched GRE yet.  However, it is next on my list to fix!
My current plan is to submit essentially the same patch there too.

It is not a good idea to tie tunnel name to VNI as it is not a unique ID to
> identify VXLAN tunnels. It is quite common for existence of multiple VXLAN
> tunnels with the same VNI with different remote end point IP addresses. A
> bridge domain (BD) may have multiple VXLAN tunnels with the same VNI to
> several remote servers where the same overlay network exist. It is quite
> common to use the ID for BD or VLAN as the VNI for all its VXLAN tunnels to
> make it easier to track their usage.
>

Excellent!  Thank you for this insight!  That makes total sense now!

I suppose it is feasible to provide another set of VXLAN tunnel
> create/delete API to allow caller to specify instance or number of the
> tunnel, similar to what you did for loopback interface.
>

This is the route I was headed:  Just like loopback interface naming!

In this case, the above example would be, perhaps, something like this:

vxlan MyVxlan23
instance 101
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface vxlan_tunnel101
do stuff to interface like apply an ACL or whatever
exit


  One complication for the new API is that, upon VXLAN tunnel deletion, the
> interface created for the tunnel is not really deleted by the current code
> but put into a reuse pool.
>

I solved that by making a trivial  vxlan_name_renumber()  function,
and then calling vnet_interface_name_remumber(sw_if_index, instance)
to update the previously captured "vxlan_tunnel" name and rename
it using the new instance number.

When more tunnels are created, any interface from reuse pool with its
> existing interface name will be used for the new tunnel, before new
> interfaces are created. If a interface is reused upon tunnel creation, its
> interface name may not match the specified tunnel instance/number of the
> new tunnel creation API.  One way to overcome this may be to not keep
> interface in reused pool on tunnel deletion. Thus, tunnel creation would
> always create new interface.  For backward compatibility, I suppose we can
> keep the tunnel create/delete API as is so interfaces of deleted tunnels
> are kept for reuse. The new API will then always create/delete interface on
> tunnel create/delete.  This would put a restriction that user should not
> mix the usage of new or old APIs.
>

To me, that sounds like a whole bunch of really unnecessary code replication
and fragile separation of API results that will invariably get mixed up and
cause mysterious failures.

Instead, I maintain the original instance allocation scheme when there is
no requested Instance Id made by the CLI or API user.

I have submitted a patch to Gerrit.  When this passes "verify", I'll add
you to be
a Reviewer!

Re: [vpp-dev] unformat %s eats newlines

[Andreas Schultz]

Florin Coras  schrieb am Fr., 2. Feb. 2018 um
18:57 Uhr:

> Not exactly the most elegant solution but have you tried adding a space
> after the string to be parsed?
>

Tried that, but doesn't help.

Andreas

Florin
>
> > On Feb 2, 2018, at 9:47 AM, Andreas Schultz <
> andreas.schu...@travelping.com> wrote:
> >
> > A typical construct to parse arguments is to use unformat in a while
> loop that checks for UNFORMAT_END_OF_INPUT.
> > For multiline input that relies on the detection of "\n" in the input
> stream.
> >
> > The problem is that a construct like:
> >
> > unformat (input, "name %_%v%_", )
> >
> > eats the newline when it is the only characted following the string to
> be parsed.
> >
> > This even break reading a multi line config with exec.
> >
> > Regards
> > Andreas
> > ___
> > vpp-dev mailing list
> > vpp-dev@lists.fd.io
> > https://lists.fd.io/mailman/listinfo/vpp-dev
>
>
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] unformat %s eats newlines

[Andreas Schultz]

Dave Barach (dbarach)  schrieb am Fr., 2. Feb. 2018 um
19:22 Uhr:

> Why not simply:
>
>
>
> while (…)
>
>   {
>
> if (unformat(input, “name %s”, ))
>
>   ;
>
> else if (…)
>
>   ;
>
> else
>
>  break;
>
>   }
>
>
>
> if ()
>
> return clib_error_return (0, "parse error: '%U'",
>
>   format_unformat_error, input);
>
>
That would mean that malformated optional and random additional stuff would
get unnoticed. CLI verification is already not that strong (the usual while
loop parsing permits random argument order even when the help strings
suggest strongly ordered arguments).

Is there a reason that unformat eats the newline or is just to hard to
change?

Andreas

>
>
> D.
>
>
>
> *From:* vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] *On
> Behalf Of *Andreas Schultz
> *Sent:* Friday, February 2, 2018 12:47 PM
> *To:* vpp-dev@lists.fd.io
> *Subject:* [vpp-dev] unformat %s eats newlines
>
>
>
> A typical construct to parse arguments is to use unformat in a while loop
> that checks for UNFORMAT_END_OF_INPUT.
>
> For multiline input that relies on the detection of "\n" in the input
> stream.
>
>
>
> The problem is that a construct like:
>
>
>
> unformat (input, "name %_%v%_", )
>
>
>
> eats the newline when it is the only characted following the string to be
> parsed.
>
>
>
> This even break reading a multi line config with exec.
>
>
>
> Regards
>
> Andreas
>
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] unformat %s eats newlines

[Dave Barach (dbarach)]

Why not simply:

while (…)
  {
if (unformat(input, “name %s”, ))
  ;
else if (…)
  ;
else
 break;
  }

if ()
return clib_error_return (0, "parse error: '%U'",
  format_unformat_error, input);

D.

From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On 
Behalf Of Andreas Schultz
Sent: Friday, February 2, 2018 12:47 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] unformat %s eats newlines

A typical construct to parse arguments is to use unformat in a while loop that 
checks for UNFORMAT_END_OF_INPUT.
For multiline input that relies on the detection of "\n" in the input stream.

The problem is that a construct like:

unformat (input, "name %_%v%_", )

eats the newline when it is the only characted following the string to be 
parsed.

This even break reading a multi line config with exec.

Regards
Andreas
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] unformat %s eats newlines

[Florin Coras]

Not exactly the most elegant solution but have you tried adding a space after 
the string to be parsed?

Florin

> On Feb 2, 2018, at 9:47 AM, Andreas Schultz  
> wrote:
> 
> A typical construct to parse arguments is to use unformat in a while loop 
> that checks for UNFORMAT_END_OF_INPUT.
> For multiline input that relies on the detection of "\n" in the input stream.
> 
> The problem is that a construct like:
> 
> unformat (input, "name %_%v%_", )
> 
> eats the newline when it is the only characted following the string to be 
> parsed.
> 
> This even break reading a multi line config with exec.
> 
> Regards
> Andreas
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] unformat %s eats newlines

[Andreas Schultz]

A typical construct to parse arguments is to use unformat in a while loop
that checks for UNFORMAT_END_OF_INPUT.
For multiline input that relies on the detection of "\n" in the input
stream.

The problem is that a construct like:

unformat (input, "name %_%v%_", )

eats the newline when it is the only characted following the string to be
parsed.

This even break reading a multi line config with exec.

Regards
Andreas
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev