Re: [vpp-dev] SNAT API Question

[Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

I share Dave's opinion regarding API documentation.
It is hard to keep it up to date. In my opinion this applies to wiki pages as 
well.
Often tests are often much better source of information.
Dave mentioned "make test" framework.
Other good source of information are CSIT tests,
which also use binary api.

Honeycomb provides YANG models for vpp features.
To design such model we need to know which binary APIs to use
for creating/reading/updating/deleting feature configuration.

We need to know correct order of api call invocation (if more than one is 
needed),
what are valid input parameters, which parameters are optional, etc.

Here is my usual workflow:


1)  check binary api documentation if exists

2)  check make test / CSIT to see what combinations of parameters and api 
calls actually work

3)  check documentation for corresponding CLI cmd (these are usually much 
better documented)

4)  compare parameter handling for CLI and api handler (it can differ!)

5)  ask vpp-dev

Regards,
Marek

From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On 
Behalf Of Dave Wallace
Sent: 21 lutego 2017 18:34
To: Jon Loeliger <j...@netgate.com>; Matus Fabian -X (matfabia - PANTHEON 
TECHNOLOGIES at Cisco) <matfa...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] SNAT API Question

Jon,

Comments inline...
On 02/20/2017 03:14 PM, Jon Loeliger wrote:
Hi Matus,

Thanks for your answers here.

On Sat, Feb 18, 2017 at 12:36 AM, Matus Fabian -X (matfabia - PANTHEON 
TECHNOLOGIES at Cisco) <matfa...@cisco.com<mailto:matfa...@cisco.com>> wrote:
If external_sw_if_index value is ~0 (-1) external_ip_address is ussed from API 
(snat.c line 363).

OK, I see that in the code, but it is nowhere described in the API itself.
That is my issue here.  Magic values like this *are* part of the API as they
will affect changes on the VPP-side of the API interface.
The lack of comprehensive VPP-API documentation is a known issue.  In my 
experience this has been a universal condition with every system I've ever 
worked on. Any contributions to improve the VPP-API documentation would be very 
greatly appreciated.


snat_add_address_range - add address range to SNAT address pool
snat_add_del_interface_addr - add address of the interface to SNAT address pool 
(address is added/removed automatically when interface address is changed by 
configuration or DHCP)

So one could use either or both, as they wish?

I guess I'm having a bit of hard time trying to figure out what the
canonical API call sequence would be to set up various useful
and standard NAT situations.  Yes, I've read the Wiki page; No it
isn't clear on the proper, expected sequence of the API calls.
This is another variation on the theme of VPP-API documentation (or lack 
thereof).  I recently added the saving of the api trace log and dump to the 
"make test" infrastructure specifically to be able to data mine the API 
sequences embedded in the test cases.

After running "make test" you will now find the binary api trace log 
(/tmp/vpp-unittest-TEST*/vpp_api_trace..log) and a dump of the api 
trace in the test log (search for "api trace" in 
/tmp/vpp-unittest-TEST*/log.txt). You can recreate the configuration using the 
vpp_api_trace..log and the custom-dump mechanism to re-create the 
configuration as described in 
https://wiki.fd.io/view/VPP/How_To_Use_The_API_Trace_Tools

While not perfect solution to the problem, it is the most pragmatic approach I 
could think of to address this issue.  Hopefully this will help encourage 
feature developers to expand the number of use cases in the "make test" 
framework.  Of course, converting the api trace data into usable documentation 
is another task to be done.  Automating this task is probably the best way to 
attack this issue, otherwise the documentation will go stale over time.  Just a 
matter of a little more typing ;^)


How does those API sequences change for the different values of the config
variables "static_mapping_only" and "static_mapping_connection_tracking"?

And on that note -- How do you change the value of those config varaibles
at run time?  I know how to alter the config file and re-read them in VAT.
I'm not using VAT.  I'm writing a totally different system.  Can those values
be changed at run-time?  There is no API to do so, so at this point in time
the answer must be "no."  Is it expected that the sequence to change these
configuration values at run time is to: 1) Stop VPP, 2) write a new config file
with the new desired values, 3) restart VPP?  That seems bad to me.
I'll let Matus address the SNAT specifics, but in general appropriate VPP-API 
interfaces should be developed to allow dynamic configuration of VPP features.  
That being said, it is less work to develop basic functionality using 
configuration variables prior to creating a compre

Re: [vpp-dev] SNAT API Question

[Dave Wallace]

Jon,

Comments inline...

On 02/20/2017 03:14 PM, Jon Loeliger wrote:

Hi Matus,

Thanks for your answers here.

On Sat, Feb 18, 2017 at 12:36 AM, Matus Fabian -X (matfabia - PANTHEON 
TECHNOLOGIES at Cisco) > wrote:


If external_sw_if_index value is ~0 (-1) external_ip_address is
ussed from API (snat.c line 363).


OK, I see that in the code, but it is nowhere described in the API itself.
That is my issue here.  Magic values like this *are* part of the API 
as they

will affect changes on the VPP-side of the API interface.
The lack of comprehensive VPP-API documentation is a known issue.  In my 
experience this has been a universal condition with every system I've 
ever worked on. Any contributions to improve the VPP-API documentation 
would be very greatly appreciated.


snat_add_address_range – add address range to SNAT address pool

snat_add_del_interface_addr – add address of the interface to SNAT
address pool (address is added/removed automatically when
interface address is changed by configuration or DHCP)


So one could use either or both, as they wish?
I guess I'm having a bit of hard time trying to figure out what the
canonical API call sequence would be to set up various useful
and standard NAT situations.  Yes, I've read the Wiki page; No it
isn't clear on the proper, expected sequence of the API calls.
This is another variation on the theme of VPP-API documentation (or lack 
thereof).  I recently added the saving of the api trace log and dump to 
the "make test" infrastructure specifically to be able to data mine the 
API sequences embedded in the test cases.


After running "make test" you will now find the binary api trace log 
(/tmp/vpp-unittest-TEST*/vpp_api_trace..log) and a dump of 
the api trace in the test log (search for "api trace" in 
/tmp/vpp-unittest-TEST*/log.txt). You can recreate the configuration 
using the vpp_api_trace..log and the custom-dump mechanism to 
re-create the configuration as described in 
https://wiki.fd.io/view/VPP/How_To_Use_The_API_Trace_Tools


While not perfect solution to the problem, it is the most pragmatic 
approach I could think of to address this issue. Hopefully this will 
help encourage feature developers to expand the number of use cases in 
the "make test" framework.  Of course, converting the api trace data 
into usable documentation is another task to be done.  Automating this 
task is probably the best way to attack this issue, otherwise the 
documentation will go stale over time.  Just a matter of a little more 
typing ;^)


How does those API sequences change for the different values of the config
variables "static_mapping_only" and "static_mapping_connection_tracking"?

And on that note -- How do you change the value of those config varaibles
at run time?  I know how to alter the config file and re-read them in VAT.
I'm not using VAT.  I'm writing a totally different system.  Can those 
values
be changed at run-time?  There is no API to do so, so at this point in 
time

the answer must be "no."  Is it expected that the sequence to change these
configuration values at run time is to: 1) Stop VPP, 2) write a new 
config file

with the new desired values, 3) restart VPP?  That seems bad to me.
I'll let Matus address the SNAT specifics, but in general appropriate 
VPP-API interfaces should be developed to allow dynamic configuration of 
VPP features.  That being said, it is less work to develop basic 
functionality using configuration variables prior to creating a 
comprehensive debug CLI + VPP-API interface.  Prototyping can be a good 
thing, but is no substitute for developing a production interface.



I think 1024 is not significant, it's just a warning that you add
a lot of addresses to SNAT address pool, it was here before I
started work on SNAT plugin.


Well, 1024 is arbitrary.  Who is to say that I don't need 2048 and 
don't care
about the warning in my environment?   To be clear, I'm not blaming 
you (Matus),
or anyone for that matter.  I'm merely pointing out that it is an 
arbitrary and

undocumented limit in the current system.
Yep.  At the very least this should be documented.  If there is a need 
to configure this, then the appropriate code should be added to do so.


Thanks,
-daw-


Matus


Thanks,
jdl


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] SNAT API Question

[Matus Fabian -X (matfabia - PANTHEON TECHNOLOGIES at Cisco)]

Hi Jon,

comments inline,


Matus

From: Jon Loeliger [mailto:j...@netgate.com]
Sent: Monday, February 20, 2017 9:14 PM
To: Matus Fabian -X (matfabia - PANTHEON TECHNOLOGIES at Cisco) 
<matfa...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] SNAT API Question

Hi Matus,

Thanks for your answers here.

On Sat, Feb 18, 2017 at 12:36 AM, Matus Fabian -X (matfabia - PANTHEON 
TECHNOLOGIES at Cisco) <matfa...@cisco.com<mailto:matfa...@cisco.com>> wrote:
If external_sw_if_index value is ~0 (-1) external_ip_address is ussed from API 
(snat.c line 363).

OK, I see that in the code, but it is nowhere described in the API itself.
That is my issue here.  Magic values like this *are* part of the API as they
will affect changes on the VPP-side of the API interface.

snat_add_address_range – add address range to SNAT address pool
snat_add_del_interface_addr – add address of the interface to SNAT address pool 
(address is added/removed automatically when interface address is changed by 
configuration or DHCP)

So one could use either or both, as they wish?
[mf:] yes you could use both

I guess I'm having a bit of hard time trying to figure out what the
canonical API call sequence would be to set up various useful
and standard NAT situations.  Yes, I've read the Wiki page; No it
isn't clear on the proper, expected sequence of the API calls.
[mf:] I can add something like this to wiki, please provide some example NAT 
situations

How does those API sequences change for the different values of the config
variables "static_mapping_only" and "static_mapping_connection_tracking"?
[mf:] in both cases only static mappings (1:1 NAT) are enabled, 
static_mapping_only do not create dynamic state data (no session data, so 
packets for specific user can be processed on each worker thread otherwise 
user’s traffic is processed always by same worker) just only translate packets 
based on configured static mappings, second mode create session data

And on that note -- How do you change the value of those config varaibles
at run time?  I know how to alter the config file and re-read them in VAT.
I'm not using VAT.  I'm writing a totally different system.  Can those values
be changed at run-time?  There is no API to do so, so at this point in time
the answer must be "no."  Is it expected that the sequence to change these
configuration values at run time is to: 1) Stop VPP, 2) write a new config file
with the new desired values, 3) restart VPP?  That seems bad to me.
[mf:] You can’t switch between SNAT modes at run-time. I think this is not 
something you need to change at runtime and at the moment there is no plan to 
do it at runtime using API.


I think 1024 is not significant, it's just a warning that you add a lot of 
addresses to SNAT address pool, it was here before I started work on SNAT 
plugin.

Well, 1024 is arbitrary.  Who is to say that I don't need 2048 and don't care
about the warning in my environment?   To be clear, I'm not blaming you (Matus),
or anyone for that matter.  I'm merely pointing out that it is an arbitrary and
undocumented limit in the current system.

Matus

Thanks,
jdl

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] SNAT API Question

[Jon Loeliger]

Hi Matus,

Thanks for your answers here.

On Sat, Feb 18, 2017 at 12:36 AM, Matus Fabian -X (matfabia - PANTHEON
TECHNOLOGIES at Cisco)  wrote:

> If external_sw_if_index value is ~0 (-1) external_ip_address is ussed from
> API (snat.c line 363).
>

OK, I see that in the code, but it is nowhere described in the API itself.
That is my issue here.  Magic values like this *are* part of the API as they
will affect changes on the VPP-side of the API interface.


> snat_add_address_range – add address range to SNAT address pool
>
> snat_add_del_interface_addr – add address of the interface to SNAT address
> pool (address is added/removed automatically when interface address is
> changed by configuration or DHCP)
>

So one could use either or both, as they wish?

I guess I'm having a bit of hard time trying to figure out what the
canonical API call sequence would be to set up various useful
and standard NAT situations.  Yes, I've read the Wiki page; No it
isn't clear on the proper, expected sequence of the API calls.

How does those API sequences change for the different values of the config
variables "static_mapping_only" and "static_mapping_connection_tracking"?

And on that note -- How do you change the value of those config varaibles
at run time?  I know how to alter the config file and re-read them in VAT.
I'm not using VAT.  I'm writing a totally different system.  Can those
values
be changed at run-time?  There is no API to do so, so at this point in time
the answer must be "no."  Is it expected that the sequence to change these
configuration values at run time is to: 1) Stop VPP, 2) write a new config
file
with the new desired values, 3) restart VPP?  That seems bad to me.


I think 1024 is not significant, it's just a warning that you add a lot of
> addresses to SNAT address pool, it was here before I started work on SNAT
> plugin.
>

Well, 1024 is arbitrary.  Who is to say that I don't need 2048 and don't
care
about the warning in my environment?   To be clear, I'm not blaming you
(Matus),
or anyone for that matter.  I'm merely pointing out that it is an arbitrary
and
undocumented limit in the current system.


> Matus
>

Thanks,
jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] SNAT API Question

[Jon Loeliger]

On Tue, Feb 14, 2017 at 11:52 PM, Matus Fabian -X (matfabia - PANTHEON
TECHNOLOGIES at Cisco)  wrote:

> Hi Jon,
>
>
>
> snat_static_mapping_dump list only static mappings with resolved outside
> address so snat_static_mapping_details doesn’t contain
> external_sw_if_index. But we missing read API for static mappigs with
> unresolved outside address, I will add those to snat_static_mapping_dump
> too.
>
>
>
> Thanks,
>
> Matus
>
>
>
How does the API handler know which of the external_ip_address
or the external_id_index to honor?  Is there an invalid value that
should be supplied in the other field when making the API call
to snat_add_static_mapping?  Or is the API user expected to
resolve the external_ip_address and supply its sw_if_index too?

Also, another question.  When placing a range of addresses on
an external interface for dynamic mappings, is the expected use
case to first use snat_add_address_range on the external IF,
then when they are all "added", issue a snat_add_del_interface_addr
to make it effective (add) or remove them (del)?

Finally, one last API implementation question.

In snat_test.c, we find this snippet of code:

static int api_snat_add_address_range (vat_main_t * vam)
{
...
start_host_order = clib_host_to_net_u32 (start_addr.as_u32);
end_host_order = clib_host_to_net_u32 (end_addr.as_u32);
...
count = (end_host_order - start_host_order) + 1;

if (count > 1024) {
errmsg ("%U - %U, %d addresses...\n",
  format_ip4_address, _addr,
  format_ip4_address, _addr,
  count);
}


Is that 1024 significant or arbitrary?  Is there some limit in the VPP
implementation that is being guarded here?  Or is that an arbitrary
UI-imposed limit?

Specifically, must other API users adhere to this limit?  If not is there
some other limit at work here?

I see under the covers that each address in this range is individually
added to some address vector.  And furthermore this vector DOES
appear to be hard limited to 1024 entries as well. (See snat.c,
vl_api_snat_add_address_range_t_handler() around line 820 or so.)

Again, arbitrary?  Or is the limit actually imposed by the FIB?

Thanks,
jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] SNAT API Question

[Jon Loeliger]

VPPers,

Somewhat recently, (commit 36532bda926f5255a323c9cac3144dd758a05667),
the external_sw_if_index was added to the SNAT API snat_add_static_mapping
message.

However, when dumping the static mappings with snat_static_mapping_dump
the corresponding snat_static_mapping_details does not contain the new
external_sw_if_index.

Is that an oversight, or is it not necessary?  Or am I confused again?

Thanks,
jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev