Re: your thoughts on a particualar ipfw action.

[Ian Smith]

On Fri, 12 Aug 2016 16:49:36 +1000, grenville armitage wrote:
 > On 08/12/2016 14:56, Julian Elischer wrote:
 > > On 11/08/2016 9:02 AM, Dr. Rolf Jansen wrote:
 > >>
 >  [...]
 > >>
 > >> I needed to change the name of the geoip tool, because GeoIP® is a
 > registered trademark of MaxMind, Inc., see www.maxmind.com. The name of the
 > tool is now 'ipup' = abbreviated form of  IP geo location table generation
 > and look- UP , that is without the boring middle part :-D

 > > Hmm I'd have gone for geotable. ipup sounds like a young dog produced by
 > > Apple.
 > > (wonder if one can change the name of a port)

:)  The Sony Aibo had the robotic dog genre pretty well covered.

 > +1 FWIW.

Portname is sysutils/ipdbtools which sounds ok, so it's only one program 
could be renamed (again).  The port is young enough that it wouldn't be 
likely to inconvenience anybody much - except Rolf! - at this stage.

cheers, Ian
___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[grenville armitage]

On 08/12/2016 14:56, Julian Elischer wrote:

On 11/08/2016 9:02 AM, Dr. Rolf Jansen wrote:



[...]


I needed to change the name of the geoip tool, because GeoIP® is a registered 
trademark of MaxMind, Inc., see www.maxmind.com. The name of the tool is now 
'ipup' = abbreviated form of  IP geo location table generation and look- UP , 
that is without the boring middle part :-D

Hmm I'd have gone for geotable. ipup sounds like a young dog produced by Apple.
(wonder if one can change the name of a port)


+1 FWIW.

cheers,
gja


___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Julian Elischer]

On 12/08/2016 8:20 AM, Dr. Rolf Jansen wrote:

Am 11.08.2016 um 14:20 schrieb Ian Smith :
On Thu, 11 Aug 2016 10:09:24 -0300, Dr. Rolf Jansen wrote:

Am 11.08.2016 um 08:06 schrieb Ian Smith :
On Wed, 10 Aug 2016 -0300, Dr. Rolf Jansen wrote:
...
...

I just submitted a PR asking to add the new port 'sysutils/ipdbtools'.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211744

Wonderful.

The port maintainers were really quick. The port has been accepted
and has been already committed.

So it has, on refreshing the page.  Smooth and fast.

Re __uint128_t I _guess_ there may be macro/s to do that maths for i386?

Yeah, I am exploring the options. Comparisons, addition and subtraction are 
working already, multiplication, division and remainder operations are a tad 
more difficult, I must leave this for some weekend.


...
A more tech-savvy article than ABC or other news media managed so far:
https://www.theguardian.com/australia-news/2016/aug/10/computer-says-no-australian-census-shambles-explanation-depends-on-who-you-ask

Well, I tend to believe that this has nothing to do with DoS attacks,

Some should have been expected, planned for, mitigation anticipated, as
well as expecting at least 5 times the legit connections/hr they tested
for, and as the guardian article pointed to, their DNS was screwed in
several ways: way too long TTL (can't move fast), hard-coded subdomain
in SSL cert (couldn't readily add load-sharing capacity?) and such.

But they admit the geo-blocking fell over - whether inline as firewall
or on another server fielding lookup requests not disclosed - but they
say that failure caused a/the/some router to fail (crash? explode? :)

Perhaps they did Geo-blocking in the way that I mentioned in the summary of the 
ipdbtool's manual to be a no-go:

...
Unfortunately, online database look-up is by far too slow for even think-
ing about being utilized on the firewall level, where IP packets need to
be processed in a microsecond time scale. Therefore, a locally maintained
IP Geo-location database is indispensable in the given respect.
...


IBM, FFS! but they'll point to govt specs and disclaim hardware failure
but still it's not great product endorsement for their SoftLayer Cloud.

Natural but non-professional reaction. My mother always told us, if you point
with your index finger to others, three fingers are pointing back to you.
So IBM not only failed technically but also the PR devision did a bad job.


I mean, of course it is DoS, but not caused by an attack. Exactly the
same happens every year on 30th of April between 17:00 and 24:00 on
the servers of the Federal Bureau of Finance here in Brazil. That is
the deadline for the online-submission of the annual tax declaration
of the Brazilian citizens. Seems that the bureaucrats all over the
world share the same deficiency of creative problem solving.

Seems it's a requirement for the job, world wide.  Creativity is scary,
but you think they could guess that ~8 million households in the eastern
timezone were going to have dinner then do their census within ~2 hours.

Of course they could not guess this, because public servants are trained
to assume that the normal citizen does not meet her/his obligations, and
for sure they were (are) prepared to send out 8 million penalty notices
in 24 hours.

Actually we have until mid September to lodge the information, but if you
forget who was at you rplace that evening (guests?) then it makes sense to
do it earlier rather than later.

Who in the bureaucrats hell told them to go with one deadline for
everybody? For the census in Australia, I would have told the
citizens that everybody got an individual deadline which is his or
her birthday in 2016 -- problem solved.


see above..  it's a 6 week window from memory.



That'd be great load-balancing .. shall I let them know? :)

Doesn't cost anything giving it a try, however, you could as well slap an
ox on his horn - same effect.

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"



___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Julian Elischer]

On 11/08/2016 9:02 AM, Dr. Rolf Jansen wrote:

Am 08.08.2016 um 18:46 schrieb Dr. Rolf Jansen :

I am almost finished with preparing the tools for geo-blocking and geo-routing 
at the firewall for submission to the FreeBSD ports.

I created a man file for the tools, see: https://cyclaero.github.io/ipdb/, and I added 
the recent suggestions on rule number/action code per country code, namely, I changed the 
formula for the x-flag to the suggestion of Ian (value = offset + ((C1 - 'A')*26 + (C2 - 
'A'))*10), and I added the idea of directly assigning a number to a country code in the 
argument for the t-flag ("CC=n:...").

Furthermore, I removed the divert filter daemon from the Makefile. The source 
is still on GitHub, though, and can be re-vamped if necessary.

Now I am going to prepare the Makefile for the port.

I just submitted a PR asking to add the new port 'sysutils/ipdbtools'.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211744

I needed to change the name of the geoip tool, because GeoIP® is a registered 
trademark of MaxMind, Inc., see www.maxmind.com. The name of the tool is now 
'ipup' = abbreviated form of  IP  geo location table generation and look- UP , 
that is without the boring middle part :-D
Hmm I'd have gone for geotable. ipup sounds like a young dog produced 
by Apple.

(wonder if one can change the name of a port)



Those, who used geoip already in some scripts, please excuse the inconvenience 
of needing to change the name.

With the great help of Julian, I was able to improve the man file and the 
latest version can be read online:

https://cyclaero.github.io/ipdb/

Best regards

Rolf

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"




___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

> Am 11.08.2016 um 14:20 schrieb Ian Smith :
> On Thu, 11 Aug 2016 10:09:24 -0300, Dr. Rolf Jansen wrote:
>>> Am 11.08.2016 um 08:06 schrieb Ian Smith :
>>> On Wed, 10 Aug 2016 -0300, Dr. Rolf Jansen wrote:
>>> ...
>>> ...
 I just submitted a PR asking to add the new port 'sysutils/ipdbtools'.
 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211744
>>> 
>>> Wonderful.
>> 
>> The port maintainers were really quick. The port has been accepted 
>> and has been already committed.
> 
> So it has, on refreshing the page.  Smooth and fast.
> 
> Re __uint128_t I _guess_ there may be macro/s to do that maths for i386?

Yeah, I am exploring the options. Comparisons, addition and subtraction are 
working already, multiplication, division and remainder operations are a tad 
more difficult, I must leave this for some weekend.

>>> ...
>>> A more tech-savvy article than ABC or other news media managed so far:
>>> https://www.theguardian.com/australia-news/2016/aug/10/computer-says-no-australian-census-shambles-explanation-depends-on-who-you-ask
>> 
>> Well, I tend to believe that this has nothing to do with DoS attacks, 
> 
> Some should have been expected, planned for, mitigation anticipated, as 
> well as expecting at least 5 times the legit connections/hr they tested 
> for, and as the guardian article pointed to, their DNS was screwed in 
> several ways: way too long TTL (can't move fast), hard-coded subdomain 
> in SSL cert (couldn't readily add load-sharing capacity?) and such.
> 
> But they admit the geo-blocking fell over - whether inline as firewall 
> or on another server fielding lookup requests not disclosed - but they 
> say that failure caused a/the/some router to fail (crash? explode? :)

Perhaps they did Geo-blocking in the way that I mentioned in the summary of the 
ipdbtool's manual to be a no-go:

...
Unfortunately, online database look-up is by far too slow for even think-
ing about being utilized on the firewall level, where IP packets need to
be processed in a microsecond time scale. Therefore, a locally maintained
IP Geo-location database is indispensable in the given respect.
...

> IBM, FFS! but they'll point to govt specs and disclaim hardware failure 
> but still it's not great product endorsement for their SoftLayer Cloud.

Natural but non-professional reaction. My mother always told us, if you point
with your index finger to others, three fingers are pointing back to you.
So IBM not only failed technically but also the PR devision did a bad job. 

>> I mean, of course it is DoS, but not caused by an attack. Exactly the 
>> same happens every year on 30th of April between 17:00 and 24:00 on 
>> the servers of the Federal Bureau of Finance here in Brazil. That is 
>> the deadline for the online-submission of the annual tax declaration 
>> of the Brazilian citizens. Seems that the bureaucrats all over the 
>> world share the same deficiency of creative problem solving.
> 
> Seems it's a requirement for the job, world wide.  Creativity is scary, 
> but you think they could guess that ~8 million households in the eastern 
> timezone were going to have dinner then do their census within ~2 hours.

Of course they could not guess this, because public servants are trained
to assume that the normal citizen does not meet her/his obligations, and
for sure they were (are) prepared to send out 8 million penalty notices
in 24 hours.

>> Who in the bureaucrats hell told them to go with one deadline for 
>> everybody? For the census in Australia, I would have told the 
>> citizens that everybody got an individual deadline which is his or 
>> her birthday in 2016 -- problem solved.
> 
> That'd be great load-balancing .. shall I let them know? :)

Doesn't cost anything giving it a try, however, you could as well slap an
ox on his horn - same effect.

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Ian Smith]

On Thu, 11 Aug 2016 10:09:24 -0300, Dr. Rolf Jansen wrote:
 > > Am 11.08.2016 um 08:06 schrieb Ian Smith :
 > > On Wed, 10 Aug 2016 -0300, Dr. Rolf Jansen wrote:
 > > 
 > > (just curious: whereabouts is -0300?  Brazil?)
 > 
 > Yes, I am a German living in Brazil for more than 10 years now. BTW, 
 > your mail provider is blocking my mails, perhaps, because the origin 
 > is Brazil, but I am using a German provider for my mail transport.

Oops.  You should have mail from smithi@someisp about sorting that out? 
Cutting to recent:

 > > Terrific work, Rolf!  Something for everyone, although I'm guessing the 
 > > pf people are going to want a piece of the action, if they need any more 
 > > than the -p option and a bit of scripting.
 > 
 > It is not that much work, to add other output options. The main 
 > obstacle for me is, that I won't be able to test it carefully 
 > together with pf. So, it would be good to do this in cooperation with 
 > someone who got a well running pf firewall -- the same holds for 
 > other possible applications as well.

Indeed.  Once again I've suggested something I can't help with and know 
next to nothing about :)

 > >> I just submitted a PR asking to add the new port 'sysutils/ipdbtools'.
 > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211744
 > > 
 > > Wonderful.
 > 
 > The port maintainers were really quick. The port has been accepted 
 > and has been already committed.

So it has, on refreshing the page.  Smooth and fast.

Re __uint128_t I _guess_ there may be macro/s to do that maths for i386?

 > >> With the great help of Julian, I was able to improve the man file and
 > >> the latest version can be read online:
 > >> 
 > >>  https://cyclaero.github.io/ipdb/
 > > 
 > > Nice manual and all.  A few typos noted below (niggly Virgo proofreader)
 > 
 > I was tempted to get these last changes into my PR, but I am sorry, 

Not at all; nothing that might confuse or deter anybody .. niggles.

 > it was too late for the initial release. I committed the corrected 
 > man file to the GitHub repository, though, it will automatically go 
 > into the next release of the ipdbtools, perhaps together with some 
 > additions for using it together with pf(8) and route(8).

Great.  Looking forward to having a play, albeit on a box not running 
any external services currently, to scope it out.

 > Nothing, to be sorry about. I like discussions.

Ok, no sorrow either way ..

 > > As a hopefully not unwelcome aside, it's a pity that IBM, of all people, 
 > > couldn't manage geo-blocking successfully for the Australian Census the 
 > > other night.  Next time around we can offer them a working geo-blocking 
 > > firewall/router for a good deal less than the AU$9.6M we've paid IBM :)
 > > 
 > > Census: How the Government says the website meltdown unfolded:
 > > http://www.abc.net.au/news/2016-08-10/census-night-how-the-shambles-unfolded/7712964
 > > 
 > > A more tech-savvy article than ABC or other news media managed so far:
 > > https://www.theguardian.com/australia-news/2016/aug/10/computer-says-no-australian-census-shambles-explanation-depends-on-who-you-ask
 > 
 > Well, I tend to believe that this has nothing to do with DoS attacks, 

Some should have been expected, planned for, mitigation anticipated, as 
well as expecting at least 5 times the legit connections/hr they tested 
for, and as the guardian article pointed to, their DNS was screwed in 
several ways: way too long TTL (can't move fast), hard-coded subdomain 
in SSL cert (couldn't readily add load-sharing capacity?) and such.

But they admit the geo-blocking fell over - whether inline as firewall 
or on another server fielding lookup requests not disclosed - but they 
say that failure caused a/the/some router to fail (crash? explode? :)

IBM, FFS! but they'll point to govt specs and disclaim hardware failure 
but still it's not great product endorsement for their SoftLayer Cloud.

 > I mean, of course it is DoS, but not caused by an attack. Exactly the 
 > same happens every year on 30th of April between 17:00 and 24:00 on 
 > the servers of the Federal Bureau of Finance here in Brazil. That is 
 > the deadline for the online-submission of the annual tax declaration 
 > of the Brazilian citizens. Seems that the bureaucrats all over the 
 > world share the same deficiency of creative problem solving.

Seems it's a requirement for the job, world wide.  Creativity is scary, 
but you think they could guess that ~8 million households in the eastern 
timezone were going to have dinner then do their census within ~2 hours.

 > Who in the bureaucrats hell told them to go with one deadline for 
 > everybody? For the census in Australia, I would have told the 
 > citizens that everybody got an individual deadline which is his or 
 > her birthday in 2016 -- problem solved.

That'd be great load-balancing .. shall I let them know? :)

 > > It's not quite clear how to specify an 'empty CC list'? ''? ""? either?
 > 
 > 

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

> Am 11.08.2016 um 08:06 schrieb Ian Smith :
> On Wed, 10 Aug 2016 -0300, Dr. Rolf Jansen wrote:
> 
> (just curious: whereabouts is -0300?  Brazil?)

Yes, I am a German living in Brazil for more than 10 years now. BTW, your mail 
provider is blocking my mails, perhaps, because the origin is Brazil, but I am 
using a German provider for my mail transport.

>>> Am 08.08.2016 um 18:46 schrieb Dr. Rolf Jansen :
>>> I am almost finished with preparing the tools for geo-blocking and 
>>> geo-routing at the firewall for submission to the FreeBSD ports.
> 
>>> I created a man file for the tools, see: 
>>> https://cyclaero.github.io/ipdb/, and I added the recent suggestions 
>>> on rule number/action code per country code, namely, I changed the 
>>> formula for the x-flag to the suggestion of Ian (value = offset + 
>>> ((C1 - 'A')*26 + (C2 - 'A'))*10), and I added the idea of directly 
>>> assigning a number to a country code in the argument for the t-flag 
>>> ("CC=n:...").  Furthermore, I removed the divert filter daemon 
>>> from the Makefile. The source is still on GitHub, though, and can be 
>>> re-vamped if necessary. Now I am going to prepare the Makefile for
>>> the port.
> 
> Terrific work, Rolf!  Something for everyone, although I'm guessing the 
> pf people are going to want a piece of the action, if they need any more 
> than the -p option and a bit of scripting.

It is not that much work, to add other output options. The main obstacle for me 
is, that I won't be able to test it carefully together with pf. So, it would be 
good to do this in cooperation with someone who got a well running pf firewall 
-- the same holds for other possible applications as well.

>> I just submitted a PR asking to add the new port 'sysutils/ipdbtools'.
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211744
> 
> Wonderful.

The port maintainers were really quick. The port has been accepted and has been 
already committed.

>> I needed to change the name of the geoip tool, because GeoIP® is a
>> registered trademark of MaxMind, Inc., see www.maxmind.com. The name 
> 
> I did wonder about that ..
> 
>> of the tool is now 'ipup' = abbreviated form of IP geo location table 
>> generation and look- UP , that is without the boring middle part :-D
>> 
>> Those, who used geoip already in some scripts, please excuse the
>> inconvenience of needing to change the name.
> 
>> With the great help of Julian, I was able to improve the man file and
>> the latest version can be read online:
>> 
>>  https://cyclaero.github.io/ipdb/
> 
> Nice manual and all.  A few typos noted below (niggly Virgo proofreader)

I was tempted to get these last changes into my PR, but I am sorry, it was too 
late for the initial release. I committed the corrected man file to the GitHub 
repository, though, it will automatically go into the next release of the 
ipdbtools, perhaps together with some additions for using it together with 
pf(8) and route(8).

> I must apologise for added exasperation earlier.  I was tending towards 
> conflating several other ipfw issues under discussion (named states, new 
> state actions, and this).  Sorry if I bumped you off course momentarily, 
> though I don't seem to have slowed you down too much ..

Nothing, to be sorry about. I like discussions.

> As a hopefully not unwelcome aside, it's a pity that IBM, of all people, 
> couldn't manage geo-blocking successfully for the Australian Census the 
> other night.  Next time around we can offer them a working geo-blocking 
> firewall/router for a good deal less than the AU$9.6M we've paid IBM :)
> 
> Census: How the Government says the website meltdown unfolded:
> http://www.abc.net.au/news/2016-08-10/census-night-how-the-shambles-unfolded/7712964
> 
> A more tech-savvy article than ABC or other news media managed so far:
> https://www.theguardian.com/australia-news/2016/aug/10/computer-says-no-australian-census-shambles-explanation-depends-on-who-you-ask

Well, I tend to believe that this has nothing to do with DoS attacks, I mean, 
of course it is DoS, but not caused by an attack. Exactly the same happens 
every year on 30th of April between 17:00 and 24:00 on the servers of the 
Federal Bureau of Finance here in Brazil. That is the deadline for the 
online-submission of the annual tax declaration of the Brazilian citizens. 
Seems that the bureaucrats all over the world share the same deficiency of 
creative problem solving.

Who in the bureaucrats hell told them to go with one deadline for everybody? 
For the census in Australia, I would have told the citizens that everybody got 
an individual deadline which is his or her birthday in 2016 -- problem solved.

> ===
> 
> It is suitable for inclusion into cron.  "for invocation by cron" maybe?

OK, "invocation by" sounds better (for me)

> ipdb_update.sh has IPRanges="/usr/local/etc/ipdb/IPRanges" but some (not 
> all) mentions in the manpage use "IP-Ranges" with a hyphen, 

Re: your thoughts on a particualar ipfw action.

[Ian Smith]

On Wed, 10 Aug 2016 -0300, Dr. Rolf Jansen wrote:

(just curious: whereabouts is -0300?  Brazil?)

 > > Am 08.08.2016 um 18:46 schrieb Dr. Rolf Jansen :
>> I am almost finished with preparing the tools for geo-blocking and 
>> geo-routing at the firewall for submission to the FreeBSD ports.

>> I created a man file for the tools, see: 
>> https://cyclaero.github.io/ipdb/, and I added the recent suggestions 
>> on rule number/action code per country code, namely, I changed the 
>> formula for the x-flag to the suggestion of Ian (value = offset + 
>> ((C1 - 'A')*26 + (C2 - 'A'))*10), and I added the idea of directly 
>> assigning a number to a country code in the argument for the t-flag 
>> ("CC=n:...").  Furthermore, I removed the divert filter daemon 
>> from the Makefile. The source is still on GitHub, though, and can be 
>> re-vamped if necessary. Now I am going to prepare the Makefile for
>> the port.

Terrific work, Rolf!  Something for everyone, although I'm guessing the 
pf people are going to want a piece of the action, if they need any more 
than the -p option and a bit of scripting.

 > I just submitted a PR asking to add the new port 'sysutils/ipdbtools'.
 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211744

Wonderful.

 > I needed to change the name of the geoip tool, because GeoIP® is a
 > registered trademark of MaxMind, Inc., see www.maxmind.com. The name 

I did wonder about that ..

 > of the tool is now 'ipup' = abbreviated form of IP geo location table 
 > generation and look- UP , that is without the boring middle part :-D
 >
 > Those, who used geoip already in some scripts, please excuse the
 > inconvenience of needing to change the name.

 > With the great help of Julian, I was able to improve the man file and
 > the latest version can be read online:
 >
 >   https://cyclaero.github.io/ipdb/

Nice manual and all.  A few typos noted below (niggly Virgo proofreader)

I must apologise for added exasperation earlier.  I was tending towards 
conflating several other ipfw issues under discussion (named states, new 
state actions, and this).  Sorry if I bumped you off course momentarily, 
though I don't seem to have slowed you down too much ..

As a hopefully not unwelcome aside, it's a pity that IBM, of all people, 
couldn't manage geo-blocking successfully for the Australian Census the 
other night.  Next time around we can offer them a working geo-blocking 
firewall/router for a good deal less than the AU$9.6M we've paid IBM :)

Census: How the Government says the website meltdown unfolded:
http://www.abc.net.au/news/2016-08-10/census-night-how-the-shambles-unfolded/7712964

A more tech-savvy article than ABC or other news media managed so far:
https://www.theguardian.com/australia-news/2016/aug/10/computer-says-no-australian-census-shambles-explanation-depends-on-who-you-ask

cheers, Ian

===

It is suitable for inclusion into cron.  "for invocation by cron" maybe?

ipdb_update.sh has IPRanges="/usr/local/etc/ipdb/IPRanges" but some (not 
all) mentions in the manpage use "IP-Ranges" with a hyphen, including 
the FILES section.  Also the last one there repeats "*bst.v4" for IPv6.

It's not quite clear how to specify an 'empty CC list'? ''? ""? either?

"from certain [countries?] we don't like .."

"piped into sort of [or?] a pre-processing command .."

===
___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

> Am 08.08.2016 um 18:46 schrieb Dr. Rolf Jansen :
> 
> I am almost finished with preparing the tools for geo-blocking and 
> geo-routing at the firewall for submission to the FreeBSD ports.
> 
> I created a man file for the tools, see: https://cyclaero.github.io/ipdb/, 
> and I added the recent suggestions on rule number/action code per country 
> code, namely, I changed the formula for the x-flag to the suggestion of Ian 
> (value = offset + ((C1 - 'A')*26 + (C2 - 'A'))*10), and I added the idea of 
> directly assigning a number to a country code in the argument for the t-flag 
> ("CC=n:...").
> 
> Furthermore, I removed the divert filter daemon from the Makefile. The source 
> is still on GitHub, though, and can be re-vamped if necessary.
> 
> Now I am going to prepare the Makefile for the port.

I just submitted a PR asking to add the new port 'sysutils/ipdbtools'.

   https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211744

I needed to change the name of the geoip tool, because GeoIP® is a registered 
trademark of MaxMind, Inc., see www.maxmind.com. The name of the tool is now 
'ipup' = abbreviated form of  IP  geo location table generation and look- UP , 
that is without the boring middle part :-D

Those, who used geoip already in some scripts, please excuse the inconvenience 
of needing to change the name.

With the great help of Julian, I was able to improve the man file and the 
latest version can be read online:

   https://cyclaero.github.io/ipdb/

Best regards

Rolf

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

I am almost finished with preparing the tools for geo-blocking and geo-routing 
at the firewall for submission to the FreeBSD ports.

I created a man file for the tools, see: https://cyclaero.github.io/ipdb/, and 
I added the recent suggestions on rule number/action code per country code, 
namely, I changed the formula for the x-flag to the suggestion of Ian (value = 
offset + ((C1 - 'A')*26 + (C2 - 'A'))*10), and I added the idea of directly 
assigning a number to a country code in the argument for the t-flag 
("CC=n:...").

Furthermore, I removed the divert filter daemon from the Makefile. The source 
is still on GitHub, though, and can be re-vamped if necessary.

Now I am going to prepare the Makefile for the port.

In the meantime, please can a native English speaker look at said man file (s. 
link above)? I know, that my English is lacking, and any corrections would be 
highly welcome.

Best regards

Rolf

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

> Am 05.08.2016 um 02:44 schrieb Julian Elischer :
> On 5/08/2016 2:22 AM, Dr. Rolf Jansen wrote:
>> I am completely free of passions on this CC encoding thingy. I won't use 
>> this feature anyway. Please, may I suggest that the experts of the ipfw 
>> community come to an agreement, and I then I will change the implementation 
>> accordingly.
>> 
>> Another possibility could be to attach the desired rule numbers directly to 
>> the country codes in the argument of the -t option, How about:
>> 
>>   geoip -t AU=5:RU=50010:US=50020:BR=50030
>> 
>> The present behaviour would be kept without attached numbers. Please let me 
>> know your choices. Furthermore, if the new ipfw allows for more 
>> sophisticated table construction directives, that could be beneficial for 
>> country code based table processing, please advice.
> 
> I can hear the exasperation in your writing :-)

Not exactly 'exasperation'. Moving targets are always kind of unpleasant - at 
least for me, perhaps I am not a sufficiently patient hunter :-)

> I've lost track..
> Was the present behaviour just a single value? or a generated number with -x 
> offset? (not sure if you actually added that or just described it).

I meant, geoip would continue allowing a -t option argument without numbers, 
for example -t AU:RU:US:BR, and in that case it would continue with its present 
behaviour (mutually exclusive either -v XOR -x):

-v  # default 0
-x   # in val = (C1-60)*1000 + C2*10 + offset

The -v and -x options as above are already on GitHub, the x-formula can be 
changed quickly. 

> your "US=50020" idea is nice but a lot of work I think for  you.

Not that much work. I like this one as well, because this is the most explicit 
way, that I can imagine, of associating a rule/action number with a country 
code.

I figure, that any kind of x-formula let people shoot themselves in the foot at 
one day or the other. Imagine you set up your sophisticated rule set in 2016 
and in 2017 a colleague is asked by the boss to add another country code. The 
trouble may start by she/he forgets to add an action rule for the implicitly 
generated table argument, and it does not end with a possible violation of the 
implicitly reserved rule number range.

> I guess you would do it with script
> geoip -t US=${LINE_US} |ipfw -q /dev/stdin
> ipfw add ${LINE_US} drop all tcp from any to any 80
> ipfw add $((${LINE_US} + 1)) skipto ${FINISH_UP}

Yes, why not? The nice thing of the "CC=n:..." feature is, that it is 
already useful as is, and that it is open for any further sophistication with 
shell script magics.  

> probably in a shell function
> it would also allow you to put 'action numbers' rather than line numbers as 
> it doesn't  interpret the values, just passes them through.
> 
> On the other hand the same thing can be achieved by embedding geoip in a loop 
> in a script.
> I think we should just let you get on with your life and be happy with what 
> you have given us.  mapping a set of country codes to a number. I can always 
> make more complicated setups using that and 15 minutes of shell script.

Yes, for us this is not a big deal. However, once this stuff is in the ports, I 
have to take into account the work of answering questions from the 
non-enlightened folks, on how to achieve the mapping between rule numbers and 
country codes. Perhaps, it is less hassle to simply add the "CC=n:..." 
feature and move on.


BTW:

In the course of preparing the packet for the ports, I am working now on a man 
file for the geoip db utilities (geoip, ipdb and ipdb-update.sh). I want to put 
it to section 1 - General Commands, OK?

Shall I remove the geod ipfw divert filter daemon from the distribution for the 
ports?

My initial incentive was Geo-blocking. However, I learned from your VPN usage 
example (in your other message), that these utilities may quite well serve for 
other objectives. Now, I am looking for a package title that does suggest a 
wider range of applications.

How about:

  "Utilities for IP based Geo-blocking/routing with IPFW"


Best regards

Rolf

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Julian Elischer]

On 5/08/2016 12:15 PM, Michael Sierchio wrote:

Wouldn't it make sense to use the ISO Numeric Code / UN M49 Numerical Code?
actually it doesn't make sense. the source of data doesn't have that 
information in it
so it would require a whole layer of mapping, including downloads. and 
it would have to

cope with unexpected ambiguities and mismatches.


___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Michael Sierchio]

Wouldn't it make sense to use the ISO Numeric Code / UN M49 Numerical Code?


On Thu, Aug 4, 2016 at 9:00 PM, Julian Elischer  wrote:

> On 5/08/2016 12:44 AM, Ian Smith wrote:
>
>> On Wed, 3 Aug 2016 18:53:38 -0300, Dr. Rolf Jansen wrote:
>>   > > Am 03.08.2016 um 11:13 schrieb Julian Elischer > >:
>>
>>> On 2/08/2016 8:50 PM, Dr. Rolf Jansen wrote:
>>>
 Am 02.08.2016 um 05:08 schrieb Julian Elischer :
>
 'scuse savage reformatting, but I had to wrap it to read it .. and pine
>> has completely mangled the quoting levels too, dunno why.
>>
>> looking for thoughts from people who know the new IPFW features well..
>
 That's not me, but I'm having fun reading 11.0-RELEASE ipfw(8) ..
>>
>> A recent addition to our armory is the geoip program that, given an
> address can tell you what country it is in and given a country code,
> can give an ipfw table that describes all the ip addresses in that
> country.
> SO I was thinking how to use this, and the obvious way would be to
> have a set of rules for each country, and use the "skipto tablearg"
> facility to skip to the right rules for each country. But the
> trouble is that a tablearg skipto is very inefficient. It's also a
> hard thing to set up with a set of rules for each country (how many
> countries are there in the internet allocation system?).
>
 Julian, have you looked into Andrey's LINEAR_SKIPTO ?  How does it work?
>> Are there any disadvantages?  And if not, why isn't it the default? :)
>>
> no, until he mentioned i I was unaware of it..
> I will be looking at it as soon as 1 have time.
>
>>
>> Also, There's a particularly useful example in new ipfw(8), showing how
>> to set and use multiple tablearg values - the example uses skipto,fib
>> with a setfib tablearg followed by a skipto tablearg both from the same
>> table entry, and you can use, among others - some fully documented, some
>> yet to catch up - dscp values (0..63) setting or testing, and notably
>> tags 1..65534, set or test, which goes some way towards 'variables' you
>> were hoping for, no? :)  Also some netgraph stuff I won't understand ..
>>
> tags "almost" do variables but  they stick on the packet after it leaves
> ipfw and may cause misinterpretation if the packet enters ipfw a second
> time. It's a question of scope.
>
>
>
>> As of today a total of 236 country codes are in use for IPv4

>>> delegations. If this helps for anything, a command line switch to the
>> geoip tool could be added for letting it output the country code (as the
>> hex encoded CC taken as a plain decimal integer) as the value for the
>> given table entry. In the moment you can give one value for all entries
>> generated by geoip, with this switch set, the output of geoip could look
>> like:
>>
>> $ geoip -t "DE:BR:US" -x
 ...
 table 0 add 93.157.48.0/21 4445
 table 0 add 93.158.236.0/22 4252
 table 0 add 93.159.96.0/19 4445
 table 0 add 93.159.248.0/21 4445
 table 0 add 93.180.72.0/21 4445
 table 0 add 93.180.152.0/21 4445
 table 0 add 93.181.0.0/18 4445
 table 0 add 93.183.0.0/18 5553
 ...

 Given that ...
 0x4445 = 'DE'
 0x4252 = 'BR'
 0x5553 = 'US'

>>> well it would have to be the decimal value so DE would be 6869, as
>>> while 4445 works 444F is a problem :-)
>>>
>> RJ> Yes, you're right, I was taken away by the wave of enthusiasm,
>> RJ> before thinking about the subject up to the end.
>>
>> 0x444F would work but we waste even more address space.  You'd have to
>>>
>> multiply the numbers by some scaler, because adjacent numbers would not
>> be enough for one rule to do something and another rule to skip on to
>> somewhere else. (well, you could have multiple rules at the same number
>> but that has its limitations. > The idea would certainly work. it would
>> mean setting aside all the rules between 6565 and 9090 however. > A more
>> compact encoding could be something like
>>
>> ((name[0]-'A') * 32)+(name[1]-'A')) multiplied by some 'step' (maybe
>>> 10 by default) and offset by a given offset.
>>> so AF (Afghanistan) would be the first 0*32+5 * 10 would give an
>>> offset of 50.. plus a user supplied offset turns it into say, 15050..
>>>
>> RJ> I understand the reasons, however, any complicated encoding will
>> defeat the idea of the value can be sort of numeric mnemonic for the
>> country code ÿÿ well, so it is. I elaborated on your idea and came-up
>> with the following formula:  val = (C1-60)*1000 + C2*10 + offset. The
>> offset can be given as the parameter to the -x flag.
>>
>> $ geoip -t "DE:BR:US" -4 -x 3
>> ...
>> table 0 add 93.157.48.0/21 38690
>> table 0 add 93.158.236.0/22 36820
>> table 0 add 93.159.96.0/19 38690
>> table 0 add 93.159.248.0/21 38690
>> table 0 add 93.180.72.0/21 38690
>> table 0 add 93.180.152.0/21 38690
>> table 0 add 93.181.0.0/18 38690
>> table 0 add 93.183.0.0/18 55830
>> ...

Re: your thoughts on a particualar ipfw action.

[Julian Elischer]

On 5/08/2016 12:44 AM, Ian Smith wrote:

On Wed, 3 Aug 2016 18:53:38 -0300, Dr. Rolf Jansen wrote:
  > > Am 03.08.2016 um 11:13 schrieb Julian Elischer :

On 2/08/2016 8:50 PM, Dr. Rolf Jansen wrote:

Am 02.08.2016 um 05:08 schrieb Julian Elischer :

'scuse savage reformatting, but I had to wrap it to read it .. and pine
has completely mangled the quoting levels too, dunno why.


looking for thoughts from people who know the new IPFW features well..

That's not me, but I'm having fun reading 11.0-RELEASE ipfw(8) ..


A recent addition to our armory is the geoip program that, given an
address can tell you what country it is in and given a country code,
can give an ipfw table that describes all the ip addresses in that
country.
SO I was thinking how to use this, and the obvious way would be to
have a set of rules for each country, and use the "skipto tablearg"
facility to skip to the right rules for each country. But the
trouble is that a tablearg skipto is very inefficient. It's also a
hard thing to set up with a set of rules for each country (how many
countries are there in the internet allocation system?).

Julian, have you looked into Andrey's LINEAR_SKIPTO ?  How does it work?
Are there any disadvantages?  And if not, why isn't it the default? :)

no, until he mentioned i I was unaware of it..
I will be looking at it as soon as 1 have time.


Also, There's a particularly useful example in new ipfw(8), showing how
to set and use multiple tablearg values - the example uses skipto,fib
with a setfib tablearg followed by a skipto tablearg both from the same
table entry, and you can use, among others - some fully documented, some
yet to catch up - dscp values (0..63) setting or testing, and notably
tags 1..65534, set or test, which goes some way towards 'variables' you
were hoping for, no? :)  Also some netgraph stuff I won't understand ..
tags "almost" do variables but  they stick on the packet after it 
leaves ipfw and may cause misinterpretation if the packet enters ipfw 
a second time. It's a question of scope.





As of today a total of 236 country codes are in use for IPv4

delegations. If this helps for anything, a command line switch to the
geoip tool could be added for letting it output the country code (as the
hex encoded CC taken as a plain decimal integer) as the value for the
given table entry. In the moment you can give one value for all entries
generated by geoip, with this switch set, the output of geoip could look
like:


$ geoip -t "DE:BR:US" -x
...
table 0 add 93.157.48.0/21 4445
table 0 add 93.158.236.0/22 4252
table 0 add 93.159.96.0/19 4445
table 0 add 93.159.248.0/21 4445
table 0 add 93.180.72.0/21 4445
table 0 add 93.180.152.0/21 4445
table 0 add 93.181.0.0/18 4445
table 0 add 93.183.0.0/18 5553
...

Given that ...
0x4445 = 'DE'
0x4252 = 'BR'
0x5553 = 'US'

well it would have to be the decimal value so DE would be 6869, as
while 4445 works 444F is a problem :-)

RJ> Yes, you're right, I was taken away by the wave of enthusiasm,
RJ> before thinking about the subject up to the end.


0x444F would work but we waste even more address space.  You'd have to

multiply the numbers by some scaler, because adjacent numbers would not
be enough for one rule to do something and another rule to skip on to
somewhere else. (well, you could have multiple rules at the same number
but that has its limitations. > The idea would certainly work. it would
mean setting aside all the rules between 6565 and 9090 however. > A more
compact encoding could be something like


((name[0]-'A') * 32)+(name[1]-'A')) multiplied by some 'step' (maybe
10 by default) and offset by a given offset.
so AF (Afghanistan) would be the first 0*32+5 * 10 would give an
offset of 50.. plus a user supplied offset turns it into say, 15050..

RJ> I understand the reasons, however, any complicated encoding will
defeat the idea of the value can be sort of numeric mnemonic for the
country code ÿÿ well, so it is. I elaborated on your idea and came-up
with the following formula:  val = (C1-60)*1000 + C2*10 + offset. The
offset can be given as the parameter to the -x flag.

$ geoip -t "DE:BR:US" -4 -x 3
...
table 0 add 93.157.48.0/21 38690
table 0 add 93.158.236.0/22 36820
table 0 add 93.159.96.0/19 38690
table 0 add 93.159.248.0/21 38690
table 0 add 93.180.72.0/21 38690
table 0 add 93.180.152.0/21 38690
table 0 add 93.181.0.0/18 38690
table 0 add 93.183.0.0/18 55830
...

The limits (without offset) are:
AA = 5650  -- actually AD = 5680
ZZ = 30900

RJ> With this formula, the offset must be less than 34735. Although,
this wastes a considerable amount of rule number space, the advantage is
that the numbers are still accessible by mental arithmetic, and the
other constraint of having a step > 1 is fulfilled as well. Anyway, this
one is not graved in stone, we can agree on another one.

Sorry, but that encoding takes up way too much (perhaps precious) rule
space for one function, and I really can't 

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

> Am 04.08.2016 um 13:44 schrieb Ian Smith :
>> On Wed, 3 Aug 2016 18:53:38 -0300, Dr. Rolf Jansen wrote:
 Am 03.08.2016 um 11:13 schrieb Julian Elischer :
> 
> 'scuse savage reformatting, but I had to wrap it to read it .. and pine 
> has completely mangled the quoting levels too, dunno why.
> 
> looking for thoughts from people who know the new IPFW features well..
> 
> That's not me, but I'm having fun reading 11.0-RELEASE ipfw(8) ..
> 
> A recent addition to our armory is the geoip program that, given an 
> address can tell you what country it is in and given a country code, 
> can give an ipfw table that describes all the ip addresses in that 
> country.
> 
> SO I was thinking how to use this, and the obvious way would be to 
> have a set of rules for each country, and use the "skipto tablearg" 
> facility to skip to the right rules for each country. But the
> trouble is that a tablearg skipto is very inefficient. It's also a 
> hard thing to set up with a set of rules for each country (how many 
> countries are there in the internet allocation system?).
> 
> Julian, have you looked into Andrey's LINEAR_SKIPTO ?  How does it work?  
> Are there any disadvantages?  And if not, why isn't it the default? :)
> 
> Also, There's a particularly useful example in new ipfw(8), showing how 
> to set and use multiple tablearg values - the example uses skipto,fib 
> with a setfib tablearg followed by a skipto tablearg both from the same 
> table entry, and you can use, among others - some fully documented, some 
> yet to catch up - dscp values (0..63) setting or testing, and notably 
> tags 1..65534, set or test, which goes some way towards 'variables' you 
> were hoping for, no? :)  Also some netgraph stuff I won't understand ..
> 
 As of today a total of 236 country codes are in use for IPv4 
 delegations. If this helps for anything, a command line switch to the 
 geoip tool could be added for letting it output the country code (as the 
 hex encoded CC taken as a plain decimal integer) as the value for the 
 given table entry. In the moment you can give one value for all entries 
 generated by geoip, with this switch set, the output of geoip could look 
 like:
 
 $ geoip -t "DE:BR:US" -x
 ...
 table 0 add 93.157.48.0/21 4445
 table 0 add 93.158.236.0/22 4252
 table 0 add 93.159.96.0/19 4445
 table 0 add 93.159.248.0/21 4445
 table 0 add 93.180.72.0/21 4445
 table 0 add 93.180.152.0/21 4445
 table 0 add 93.181.0.0/18 4445
 table 0 add 93.183.0.0/18 5553
 ...
 
 Given that ...
 0x4445 = 'DE'
 0x4252 = 'BR'
 0x5553 = 'US'
>>> 
>>> well it would have to be the decimal value so DE would be 6869, as 
>>> while 4445 works 444F is a problem :-)
>> 
>> Yes, you're right, I was taken away by the wave of enthusiasm, 
>> before thinking about the subject up to the end.
>> 
>>> 0x444F would work but we waste even more address space.  You'd have to 
>>> multiply the numbers by some scaler, because adjacent numbers would not 
>>> be enough for one rule to do something and another rule to skip on to 
>>> somewhere else. (well, you could have multiple rules at the same number 
>>> but that has its limitations. > The idea would certainly work. it would 
>>> mean setting aside all the rules between 6565 and 9090 however. > A more 
>>> compact encoding could be something like
>>> 
>>> ((name[0]-'A') * 32)+(name[1]-'A')) multiplied by some 'step' (maybe 
>>> 10 by default) and offset by a given offset.
>>> 
>>> so AF (Afghanistan) would be the first 0*32+5 * 10 would give an 
>>> offset of 50.. plus a user supplied offset turns it into say, 15050..
>> 
>> I understand the reasons, however, any complicated encoding will 
>> defeat the idea of the value can be sort of numeric mnemonic for the 
>> country code ÿÿ well, so it is. I elaborated on your idea and came-up 
>> with the following formula:  val = (C1-60)*1000 + C2*10 + offset. The 
>> offset can be given as the parameter to the -x flag.
>> 
>> $ geoip -t "DE:BR:US" -4 -x 3
>> ...
>> table 0 add 93.157.48.0/21 38690
>> table 0 add 93.158.236.0/22 36820
>> table 0 add 93.159.96.0/19 38690
>> table 0 add 93.159.248.0/21 38690
>> table 0 add 93.180.72.0/21 38690
>> table 0 add 93.180.152.0/21 38690
>> table 0 add 93.181.0.0/18 38690
>> table 0 add 93.183.0.0/18 55830
>> ...
>> 
>> The limits (without offset) are:
>> AA = 5650  -- actually AD = 5680
>> ZZ = 30900
>> 
>> With this formula, the offset must be less than 34735. Although, 
>> this wastes a considerable amount of rule number space, the advantage is 
>> that the numbers are still accessible by mental arithmetic, and the 
>> other constraint of having a step > 1 is fulfilled as well. Anyway, this 
>> one is not graved in stone, we can agree on another one.
> 
> Sorry, but that encoding takes up way too much (perhaps precious) rule 

Re: your thoughts on a particualar ipfw action.

[Ian Smith]

On Fri, 5 Aug 2016 01:38:45 +1000, Ian Smith wrote:

 > <<< No Message Collected >>>

Yeah, sorry about that .. this got stuck in mailq somehow in 'locked' 
EHLO state .. never seen that before in many years; had to kill and 
resend it from sent-mail as a fwd, losing 'References:'.  Weird.

Ian
___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Ian Smith]

<<< No Message Collected >>>
___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Julian Elischer]

Wow, this is getting to be a very useful tool.
thanks for all the work. I look forward to the port..

On 4/08/2016 5:53 AM, Dr. Rolf Jansen wrote:

Am 03.08.2016 um 11:13 schrieb Julian Elischer :

On 2/08/2016 8:50 PM, Dr. Rolf Jansen wrote:

Am 02.08.2016 um 05:08 schrieb Julian Elischer :

looking for thoughts from people who know the new IPFW features well..


A recent addition to our armory is the geoip program that, given an address can 
tell you what country it is in and given a country code, can give an ipfw table 
that describes all the ip addresses in that country.

SO I was thinking how to use this, and the obvious way would be to have a set of rules 
for each country, and use the "skipto tablearg" facility to skip to the right 
rules for each country. But the trouble is that a tablearg skipto is very inefficient. 
It's also a hard thing to set up with a set of rules for each country (how many countries 
are there in the internet allocation system?).

As of today a total of 236 country codes are in use for IPv4 delegations. If 
this helps for anything, a command line switch to the geoip tool could be added 
for letting it output the country code (as the hex encoded CC taken as a plain 
decimal integer) as the value for the given table entry. In the moment you can 
give one value for all entries generated by geoip, with this switch set, the 
output of geoip could look like:

$ geoip -t "DE:BR:US" -x
...
table 0 add 93.157.48.0/21 4445
table 0 add 93.158.236.0/22 4252
table 0 add 93.159.96.0/19 4445
table 0 add 93.159.248.0/21 4445
table 0 add 93.180.72.0/21 4445
table 0 add 93.180.152.0/21 4445
table 0 add 93.181.0.0/18 4445
table 0 add 93.183.0.0/18 5553
...

Given that ...
0x4445 = 'DE'
0x4252 = 'BR'
0x5553 = 'US'

well it would have to be the decimal value so DE would be 6869, as while 4445 
works 444F is a problem :-)

Yes, you're right, I was taken away by the wave of enthusiasm, before thinking 
about the subject up to the end.


0x444F would work but we waste even more address space.  You'd have to multiply 
the numbers by some scaler, because adjacent numbers would not be enough for 
one rule to do something and another rule to skip on to somewhere else. (well, 
you could have multiple rules at the same number but that has its limitations.
The idea would certainly work. it would mean setting aside all  the rules  
between 6565 and 9090 however.
A more compact encoding could be something like ((name[0]-'A') * 
32)+(name[1]-'A')) multiplied by some 'step' (maybe 10 by default) and offset 
by a given offset.

so AF (Afghanistan) would be the first 0*32+5  * 10 would give an offset of 
50.. plus a user supplied offset turns it into say, 15050..

I understand the reasons, however, any complicated encoding will defeat the 
idea of the value can be sort of numeric mnemonic for the country code – well, 
so it is. I elaborated on your idea and came-up with the following formula:  
val = (C1-60)*1000 + C2*10 + offset. The offset can be given as the parameter 
to the -x flag.

$ geoip -t "DE:BR:US" -4 -x 3
...
table 0 add 93.157.48.0/21 38690
table 0 add 93.158.236.0/22 36820
table 0 add 93.159.96.0/19 38690
table 0 add 93.159.248.0/21 38690
table 0 add 93.180.72.0/21 38690
table 0 add 93.180.152.0/21 38690
table 0 add 93.181.0.0/18 38690
table 0 add 93.183.0.0/18 55830
...

The limits (without offset) are:
AA = 5650  -- actually AD = 5680
ZZ = 30900

With this formula, the offset must be less than 34735. Although, this wastes a 
considerable amount of rule number space, the advantage is that the numbers are 
still accessible by mental arithmetic, and the other constraint of having a step 
> 1 is fulfilled as well. Anyway, this one is not graved in stone, we can agree 
on another one.


or there could be a translation into iso3166 numeric codes where Afghanistan is 
004, but then you have the worry of keeping the data up to date as they add new 
country codes, which in my opinion makes it a bad solution.

Agreed, too much dependencies, let's forget the numeric iso codes.


BTW: The ipdb tools are now IPv6 aware.

$ geoip 2001:1900:2254:206a::50:5
2001:1900:2254:206a::50:5 in 2001:1900:0:0:0:0:0:0 - 
2001:1900:::::: in US

$ geoip -t "DE:BR:US" -p
...
...
217.194.64.0/20
217.194.224.0/20
217.195.0.0/20
217.195.32.0/20
217.197.80.0/20
217.198.128.0/20
217.198.240.0/20
217.199.64.0/20
217.199.192.0/20
217.224.0.0/11
2001:400:0:0:0:0:0:0/32
2001:408:0:0:0:0:0:0/32
2001:418:0:0:0:0:0:0/32
2001:420:0:0:0:0:0:0/32
2001:428:0:0:0:0:0:0/32
2001:430:0:0:0:0:0:0/32
2001:438:0:0:0:0:0:0/32
...
...

$ geoip -t "" -p | wc -l
   137097

For processing only IPv4 addresses, add the -4 switch (this reproduces the old 
behaviour)
$ geoip -4 -t "" -p | wc -l
   106637

For processing only IPv6 addresses, add the -6 switch
$ geoip -6 -t "" -p | wc -l
30460

106637+30460 = 137097

After running 'make install' of the new version, the IP database 

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

> Am 03.08.2016 um 11:13 schrieb Julian Elischer :
> 
> On 2/08/2016 8:50 PM, Dr. Rolf Jansen wrote:
>>> Am 02.08.2016 um 05:08 schrieb Julian Elischer :
>>> 
>>> looking for thoughts from people who know the new IPFW features well..
>>> 
>>> 
>>> A recent addition to our armory is the geoip program that, given an address 
>>> can tell you what country it is in and given a country code, can give an 
>>> ipfw table that describes all the ip addresses in that country.
>>> 
>>> SO I was thinking how to use this, and the obvious way would be to have a 
>>> set of rules for each country, and use the "skipto tablearg" facility to 
>>> skip to the right rules for each country. But the trouble is that a 
>>> tablearg skipto is very inefficient. It's also a hard thing to set up with 
>>> a set of rules for each country (how many countries are there in the 
>>> internet allocation system?).
>> 
>> As of today a total of 236 country codes are in use for IPv4 delegations. If 
>> this helps for anything, a command line switch to the geoip tool could be 
>> added for letting it output the country code (as the hex encoded CC taken as 
>> a plain decimal integer) as the value for the given table entry. In the 
>> moment you can give one value for all entries generated by geoip, with this 
>> switch set, the output of geoip could look like:
>> 
>> $ geoip -t "DE:BR:US" -x
>> ...
>> table 0 add 93.157.48.0/21 4445
>> table 0 add 93.158.236.0/22 4252
>> table 0 add 93.159.96.0/19 4445
>> table 0 add 93.159.248.0/21 4445
>> table 0 add 93.180.72.0/21 4445
>> table 0 add 93.180.152.0/21 4445
>> table 0 add 93.181.0.0/18 4445
>> table 0 add 93.183.0.0/18 5553
>> ...
>> 
>> Given that ...
>> 0x4445 = 'DE'
>> 0x4252 = 'BR'
>> 0x5553 = 'US'
> 
> well it would have to be the decimal value so DE would be 6869, as while 4445 
> works 444F is a problem :-)

Yes, you're right, I was taken away by the wave of enthusiasm, before thinking 
about the subject up to the end.

> 0x444F would work but we waste even more address space.  You'd have to 
> multiply the numbers by some scaler, because adjacent numbers would not be 
> enough for one rule to do something and another rule to skip on to somewhere 
> else. (well, you could have multiple rules at the same number but that has 
> its limitations.
> The idea would certainly work. it would mean setting aside all  the rules  
> between 6565 and 9090 however.
> A more compact encoding could be something like ((name[0]-'A') * 
> 32)+(name[1]-'A')) multiplied by some 'step' (maybe 10 by default) and offset 
> by a given offset.
> 
> so AF (Afghanistan) would be the first 0*32+5  * 10 would give an offset of 
> 50.. plus a user supplied offset turns it into say, 15050..

I understand the reasons, however, any complicated encoding will defeat the 
idea of the value can be sort of numeric mnemonic for the country code – well, 
so it is. I elaborated on your idea and came-up with the following formula:  
val = (C1-60)*1000 + C2*10 + offset. The offset can be given as the parameter 
to the -x flag.

$ geoip -t "DE:BR:US" -4 -x 3
...
table 0 add 93.157.48.0/21 38690
table 0 add 93.158.236.0/22 36820
table 0 add 93.159.96.0/19 38690
table 0 add 93.159.248.0/21 38690
table 0 add 93.180.72.0/21 38690
table 0 add 93.180.152.0/21 38690
table 0 add 93.181.0.0/18 38690
table 0 add 93.183.0.0/18 55830
...

The limits (without offset) are:
AA = 5650  -- actually AD = 5680
ZZ = 30900

With this formula, the offset must be less than 34735. Although, this wastes a 
considerable amount of rule number space, the advantage is that the numbers are 
still accessible by mental arithmetic, and the other constraint of having a 
step > 1 is fulfilled as well. Anyway, this one is not graved in stone, we can 
agree on another one.

> or there could be a translation into iso3166 numeric codes where Afghanistan 
> is 004, but then you have the worry of keeping the data up to date as they 
> add new country codes, which in my opinion makes it a bad solution.

Agreed, too much dependencies, let's forget the numeric iso codes.


BTW: The ipdb tools are now IPv6 aware.

$ geoip 2001:1900:2254:206a::50:5
2001:1900:2254:206a::50:5 in 2001:1900:0:0:0:0:0:0 - 
2001:1900:::::: in US

$ geoip -t "DE:BR:US" -p
...
...
217.194.64.0/20
217.194.224.0/20
217.195.0.0/20
217.195.32.0/20
217.197.80.0/20
217.198.128.0/20
217.198.240.0/20
217.199.64.0/20
217.199.192.0/20
217.224.0.0/11
2001:400:0:0:0:0:0:0/32
2001:408:0:0:0:0:0:0/32
2001:418:0:0:0:0:0:0/32
2001:420:0:0:0:0:0:0/32
2001:428:0:0:0:0:0:0/32
2001:430:0:0:0:0:0:0/32
2001:438:0:0:0:0:0:0/32
...
...

$ geoip -t "" -p | wc -l
  137097

For processing only IPv4 addresses, add the -4 switch (this reproduces the old 
behaviour)
$ geoip -4 -t "" -p | wc -l
  106637

For processing only IPv6 addresses, add the -6 switch
$ geoip -6 -t "" -p | wc -l
   30460

106637+30460 = 137097

After running 'make install' of the new 

Re: your thoughts on a particualar ipfw action.

[Julian Elischer]

On 2/08/2016 8:50 PM, Dr. Rolf Jansen wrote:

Am 02.08.2016 um 05:08 schrieb Julian Elischer :

looking for thoughts from people who know the new IPFW features well..


A recent addition to our armory is the geoip program that, given an address can 
tell you what country it is in and given a country code, can give an ipfw table 
that describes all the ip addresses in that country.

SO I was thinking how to use this, and the obvious way would be to have a set of rules 
for each country, and use the "skipto tablearg" facility to skip to the right 
rules for each country. But the trouble is that a tablearg skipto is very inefficient. 
It's also a hard thing to set up with a set of rules for each country (how many countries 
are there in the internet allocation system?).

As of today a total of 236 country codes are in use for IPv4 delegations. If 
this helps for anything, a command line switch to the geoip tool could be added 
for letting it output the country code (as the hex encoded CC taken as a plain 
decimal integer) as the value for the given table entry. In the moment you can 
give one value for all entries generated by geoip, with this switch set, the 
output of geoip could look like:

$ geoip -t "DE:BR:US" -x
...
table 0 add 93.157.48.0/21 4445
table 0 add 93.158.236.0/22 4252
table 0 add 93.159.96.0/19 4445
table 0 add 93.159.248.0/21 4445
table 0 add 93.180.72.0/21 4445
table 0 add 93.180.152.0/21 4445
table 0 add 93.181.0.0/18 4445
table 0 add 93.183.0.0/18 5553
...

Given that ...
0x4445 = 'DE'
0x4252 = 'BR'
0x5553 = 'US'
well it would have to be the decimal value so DE would be 6869, as 
while 4445 works 444F is a problem :-)
0x444F would work but we waste even more address space.  You'd have to 
multiply the numbers by some scaler, because adjacent numbers would 
not be enough for one rule to do something and another rule to skip on 
to somewhere else. (well, you could have multiple rules at the same 
number but that has its limitations.
The idea would certainly work. it would mean setting aside all  the 
rules  between 6565 and 9090 however.
A more compact encoding could be something like ((name[0]-'A') * 
32)+(name[1]-'A')) multiplied by some 'step' (maybe 10 by default) and 
offset by a given offset.


so AF (Afghanistan) would be the first 0*32+5  * 10 would give an 
offset of 50.. plus a user supplied offset turns it into say, 15050..


or there could be a translation into iso3166 numeric codes where 
Afghanistan is 004, but then you have the worry of keeping the data up 
to date as they add new country codes, which in my opinion makes it a 
bad solution.


..., IT people who know by heart the low ASCII table like chemists (are 
supposed to) know the periodic table of the elements, this should be not too 
hard to remember.

true



Another way would be to just put 'action numbers' in the tablearg field and 
have a few actions, shared by countries, but the trouble comes when you want to 
 change the action for  a country, you need to rewrite potentially thousands of 
entries (USA has over 15800 allocations).

Two or more geoip commands can be used for populating ipfw tables for different 
utilization in ipfw directives:

# Europe
geoip -t "FR:IT:DE:NL:BE:GB:..." -n 1 -x | ipfw -q > /dev/stdin

# North America
geoip -t "US:CA" -n 2 -x | ipfw -q > /dev/stdin

# South America
geoip -t "AR:BR:UR:CL:PY:BO:PE..." -n 3 -x | ipfw -q > /dev/stdin

...


A second way woudl be to somehow map the tablearg of the country, into a table 
of actions. effectively doing two levels of lookup.

The first table converting IP addresses to a country number and a second lookup 
converting that to an action.

the only trouble is that I don't know of a way to do that.  If the new changes 
allow that, and anyone knows how, please let me know :-).

Looking-up a given IP in the totally balanced binary search tree takes on a 
decent system on average about 10-20 nanoseconds. So in theory 50 to 100 
million packets per second could be filtered by this algorithm. In order to 
come more close to this performance in reality, it might be an option to move 
the search algorithm into ipfw.

Best regards

Rolf

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"



___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Michael Sierchio]

On Tue, Aug 2, 2016 at 1:08 AM, Julian Elischer  wrote:


>
> A recent addition to our armory is the geoip program that, given an
> address can tell you what country it is in and given a country code, can
> give an ipfw table that describes all the ip addresses in that country.
>
>
I look forward to getting acquainted with the new features, but I have an
observation - a database of networks by country is not invariably a
geographic database. If you were to look at IP allocations in the
Caribbean, or other overseas territories of the Netherlands, France, etc.
you'd see what I mean. There's even a bit of FR in North America,
Saint-Pierre & Miquelon.

It works pretty well for excluding North Korea, Afghanistan, Yemen,
Somalia, etc. but can sometimes be confusing.

-- 
"Well," Brahma said, "even after ten thousand explanations, a fool is no
wiser, but an intelligent man requires only two thousand five hundred."

- The Mahābhārata
___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Dr. Rolf Jansen]

> Am 02.08.2016 um 05:08 schrieb Julian Elischer :
> 
> looking for thoughts from people who know the new IPFW features well..
> 
> 
> A recent addition to our armory is the geoip program that, given an address 
> can tell you what country it is in and given a country code, can give an ipfw 
> table that describes all the ip addresses in that country.
> 
> SO I was thinking how to use this, and the obvious way would be to have a set 
> of rules for each country, and use the "skipto tablearg" facility to skip to 
> the right rules for each country. But the trouble is that a tablearg skipto 
> is very inefficient. It's also a hard thing to set up with a set of rules for 
> each country (how many countries are there in the internet allocation 
> system?).

As of today a total of 236 country codes are in use for IPv4 delegations. If 
this helps for anything, a command line switch to the geoip tool could be added 
for letting it output the country code (as the hex encoded CC taken as a plain 
decimal integer) as the value for the given table entry. In the moment you can 
give one value for all entries generated by geoip, with this switch set, the 
output of geoip could look like:

$ geoip -t "DE:BR:US" -x
...
table 0 add 93.157.48.0/21 4445
table 0 add 93.158.236.0/22 4252
table 0 add 93.159.96.0/19 4445
table 0 add 93.159.248.0/21 4445
table 0 add 93.180.72.0/21 4445
table 0 add 93.180.152.0/21 4445
table 0 add 93.181.0.0/18 4445
table 0 add 93.183.0.0/18 5553
...

Given that ...
0x4445 = 'DE'
0x4252 = 'BR'
0x5553 = 'US'

..., IT people who know by heart the low ASCII table like chemists (are 
supposed to) know the periodic table of the elements, this should be not too 
hard to remember.

> Another way would be to just put 'action numbers' in the tablearg field and 
> have a few actions, shared by countries, but the trouble comes when you want 
> to  change the action for  a country, you need to rewrite potentially 
> thousands of entries (USA has over 15800 allocations).

Two or more geoip commands can be used for populating ipfw tables for different 
utilization in ipfw directives:

# Europe
geoip -t "FR:IT:DE:NL:BE:GB:..." -n 1 -x | ipfw -q > /dev/stdin

# North America
geoip -t "US:CA" -n 2 -x | ipfw -q > /dev/stdin

# South America
geoip -t "AR:BR:UR:CL:PY:BO:PE..." -n 3 -x | ipfw -q > /dev/stdin

...

> A second way woudl be to somehow map the tablearg of the country, into a 
> table of actions. effectively doing two levels of lookup.
> 
> The first table converting IP addresses to a country number and a second 
> lookup converting that to an action.
> 
> the only trouble is that I don't know of a way to do that.  If the new 
> changes allow that, and anyone knows how, please let me know :-).

Looking-up a given IP in the totally balanced binary search tree takes on a 
decent system on average about 10-20 nanoseconds. So in theory 50 to 100 
million packets per second could be filtered by this algorithm. In order to 
come more close to this performance in reality, it might be an option to move 
the search algorithm into ipfw.

Best regards

Rolf

___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"

Re: your thoughts on a particualar ipfw action.

[Andrey V. Elsukov]

On 02.08.16 11:08, Julian Elischer wrote:
> SO I was thinking how to use this, and the obvious way would be to have
> a set of rules for each country, and use the "skipto tablearg" facility
> to skip to the right rules for each country. But the trouble is that a
> tablearg skipto is very inefficient. It's also a hard thing to set up
> with a set of rules for each country (how many countries are there in
> the internet allocation system?).

You can build ipfw with enabled LINEAR_SKIPTO and use the same rules for
most countries.

-- 
WBR, Andrey V. Elsukov



signature.asc
Description: OpenPGP digital signature

your thoughts on a particualar ipfw action.

[Julian Elischer]

looking for thoughts from people who know the new IPFW features well..


A recent addition to our armory is the geoip program that, given an 
address can tell you what country it is in and given a country code, 
can give an ipfw table that describes all the ip addresses in that 
country.


SO I was thinking how to use this, and the obvious way would be to 
have a set of rules for each country, and use the "skipto tablearg" 
facility to skip to the right rules for each country. But the trouble 
is that a tablearg skipto is very inefficient. It's also a hard thing 
to set up with a set of rules for each country (how many countries are 
there in the internet allocation system?).


Another way would be to just put 'action numbers' in the tablearg 
field and have a few actions, shared by countries, but the trouble 
comes when you want to  change the action for  a country, you need to 
rewrite potentially thousands of entries (USA has over 15800 allocations).


A second way woudl be to somehow map the tablearg of the country, into 
a table of actions. effectively doing two levels of lookup.


The first table converting IP addresses to a country number and a 
second lookup converting that to an action.


the only trouble is that I don't know of a way to do that.  If the new 
changes allow that, and anyone knows how, please let me know :-).




___
freebsd-ipfw@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscr...@freebsd.org"