Re: Software or PHP/PERL scripts for simple network management?

[David Nolan]

--On June 20, 2007 1:02:05 AM +0100 Leigh Porter 
<[EMAIL PROTECTED]> wrote:



Do Pilosoft supply such a product? All the ones I tried so far suck soo
much that I could never use them.

Right now we manage address space with mysql and perl scripts...


Carnegie Mellon's NetReg  (*) is an open 
source system that provides a pretty complete IP Address Management 
toolset, including management of DNS & DHCP configurations for ISC 
bind/dhcpd.  We manage DNS & DHCP for 50K machines, and NetReg does it all. 
It is available under an OSS license and is in use at several other 
locations. NetReg provides a self service web interface with flexible 
permissions, privilege delegation, IP address space management, DNS record 
validation, and more.


As the current primary developer of the system I'm a bit biased, but I 
think its a great system. It has a steep learning curve, and the 
documentation leaves something to be desired (like a tech writer...), but 
once you hit a certain scale the benefits outway the cost. On our site 
you'll find several screen shots and a working demo with some base data you 
can experiment with, but obviously the full power of the system isn't 
utilized until you have lots of data and can see the resulting zones & 
config files.


There is an active mailing list, feel free to join it and ask questions.


*: Not to be confused with Southwestern University's NetReg, which is a 
completely different system developed in parallel around the same time.




-David Nolan
Network Software Designer
Computing Services
Carnegie Mellon University

Re: Software or PHP/PERL scripts for simple network management?

[alex]

On Tue, 19 Jun 2007, William Allen Simpson wrote:

> 
> [EMAIL PROTECTED] wrote:
> > Neither 'show ip route' or 'have a text file' scale beyond a hundred 
> > customers. 
> > 
> Hogwash.  Used text file allocation for ~3,000 customers.  After all, it
> is *REQUIRED* to exist (for bind).  You need *a* canonical place that is
> authoritative for all others.  Existing tools easily track commits.
> 
> DNS should always reflect reality.  Then automated tools will show human
> readable information.  Someday, it may even be authenticated (but I've
> been beating that horse for a decade).  I'm sick and tired of bad NS
> data.
I agree, DNS should *reflect* reality, but I think it is very much 
misguided to say that DNS should be the place to have canonical 
information (i.e. source of all data). Canonical data is in 
routing/forwarding tables on routers/switches. That's the operational 
reality.

The amount of data that you need to track IP allocations just doesn't fit
well into DNS - there's no place to store customer id/service id, the
length of allocation (is this IP part of a /28? /29?), etc. So you'll have
to have "canonical data" somewhere else anyway.

> Yes, we used a separate database for billing, and maybe could have
> automatically generated the text file.  Didn't want the customer
> service/billing folks to have access to network configuration ;-)
> 
> Any time you have more than a single location for maintaining network
> configuration data, or allow technicians to just slap a route into a
> router on a whim, you are bound for future difficulties!
> 
> And when the routing table doesn't match, withdraw the route, and fire
> the miscreant that failed to properly maintain the allocation data!
Unfortunately, I'll have to say again that this doesn't scale. :)

-alex

Re: Software or PHP/PERL scripts for simple network management?

[William Allen Simpson]

[EMAIL PROTECTED] wrote:
Neither 'show ip route' or 'have a text file' scale beyond a hundred 
customers. 


Hogwash.  Used text file allocation for ~3,000 customers.  After all, it
is *REQUIRED* to exist (for bind).  You need *a* canonical place that is
authoritative for all others.  Existing tools easily track commits.

DNS should always reflect reality.  Then automated tools will show human
readable information.  Someday, it may even be authenticated (but I've
been beating that horse for a decade).  I'm sick and tired of bad NS data.

Yes, we used a separate database for billing, and maybe could have
automatically generated the text file.  Didn't want the customer
service/billing folks to have access to network configuration ;-)

Any time you have more than a single location for maintaining network
configuration data, or allow technicians to just slap a route into a router
on a whim, you are bound for future difficulties!

And when the routing table doesn't match, withdraw the route, and fire the
miscreant that failed to properly maintain the allocation data!

Re: Software or PHP/PERL scripts for simple network management?

[alex]

On Wed, 20 Jun 2007, Leigh Porter wrote:

> Do Pilosoft supply such a product? All the ones I tried so far suck soo
> much that I could never use them.
> 
> Right now we manage address space with mysql and perl scripts...
It is very much an internal system, designed to meet our needs, as such it 
is tightly integrated with the rest of the systems - billing, customer 
management, network mapping, etc. 

I've been giving some thought to cleaning it up and releasing it under
some sort of a public license in hope it'll be useful to someone, but
unfortunately hasn't found time yet :(

I think realistically, even if you have full source, it'll be good for the
ideas how to do things, it will be *very hard* to separate the IP 
management out of everything else.

(IP management is maybe few hundred lines of perl pl/pgsql code total)

hth

-alex

Re: Software or PHP/PERL scripts for simple network management?

[Leigh Porter]

[EMAIL PROTECTED] wrote:

On Tue, 19 Jun 2007, William Allen Simpson wrote:

  

Drew Weaver wrote:


Does anyone have a recommendation of any software products
either commercial or freeware which will import the ip routing table
from one of my routers/switches and display it in a sorted manner? We
just need an easier distributed method than logging into our Black
Diamond and typing sh iproute sorted every time we need to find an
available subnet.

  

Wow, LOL!

The software product is called a "text editor".

Look at your list of assignments in your NS .arpa. file:
  1) Find a subnet that hasn't been assigned.
  2) Update the text file.
  3) Wait for it to propagate.
  4) Tell the customer.

The concomitant procedure for static host assignment is:
  1) Find a number that hasn't been assigned.
  2) Update the text file.
  3) Wait for it to propagate.
  4) Then, and only then, update the forward NS file(s).
  5) Tell the customer.

Of course, there is software that will automatically maintain the files,
and even send a signal to bind, but I've alway found them to be weak at
subnet management.  Text editor is the way to go -- using subversion for
"distributed" file management (that is, knowing who to blame for
mangling the assignment commit).


In words of Vijay, "It does not scale".
In words of Randy, "I encourage my competitors to do this".

Neither 'show ip route' or 'have a text file' scale beyond a hundred 
customers. 


Proper IP management is complicated. You want to have following things:

a) easy IP allocation

b) IP association with customer and specific service for following
purposes: 

* future IP justification with RIR's 


* abuse trackback
 
c) easy IP deallocation when customer leaves


d) minimizing additional fragmentation of blocks - for example, if you
need a /29 and you have a /29 and a /28 available - you want to take /29
before fragmenting /28.

e) support for 'special-purpose blocks' - ie, /30 for pt-pt and 
/32 for loopbacks are to be assigned from blocks that are not used for any 
other purpose.


f) (similar to above) regional/local allocations: "give me a /32 out of 
dallas loopback blocks"


g) two-way sync (or at least diff) of your databases to operational data 
(the configs in routers) - so you can see what it *should* be vs what it 
actually is.  Ideally, generate commands to update configs to the 
database.


I think everyone ends up writing their own systems to manage IP space as
part of general network management.  Unfortunately, they end up being very
specific to the network in question (for example, my stuff is very geared 
toward terminating a large number of vlans on a l3 switches, etc)...



--
Alex Pilosov| DSL, Colocation, Hosting Services
President   | [EMAIL PROTECTED]877-PILOSOFT x601
Pilosoft, Inc.  | http://www.pilosoft.com
  
Do Pilosoft supply such a product? All the ones I tried so far suck soo 
much that I could never use them.


Right now we manage address space with mysql and perl scripts...

--
Leigh

Re: Quarantining infected hosts (Was: FBI tells the public to call their ISP for help)

[Leigh Porter]

Douglas Otis wrote:



On Jun 19, 2007, at 8:35 AM, Suresh Ramasubramanian wrote:

On 6/19/07, Leigh Porter <[EMAIL PROTECTED]> wrote:
Agreed, SMTP is not really a special vector, other than it's obvious 
commercial spam use. So just block all the usual virus vector ports, 
block 25 and force people to use your own SMTP servers and the 
problem [for] this particular one goes away..


No. the part of it you target (outbound spam) merely relocates 
itself, and your smtp servers become huge spam sinks.  Filter all you 
want and you'll still leak spam unless you take those hosts down


And in the meantime those hosts will also be launching dos attacks, 
hosting "fast flux" pills / warez / kiddy pr0n sites, carrying out id 
/ card theft .. best to isolate and take them down.


You can port block at your edge till you burst and you'll still be in 
a lot of hot water.


Web-site/browser vulnerabilities make ISP efforts largely futile.  
Infection rates easily overwhelm aggressive automated detection and 
wall-garden strategies.  Nevertheless, blocking port 25 offers several 
benefits even for this seemingly failing effort.  Messages can be rate 
limited, where delivery errors also provide direct clues as to which 
system are likely infected.


Web related script vulnerabilities impact some of the largest online 
email providers!  In the zeal to enable advertising, customer accounts 
are easily harvested.  These accounts may also receive password 
updates from other accounts, placing even critical financial 
information at risk.  Every compromised account is then able to 
impersonate owners, utilize their address book and entice further 
infections by offering malware related messages.  The malware might 
appear as seemingly harmless links or documents.  Email is a vector 
that must be watched carefully, however the greater danger is with 
web/browser vulnerabilities.


Complacency permitting, and at times even promoting use of known 
defective products must end.  The era of combining scripts and active 
code along with every piece of information conveyed must end.  Unless 
the Internet industry responds effectively, legislators will likely to 
react in their own futile way.


Less is more.  A document MUST NOT require active code to convey 
information.


-Doug

This is a great point Doug. Port based vulns are, IMO, starting to 
decline due to update of SP2 etc. There's still a lot there but in a few 
years it will be quite low as hopefully most people will either filter 
it or customers will have default on firewalls.


Browsers and dumb customers opening emails are where it's at now. The 
only way to filter that is to look at ALL traffic using some horrid DPI 
box or proxy or something.


life really sucks.

--
Leigh

Re: Software or PHP/PERL scripts for simple network management?

[Leigh Porter]

William Allen Simpson wrote:


Drew Weaver wrote:
Does anyone have a recommendation of any software products 
either commercial or freeware which will import the ip routing table 
from one of my routers/switches and display it in a sorted manner? We 
just need an easier distributed method than logging into our Black 
Diamond and typing sh iproute sorted every time we need to find an 
available subnet.



Wow, LOL!

The software product is called a "text editor".

Look at your list of assignments in your NS .arpa. file:
 1) Find a subnet that hasn't been assigned.
 2) Update the text file.
 3) Wait for it to propagate.
 4) Tell the customer.

The concomitant procedure for static host assignment is:
 1) Find a number that hasn't been assigned.
 2) Update the text file.
 3) Wait for it to propagate.
 4) Then, and only then, update the forward NS file(s).
 5) Tell the customer.

Of course, there is software that will automatically maintain the files,
and even send a signal to bind, but I've alway found them to be weak at
subnet management.  Text editor is the way to go -- using subversion for
"distributed" file management (that is, knowing who to blame for mangling
the assignment commit).


However Drew suffers because some idiots in his org fail to update the 
files correctly. I used to have the same problem when I took over ops at 
a small ISP. They were using the routing table to store assigned subnets 
trick. It was OK until a link died so a subnet dropped out of the 
routing table. They thought "Oh look spare space" and assigned it to 
somebody else.


There are also a load of decent (not good) free IP address management 
systems available, some with built in DNS updaters.


I do not use these because they all drove me mad. Now I just have 
somebody else do it for me. It's worth it ;-)



--
Leigh

Re: Software or PHP/PERL scripts for simple network management?

[alex]

On Tue, 19 Jun 2007, William Allen Simpson wrote:

> 
> Drew Weaver wrote:
> > Does anyone have a recommendation of any software products
> > either commercial or freeware which will import the ip routing table
> > from one of my routers/switches and display it in a sorted manner? We
> > just need an easier distributed method than logging into our Black
> > Diamond and typing sh iproute sorted every time we need to find an
> > available subnet.
> > 
> Wow, LOL!
> 
> The software product is called a "text editor".
> 
> Look at your list of assignments in your NS .arpa. file:
>   1) Find a subnet that hasn't been assigned.
>   2) Update the text file.
>   3) Wait for it to propagate.
>   4) Tell the customer.
> 
> The concomitant procedure for static host assignment is:
>   1) Find a number that hasn't been assigned.
>   2) Update the text file.
>   3) Wait for it to propagate.
>   4) Then, and only then, update the forward NS file(s).
>   5) Tell the customer.
> 
> Of course, there is software that will automatically maintain the files,
> and even send a signal to bind, but I've alway found them to be weak at
> subnet management.  Text editor is the way to go -- using subversion for
> "distributed" file management (that is, knowing who to blame for
> mangling the assignment commit).
In words of Vijay, "It does not scale".
In words of Randy, "I encourage my competitors to do this".

Neither 'show ip route' or 'have a text file' scale beyond a hundred 
customers. 

Proper IP management is complicated. You want to have following things:

a) easy IP allocation

b) IP association with customer and specific service for following
purposes: 

* future IP justification with RIR's 

* abuse trackback
 
c) easy IP deallocation when customer leaves

d) minimizing additional fragmentation of blocks - for example, if you
need a /29 and you have a /29 and a /28 available - you want to take /29
before fragmenting /28.

e) support for 'special-purpose blocks' - ie, /30 for pt-pt and 
/32 for loopbacks are to be assigned from blocks that are not used for any 
other purpose.

f) (similar to above) regional/local allocations: "give me a /32 out of 
dallas loopback blocks"

g) two-way sync (or at least diff) of your databases to operational data 
(the configs in routers) - so you can see what it *should* be vs what it 
actually is.  Ideally, generate commands to update configs to the 
database.

I think everyone ends up writing their own systems to manage IP space as
part of general network management.  Unfortunately, they end up being very
specific to the network in question (for example, my stuff is very geared 
toward terminating a large number of vlans on a l3 switches, etc)...


--
Alex Pilosov| DSL, Colocation, Hosting Services
President   | [EMAIL PROTECTED]877-PILOSOFT x601
Pilosoft, Inc.  | http://www.pilosoft.com

Re: Software or PHP/PERL scripts for simple network management?

[Warren Kumari]

Many years ago I worked for a small Mom-and-Pop type ISP in New York  
state (I was the only network / technical person there) -- it was a  
very free wheeling place and I built the network by doing whatever  
made sense at the time.


One of my "favorite" customers (Joe somebody) was somehow related to  
the owner of the ISP and was a gamer. This was back in the day when  
the gaming magazines would give you useful tips like "Type 'tracert  
$gameserver' and make sure that there are less than N hops".  Joe  
would call up tech support, me, the owner, etc and complain that  
there was N+3 hops and most of them were in our network. I spent much  
time explaining things about packet-loss, latency, etc but couldn't  
shake his belief that hop count was the only metric that mattered.


Finally, one night he called me at home well after midnight (no, I  
didn't give him my home phone number, he looked me up in the  
phonebook!) to complain that his gaming was suffering because it was  
"too many hops to get out of your network". I finally snapped and  
built a static GRE tunnel from the RAS box that he connected to all  
over the network -- it was a thing of beauty, it went through almost  
every device that we owned and took the most convoluted path I could  
come up with. "Yay!", I figured, "now I can demonstrate that latency  
is more important than hop count" and I went to bed.


The next morning I get a call from him. He is ecstatic and wildly  
impressed by how well the network is working for him now and how  
great his gaming performance is. "Oh well", I think, "at least he is  
happy and will leave me alone now". I don't document the purpose of  
this GRE anywhere and after some time forget about it.


A few months later I am doing some routine cleanup work and stumble  
across a weird looking tunnel -- its bizarre, it goes all over the  
place and is all kinds of crufty -- there are static routes and  
policy routing and bizarre things being done on the RADIUS server to  
make sure some user always gets a certain IP... I look in my pile of  
notes and old configs and then decide to just yank it out.


That night I get an enraged call (at home again) from Joe *screaming*  
that the network is all broken again because it is now way too many  
hops to get out of the network and that people keep shooting him...


What I learnt from this:
1: Make sure you document everything (and no, the network isn't  
documentation)

2: Gamers are weird.
3: Making changes to your network in anger provides short term  
pleasure but long term pain.


---
Warren Kumari.
http://www.kumari.net



On Jun 19, 2007, at 2:05 PM, [EMAIL PROTECTED] wrote:


On Mon, 18 Jun 2007 21:18:06 BST, Leigh Porter said:
Just out of interest, why are you looking at routing tables to  
find an

available subnet?


If your predecessor wasn't quite as careful documenting  
allocations, it can
be useful to see if your paperwork says a /28 is dark, but you're  
in fact
routing traffic for it down some customer's link.  Then you get to  
do two
things:  (a) check if there's any *return* traffic and (b) call the  
customer
and ask if *they* think it's dark or not.  Hilarity ensues for some  
combinations

of answers...

(And yes, I once had a co-worker looking for a free /24, found one  
that was
nice and empty except for smack dab in the middle, a route for a / 
28 that for
no apparent reason pointed at an unused but registered static IP of  
mine in the
middle of our modem pool space.  After some digging, we remembered  
that it was
a work-around for when I had 2 IBM RTs at home, that did SLIP and  
static
addresses, but not NAT or DHCP, so my home net had some routing  
workarounds
that never got taken down when I replaced the 2 RTs with one box  
that was happy

to accept whatever address PPP handed it)



Life is a concentration camp.  You're stuck here and there's no way  
out and you can only rage impotently against your persecutors.

-- Woody Allen

Re: Software or PHP/PERL scripts for simple network management?

[Valdis . Kletnieks]

On Mon, 18 Jun 2007 21:18:06 BST, Leigh Porter said:
> Just out of interest, why are you looking at routing tables to find an 
> available subnet?

If your predecessor wasn't quite as careful documenting allocations, it can
be useful to see if your paperwork says a /28 is dark, but you're in fact
routing traffic for it down some customer's link.  Then you get to do two
things:  (a) check if there's any *return* traffic and (b) call the customer
and ask if *they* think it's dark or not.  Hilarity ensues for some combinations
of answers...

(And yes, I once had a co-worker looking for a free /24, found one that was
nice and empty except for smack dab in the middle, a route for a /28 that for
no apparent reason pointed at an unused but registered static IP of mine in the
middle of our modem pool space.  After some digging, we remembered that it was
a work-around for when I had 2 IBM RTs at home, that did SLIP and static
addresses, but not NAT or DHCP, so my home net had some routing workarounds
that never got taken down when I replaced the 2 RTs with one box that was happy
to accept whatever address PPP handed it)



pgpkMkdAuXpfn.pgp
Description: PGP signature

Re: Quarantining infected hosts (Was: FBI tells the public to call their ISP for help)

[Douglas Otis]

On Jun 19, 2007, at 8:35 AM, Suresh Ramasubramanian wrote:

On 6/19/07, Leigh Porter <[EMAIL PROTECTED]> wrote:
Agreed, SMTP is not really a special vector, other than it's  
obvious commercial spam use. So just block all the usual virus  
vector ports, block 25 and force people to use your own SMTP  
servers and the problem [for] this particular one goes away..


No. the part of it you target (outbound spam) merely relocates  
itself, and your smtp servers become huge spam sinks.  Filter all  
you want and you'll still leak spam unless you take those hosts down


And in the meantime those hosts will also be launching dos attacks,  
hosting "fast flux" pills / warez / kiddy pr0n sites, carrying out  
id / card theft .. best to isolate and take them down.


You can port block at your edge till you burst and you'll still be  
in a lot of hot water.


Web-site/browser vulnerabilities make ISP efforts largely futile.   
Infection rates easily overwhelm aggressive automated detection and  
wall-garden strategies.  Nevertheless, blocking port 25 offers  
several benefits even for this seemingly failing effort.  Messages  
can be rate limited, where delivery errors also provide direct clues  
as to which system are likely infected.


Web related script vulnerabilities impact some of the largest online  
email providers!  In the zeal to enable advertising, customer  
accounts are easily harvested.  These accounts may also receive  
password updates from other accounts, placing even critical financial  
information at risk.  Every compromised account is then able to  
impersonate owners, utilize their address book and entice further  
infections by offering malware related messages.  The malware might  
appear as seemingly harmless links or documents.  Email is a vector  
that must be watched carefully, however the greater danger is with  
web/browser vulnerabilities.


Complacency permitting, and at times even promoting use of known  
defective products must end.  The era of combining scripts and active  
code along with every piece of information conveyed must end.  Unless  
the Internet industry responds effectively, legislators will likely  
to react in their own futile way.


Less is more.  A document MUST NOT require active code to convey  
information.


-Doug

Re: Software or PHP/PERL scripts for simple network management?

[William Allen Simpson]

Drew Weaver wrote:

Does anyone have a recommendation of any software products either 
commercial or freeware which will import the ip routing table from one of my 
routers/switches and display it in a sorted manner? We just need an easier 
distributed method than logging into our Black Diamond and typing sh iproute 
sorted every time we need to find an available subnet.


Wow, LOL!

The software product is called a "text editor".

Look at your list of assignments in your NS .arpa. file:
 1) Find a subnet that hasn't been assigned.
 2) Update the text file.
 3) Wait for it to propagate.
 4) Tell the customer.

The concomitant procedure for static host assignment is:
 1) Find a number that hasn't been assigned.
 2) Update the text file.
 3) Wait for it to propagate.
 4) Then, and only then, update the forward NS file(s).
 5) Tell the customer.

Of course, there is software that will automatically maintain the files,
and even send a signal to bind, but I've alway found them to be weak at
subnet management.  Text editor is the way to go -- using subversion for
"distributed" file management (that is, knowing who to blame for mangling
the assignment commit).

Straw Poll - Multipoint Ethernet Connections Best Practices

[Michael K. Smith - Adhost]

Hello All:

As more and more providers move towards Ethernet as a common connection
type in Data Centers, Metro Areas and beyond, I am curious to know what
people are using for Best Practices regarding customer connectivity into
your Layer 2 networks.  Most importantly, how do you handle customers
that want redundant Ethernet connections?  I've put together the
following list of questions and I'd be happy to summarize to the list if
there's any interest.  Also, if individuals have any other pertinent
questions they feel I've left out, please don't hesitate to let me know.

1) Do you allow redundant Ethernet (Layer 2) connections into your
network?
2) Do you allow customers to connect at Layer 2?  And, if so:
a) What, if any, port-based limitations on protocols? (Root
Guard, BPUD-filtering, etc.?)
b) What, if any, MAC restrictions do you have on the ports?
3) Do you allow customers to transmit HSRP/VRRP/GLBP state information
across your Layer 2 infrastructure?
4) Do you require Layer 3 terminations for customers with redundant
connections?
5) Do you provide OSPF "peering" to customers with redundant
connections?
6) Do you provide BGP peering to customers with redundant connections?

Regards,

Michael K. Smith (my real, given name)  :-)
Adhost