Benjamin.
> I think this line in your example should be changed ;)
> > % flow-cat -m 1000 >> test1
> ...to
> > % flow-gen -n 1000 >> test1
Oops, some over-zealous editing, sorry. (I use "fcat" as an alias for
"flow-cat -m", and almost forgot the rest of the world doesn't use that
before I pressed send; hence the dodgy post cut-and-paste change)
> Yes, that's a good solution. Now I just need to recompile
> since the Debian maintainers apparently left out support for
> large files. Does anybody know why they'd do that? Is there
> some catch here?
I am using AIX which does support > 4GB files via ksh but for some
reason bash still doesn't like them. I've definitely done something
wrong if a query returns that much data and ksh is suitable punishment.
Perhaps ksh will work for you?
> Should I rather split filtered flows into a
> lot of small files because of performance? For me it makes
> perfect sense to just store everything in one file, since I
> need it all at once...
With 20GB/day uncompressed I only want to scan the files I really need.
Using 5 minute files with "glob"-able timestamps you can pull out stats
for just (say) 9am without having to read the whole lot from disk. Eg
fcat 2007-06/*/ft-v07.*.09* | flow-nfilter etc...
It also allows for a naive sampling of data. Eg at 5min past the hour
fcat 2007-08-09/ft-v07.*.??05* | flow-nfilter etc...
I used to store separate filtered flow files for key production
applications, but decided that aggregated CSV was better plan. They are
something you can directly pass to colleagues and I always needed to go
return to the full flow file when there were problems anyway.
YMMV
Cheers
Alistair
**********************************************************************
Registered Office:
Marks and Spencer plc
Waterside House
35 North Wharf Road
London
W2 1NW
Registered No. 214436 in England and Wales.
Telephone (020) 7935 4422
Facsimile (020) 7487 2670
<<www.marksandspencer.com>>
Please note that electronic mail may be monitored.
This e-mail is confidential. If you received it by mistake, please let us know
and then delete it from your system; you should not copy, disclose, or
distribute its contents to anyone nor act in reliance on this e-mail, as this
is prohibited and may be unlawful.
2005
_______________________________________________
Flow-tools mailing list
[EMAIL PROTECTED]
http://mailman.splintered.net/mailman/listinfo/flow-tools
