At work I mostly use Splunk, but then, I don't have to pay for it. At home, and sometimes at work if I have the raw logs available, I use grep to chop up the files into manageable chunks based on what I'm looking for. I don't mean the weak, out-of-date BSD grep Apple ships with Xcode. I get the latest version of gnu grep and build it with the latest PCRE, and not just because I find PCRE indispensible for this kind of work, but also because gnu grep is wicked fast.
Usually I can exctract what I want in one pass and redirect it to a file, but occasionally I pipe it back to grep or run it through sed (also the latest gnu version) and then to disk. This way you get smaller, more manageable files you can open in BBEdit or whatever and and analyze further. You can even get basic statistics like browser share by grepping specific patterns in User-Agent strings into separate files and counting the lines with wc -l. (If you don't need to keep the files you can just pipe grep to wc.) awk is another tool people find useful, but I'm not that proficient with it. I don't know if I'm less proficient because I've found it less useful or the other way around, but the thing I love about Unix is that it has this amazing programming environment filled with useful, interoperable tools that let you accomplish tasks in all sorts of different ways. Most people probably never learn more than a few of them, and which ones they latch onto is either a historical happenstance or because different people just think differently and approach problems idiosyncratically. On Thursday, November 17, 2016 at 12:06:53 PM UTC-5, Jonathan Duke wrote: > > I love BBEdit (since the classic days), but when I'm trying to open and > search some immense log files from our server it just isn't doing the job. > > I know it isn't the best tool for something like this, but when all you > have is a hammer… > > What would people suggest for files of this size (I'm downloading a log > file right now that is already 14 GB of an unknown total). > > Thanks. > > Cheers, > Jon > -- This is the BBEdit Talk public discussion group. If you have a feature request or would like to report a problem, please email "supp...@barebones.com" rather than posting to the group. Follow @bbedit on Twitter: <http://www.twitter.com/bbedit> --- You received this message because you are subscribed to the Google Groups "BBEdit Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+unsubscr...@googlegroups.com. To post to this group, send email to bbedit@googlegroups.com. Visit this group at https://groups.google.com/group/bbedit.
