[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=17402#comment-17402 ] Jon Siwek commented on BIT-1215: Just an FYI: I've added a job to Jenkins to run the bro-aux test suite, so bro-cut is now being regression tested automatically. bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Robin Sommer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.4-OD-02-003#64000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1215: - Assignee: Robin Sommer (was: Daniel Thayer) bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Robin Sommer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.4-OD-02-003#64000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1215: -- Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Robin Sommer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.4-OD-02-003#64000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=17300#comment-17300 ] Daniel Thayer commented on BIT-1215: In branch topic/dnthayer/ticket1215, I've made the following changes: 1) bro-cut now handles time conversion for multiple time columns in a log file (and there is a new test case), 2) bro-cut no longer has a hard-coded limit on the number of columns that it can handle, 3) all tests now pass on OS X (previously, some were failing due to strftime(%z) behavior on OS X) bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Daniel Thayer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.4-OD-02-003#64000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1215: --- Status: Merge Request (was: Open) bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Daniel Thayer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.4-OD-02-003#64000) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1215: -- Status: Open (was: Merge Request) bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Robin Sommer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.3-OD-08-005-WN#6328) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1215: - Assignee: Daniel Thayer (was: Robin Sommer) bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Daniel Thayer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.3-OD-08-005-WN#6328) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[
https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=17200#comment-17200
]
Robin Sommer commented on BIT-1215:
---
I noticed a regression compared to the awk-version: the C bro-cut cannot handle
more than one time column when converting to readable output. The branch
{{topic/robin/ticket1215-merge}} has a test case in
{{bro-cut/multiple-times.test}}. Might be a bit painful to fix, but I think we
should ...
bro-cut should be rewritten for speed and to not depend on gawk
---
Key: BIT-1215
URL: https://bro-tracker.atlassian.net/browse/BIT-1215
Project: Bro Issue Tracker
Issue Type: Improvement
Components: Bro, bro-aux
Reporter: Daniel Thayer
Assignee: Robin Sommer
Fix For: 2.4
The current implementation of bro-cut is too slow when processing large log
files (takes more than a minute to process a single log file a few hundred MB
in size). Justin Azoff rewrote bro-cut in C and found that it runs an order
of magnitude faster. Another benefit of a C version of bro-cut is that we
will no longer depend on gawk for anything (and some of Bro's supported
platforms do not include gawk by default).
--
This message was sent by Atlassian JIRA
(v6.3-OD-08-005-WN#6328)
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1215: - Assignee: Robin Sommer bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Assignee: Robin Sommer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.3-OD-08-005-WN#6328) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
[Bro-Dev] [JIRA] (BIT-1215) bro-cut should be rewritten for speed and to not depend on gawk
[ https://bro-tracker.atlassian.net/browse/BIT-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1215: --- Summary: bro-cut should be rewritten for speed and to not depend on gawk (was: bro-cut should be rewritten in C for speed and to not depend on gawk) The current implementation can be compiled with a C++ compiler (and it works), so I guess it's already C++. bro-cut should be rewritten for speed and to not depend on gawk --- Key: BIT-1215 URL: https://bro-tracker.atlassian.net/browse/BIT-1215 Project: Bro Issue Tracker Issue Type: Improvement Components: Bro, bro-aux Reporter: Daniel Thayer Fix For: 2.4 The current implementation of bro-cut is too slow when processing large log files (takes more than a minute to process a single log file a few hundred MB in size). Justin Azoff rewrote bro-cut in C and found that it runs an order of magnitude faster. Another benefit of a C version of bro-cut is that we will no longer depend on gawk for anything (and some of Bro's supported platforms do not include gawk by default). -- This message was sent by Atlassian JIRA (v6.3-OD-08-005-WN#6328) ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
