Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/503#discussion_r109336484
--- Diff: metron-deployment/roles/sensor-stubs/templates/start-bro-stub ---
@@ -47,8 +47,8 @@ TOPIC="bro"
while true; do
# transform the bro timestamp and push to kafka
- SEARCH="\"ts\"\:[0-9]\+.[0-9]\{6\}"
- REPLACE="\"ts\"\:`date +%s`.000000"
+ SEARCH="\"ts\"\:[0-9]\+\."
+ REPLACE="\"ts\"\:`date +%s`\."
--- End diff --
@JonZeolla , good catch. Leaving the fractional portion of the timestamp
the same as it is, is appealing. However, since the granularity of `date +%s`
is only seconds, and we might transform a bunch of timestamps in one second of
wallclock realtime, this may result in apparently out-of-order timestamps, no?
Eg, if we start with data whose first three records have timestamps:
1491190032.222222 1491190032.777777 1491190033.111111
The transformed data will have timestamps
1491190442.222222 1491190442.777777 1491190442.111111
with later ones being (at least potentially) out of order. The original
code would have generated
1491190442.000000 1491190442.000000 1491190442.000000
which is rather monotone, but at least not out of order.
Is this okay, or potentially bad?
Perhaps it would be better to just change the `.[0-9]\{6\}` to `\.[0-9]\+`
in line 50, and leaving line 51 unchanged?
(I'm asking, I don't know. Maybe bro data can naturally be out of order?)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
