hi Yuheng,
if deviceID and readingN lines are always properly ordered, with the
deviceID line always immediately followed by all its readingN lines without
any lines from other records, you can set an action list variable to
deviceID when the corresponding line is observed, and use this variable for
the following readingN lines. The arrival rate of input lines does not
really matter, and the content of the action list variable will always
reflect the proper device ID.
However, this approach obviously does not work if the proper ordering of
input lines is not guaranteed and lines from different records are mixed,
for example:
deviceID: dev1
reading0: 0.1
reading1: 0.7
deviceID: dev2
reading2: 0.6 <--- belongs to deviceID: dev1
reading0: 62
reading1: 83
reading2: 44
reading3: 1.9 <--- belongs to deviceID: dev1
In that case, the problem can not be addressed with multiline matching,
since you might encounter unresolvable scenarios. For instance, when
looking at the input only without any external knowledge, it is actually
impossible to tell if the last line from the previous example belongs to
dev1 or dev2. In order to handle such cases, you need to have the deviceID
included in each readingN line.
hope this helps,
risto
2014-08-20 15:10 GMT+03:00 Yuheng Du <[email protected]>:
> Hi guys,
>
> I have a scenario of events I need to match, the coming logstash messages
> is like:
>
> deviceId: aiken_2,
> ......
> reading0: 0.235
> reading1: 3.235
> ....
> readingN: -235.3
>
> The number of readings N varies from 1 to 24.
> For each individual reading, I need a pattern to recognize its sequence
> number, like pattern=reading(\d+), and I also need the deviceId of this
> message. The deviceId and sequence number is passed to a perl script for
> query purpose.
>
> I cannot assign the 'deviceId' to a global variable using a Single rule
> and use that variable in another rule that captures the reading sequence
> number because when multiple messages are coming at high speed, the
> captured 'deviceId' might be inconsistent (wrong deviceId used for a
> node's reading).
>
> What is the best way for me to do this?
>
> If I can not find an efficient solution in SEC to solve this, I can simply
> change my input logstash message's format to:
>
> ...
> aiken_2 reading0: 0.235
> aiken_2 reading1: 3.235
> ....
> aiken_2 readingN: -235.3
>
> which will increase the message size I stored in my disk but will help me
> capture the deviceId and sequence number at the same time.
>
> Thanks!
>
>
>
>
> ------------------------------------------------------------------------------
> Slashdot TV.
> Video for Nerds. Stuff that matters.
> http://tv.slashdot.org/
> _______________________________________________
> Simple-evcorr-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
>
------------------------------------------------------------------------------
Slashdot TV.
Video for Nerds. Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
Simple-evcorr-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
