[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1083 You got it [METRON-1655](https://issues.apache.org/jira/browse/METRON-1655) ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1083 Why don't you create a jira for the REGEXP_MATCH ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1083 A big thumbs up to not embedding Stellar directly in JSON. I think that would solve a number of problems. ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/1083
I think a decent UI that would make this a hell of a lot easier, but also
not embedding stellar in JSON would help a lot. Something like:
```
match {
REGEXP_MATCH(pix_type, '^6-302.*') => 'cisco-6-302',
REGEXP_MATCH(pix_type, '^5-304.*') => 'cisco-5-304',
default => NULL
}
```
is a lot easier to read and deal with. Also, enabling REGEXP_MATCH to take
multiple regexes via a list in the 2nd arg would make things easier.
---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1083 " Ultimately, I consider this a stop-gap." Yes. What we are basically doing is writing a meta language on top of stellar. In this case we are using that to make up for the meta language itself not supporting multi-line input ( and stellar's support for that as well ) A question would be, why would this transform not just be transformed into a match statement instead of adding a new function? How much of this pain is from not using the UI for editing? Would a ui editor that let you put in a multi line match make this pain better? ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/1083
Sure, so the difference in the parser chaining example would be between the
following
# Stellar
```
"fieldTransformations" : [
{
"transformation" : "STELLAR"
,"input" : "pix_type"
,"output" : "logical_source_type"
,"config" : {
"logical_source_type" : "match{REGEXP_MATCH(pix_type, '^6-302.*') =>
'cisco-6-302', REGEXP_MATCH(pix_type, '^5-304.*') => 'cisco-5-304', default =>
NULL}",
}
}
]
```
# `REGEX_SELECT`
```
"fieldTransformations" : [
{
"transformation" : "REGEX_SELECT"
,"input" : "pix_type"
,"output" : "logical_source_type"
,"config" : {
"cisco-6-302" : "^6-302.*",
"cisco-5-304" : "^5-304.*"
}
}
]
```
It bears mentioning that things get worse in stellar when:
1. You want to match on one of a set of regexs (e.g. `match {
REGEXP_MATCH(pix_type, 'regex1') or REGEXP_MATCH(pix_type, 'regex2') => 'foo',
...`)
2. The set of target kafka topics grows. In this example there are 2, but
if there were 5, they'd all have to fit in a long line of stellar.
I think it's not that stellar isn't equipped to handle it theoretically,
it's just that for these kinds of use-cases, it's more readable to create
something a bit more specific until we get multi-line Stellar available.
Ultimately, I consider this a stop-gap.
---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/1083 Can you provide a "routing in Stellar" versus "routing with REGEX_ROUTING" example that highlights the benefits of this approach? This go against our current trajectory of "do everything in Stellar" and I want to make sure I understand the benefits. ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1083 Ok, I updated with manual testing plan and I think I've made the changes requested. Let me know what you guys think. ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1083 @ottobackwards Makes sense; I made the change and percolated it to METRON-1644 aka #1084 ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1083 +1 to SELECT What you are saying is SET FIELD to X if SELECT. It would be SWITCH if it were a different X per matching regex ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user cestella commented on the issue: https://github.com/apache/metron/pull/1083 @ottobackwards Yeah, my history of really crappy naming continues. How about `REGEX_SELECT` or `REGEX_SWITCH`? ---
[GitHub] metron issue #1083: METRON-1643: Create a REGEX_ROUTING field transformation
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/1083 I am not sure ROUTING is a good name for this. This is more like a SELECT. ---
