[GitHub] metron pull request #693: METRON-739 Create Local Profile Runner

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/693


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

[ottobackwards]

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/667#discussion_r133273527
  
--- Diff: 
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java
 ---
@@ -0,0 +1,95 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more 
contributor license
+ * agreements.  See the NOTICE file distributed with this work for 
additional information regarding
+ * copyright ownership.  The ASF licenses this file to you under the 
Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with 
the License.  You may obtain
+ * a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software 
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF 
ANY KIND, either express
+ * or implied. See the License for the specific language governing 
permissions and limitations under
+ * the License.
+ */
+
+package org.apache.metron.stellar.dsl.functions;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Locale;
+import org.apache.commons.lang.StringUtils;
+import org.apache.commons.text.similarity.FuzzyScore;
+import org.apache.metron.stellar.dsl.BaseStellarFunction;
+import org.apache.metron.stellar.dsl.Stellar;
+
+public class TextFunctions {
+
+  @Stellar(name = "GET_AVAILABLE_LANGUAGE_TAGS",
--- End diff --

I always forget about the undocumented Namespace feature... I sure will do 
that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/681
  
Yeah, it's true, I suppose having `==` double as a `NaN` check is probably 
not the right thing due to the transitivity of equality (I'd like to `SQRT(-1) 
== NaN` to be `true` but `SQRT(-1) == SQRT(-2)` to not be true).  Probably it's 
better to have an `IS_NAN(x)` like other languages.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/667#discussion_r133258977
  
--- Diff: 
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java
 ---
@@ -0,0 +1,95 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more 
contributor license
+ * agreements.  See the NOTICE file distributed with this work for 
additional information regarding
+ * copyright ownership.  The ASF licenses this file to you under the 
Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with 
the License.  You may obtain
+ * a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software 
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF 
ANY KIND, either express
+ * or implied. See the License for the specific language governing 
permissions and limitations under
+ * the License.
+ */
+
+package org.apache.metron.stellar.dsl.functions;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Locale;
+import org.apache.commons.lang.StringUtils;
+import org.apache.commons.text.similarity.FuzzyScore;
+import org.apache.metron.stellar.dsl.BaseStellarFunction;
+import org.apache.metron.stellar.dsl.Stellar;
+
+public class TextFunctions {
+
+  @Stellar(name = "GET_AVAILABLE_LANGUAGE_TAGS",
--- End diff --

Do the language tags have any use outside of `FUZZY_SCORE`?

If its only use is for `FUZZY_SCORE`, I was thinking we could the same 
namespace for both functions.
`FUZZY_SCORE` -> namespace = "FUZZY", name = "SCORE"
`FUZZY_LANGS` -> namespace = "FUZZY", name = "LANGS"

Or any other logical naming scheme that ties them together with a 
namespace.  I mean, we do have this cool namespace feature at our disposal.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/681
  
Thanks for clarifying @mattf-horton .  That makes sense now that I've had 
lunch.

Based on the original use case of detecting a function returning NaN, it 
seems what we really need is something like `IS_NAN(x)` rather than a keyword 
for `NaN`.  Implementing `IS_NAN(x)` doesn't technically require a keyword for 
`NaN`.

But I can see a potential use for the `NaN` keyword when creating a lambda 
expression.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[mattf-horton]

Github user mattf-horton commented on the issue:

https://github.com/apache/metron/pull/681
  
@ottobackwards , the validator for IS_NAN(x) in Java would be 
Double.isNaN(double x).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[mattf-horton]

Github user mattf-horton commented on the issue:

https://github.com/apache/metron/pull/681
  
BTW, +/- Infinity **are** considered "real" values (essentially as tho via 
rounding) and can be compared with equality and inequality.  NaN is considered 
a nonsensical or impossible-to-calculate value, not a number at all.  The fact 
that 1.0/0.0 returns positive Infinity rather than NaN is a mathematical 
choice, presumably based on limit logic and a preference for continuity. (I 
didn't write the standard! :-)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #686: METRON-711 STELLAR SHELL Do not set variables when except...

[ottobackwards]

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/686
  
@cestella do you have any inputs to the questions posed in the pr 
description?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[ottobackwards]

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/681
  
We are not 'doing' the math here.  What we are doing is introducing the 
ability to use NaN in an expression correctly.  Java is doing the math.  The 
tests are to verify that through stellar, the specified behavior still is 
adhered to.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

[ottobackwards]

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/667#discussion_r133247450
  
--- Diff: metron-stellar/stellar-common/README.md ---
@@ -411,6 +412,14 @@ In the core language functions, we support basic 
functional programming primitiv
 * format - string
 * arguments... - object(s)
   * Returns: A formatted string.
+  
+### `FUZZY_SCORE`
+  * Description: Returns the Fuzzy Score which indicates the similarity 
score between two strings. One point is given for every matched character. 
Subsequent matches yield two bonus points. A higher score indicates a higher 
similarity.
+  * Input:
+* string - The full term that should be matched against.
+* string - The query that will be matched against a term.
+* string - The IETF BCP 47 language code to use.
--- End diff --

I did both



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #697: METRON-1110: REST jvm flags property is being overwritten...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/697
  
+1 by inspection. Thanks for the fix and for tweaking that @merrimanr 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/681
  
Based on the link that you embedded in the tests, your implementation seems 
correct.  I assume that these grammar rules are well reasoned and necessary, 
but it surely seems counter-intuitive to me for this use case.

> The equality operator == returns false if either operand is NaN. 
> The inequality operator != returns true if either operand is NaN 
(§15.21.1). 
> In particular, x!=x is true if and only if x is NaN, and (x=y) 
will be false if x or y is NaN.

 Am I thinking about this all wrong?  Maybe I just need some lunch to get 
my blood sugar up. :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/681
  
I see in the unit tests that @ottobackwards intended that any equality with 
`NaN` to be false.  But then how is that useful for the basic use case of 
`FUNCTION(x) != NaN`?

```
[Stellar]>>> SQRT(-1)
NaN
```

I would expect the following to return `true`.
```
[Stellar]>>> SQRT(-1) == NaN
false
```

I would expect the following to return `false`.
```
[Stellar]>>> SQRT(-1) != NaN
true
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/681
  
Ok, that makes sense to me.  `FUNCTION(x) != NaN`

The code changes look good to me @ottobackwards.  I am going to pull it 
down and play with NaN in the REPL just as a sanity check.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/667#discussion_r133211499
  
--- Diff: 
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java
 ---
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more 
contributor license
+ * agreements.  See the NOTICE file distributed with this work for 
additional information regarding
+ * copyright ownership.  The ASF licenses this file to you under the 
Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with 
the License.  You may obtain
+ * a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software 
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF 
ANY KIND, either express
+ * or implied. See the License for the specific language governing 
permissions and limitations under
+ * the License.
+ */
+
+package org.apache.metron.stellar.dsl.functions;
+
+import java.util.List;
+import java.util.Locale;
+import org.apache.commons.lang.StringUtils;
+import org.apache.commons.text.similarity.FuzzyScore;
+import org.apache.metron.stellar.dsl.BaseStellarFunction;
+import org.apache.metron.stellar.dsl.Stellar;
+
+public class TextFunctions {
+
+  @Stellar(name = "FUZZY_SCORE",
+  description =
+  "Returns the Fuzzy Score which indicates the similarity score 
between two Strings "
+  +
+  "One point is given for every matched character. Subsequent 
matches yield two bonus "
+  +
+  "points. A higher score indicates a higher similarity",
+  params = {
+  "string - The full term that should be matched against",
+  "string - The query that will be matched against a term",
+  "string - The IETF BCP 47 language code to use"
+  },
+  returns = "integer representing the score")
+  /**
+   * FuzzyScoreFunction exposes the Apache Commons Text Similarity 
FuzzyScore through
+   * Stellar.
+   */
+  public static class FuzzyScoreFunction extends BaseStellarFunction {
+
+@Override
+public Object apply(List list) {
+  if (list.size() < 3) {
+throw new IllegalStateException("FUZZY_SCORE expects three args: 
[string, string, string]");
+  }
+  String term = (String) list.get(0);
--- End diff --

> I think we want to return 0 for invalid args.

Hey @ottobackwards , why do you like returning 0 for invalid args?  Maybe 
there is some reasoning that I am missing.

To me it seems like I can't distinguish between an error condition and two 
dissimilar strings.  A 0 would be returned for both.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #686: METRON-711 STELLAR SHELL Do not set variables when except...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/686
  
+1 by inspection, this is good work!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #667: METRON-1061 Add FUZZY_SCORE STELLAR function

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/667
  
I'm fine with this so far, but will hold for @nickwallen 's +1. :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron pull request #695: METRON-1108: Metron configuration tabs are not sho...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/695


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/681
  
Just thinking, we might want +Infinity and -Infinity too as they're part of 
the floating point standard.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #681: METRON-1079 Add NaN as a keyword in STELLAR language

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/metron/pull/681
  
Some of our math functions return `NaN` (e.g. `SQRT(-1)`) and being able to 
do `!= NaN` is a useful thing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] metron issue #696: METRON-1109: Indexing topology fails to start after Kerbe...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/696
  
+1 by inspection


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---