[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/516 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r111098953
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
--- End diff --
Hi @mattf-horton , with the above change where we check for `strings.get(0)
== null || strings.get(0).toString().length() `, the following test results in
a failure:
`Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));`
Pardon my limited understanding, but this is what I inferred:
- The `if` condition for null and 0 length check occurs twice.
- In the case of the `msg` test case where we check for `\n\r`, the initial
contents of the `strings` is null and the variable substitution for `msg`
occurs subsequently. So, this would end up failing for the 1st check.
Due to this, I will modify the checks to only verify for `strings == null`
and `string.size() == 0`. Please let me know if there is a better way this can
be done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r111076022
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
--- End diff --
Thanks, made this change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/516#discussion_r111075935 --- Diff: metron-platform/metron-common/README.md --- @@ -167,6 +172,14 @@ The `!=` operator is the negation of the above. | [ `WEEK_OF_YEAR`](#week_of_year) | | [ `YEAR`](#year) | +### `APPEND_IF_MISSING` + * Description: Appends the suffix to the end of the string if the string does not already end with any of the suffixes. + * Input: +* string - The string to be appended. +* suffix - The string suffix to append to the end of the string. +* suffixes - Optional - Additional string suffixes that are valid terminators. --- End diff -- Okay, I will retain additionalsuffix for now. Created METRON-847 to track the enhancement. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/516#discussion_r111012694 --- Diff: metron-platform/metron-common/README.md --- @@ -167,6 +172,14 @@ The `!=` operator is the negation of the above. | [ `WEEK_OF_YEAR`](#week_of_year) | | [ `YEAR`](#year) | +### `APPEND_IF_MISSING` + * Description: Appends the suffix to the end of the string if the string does not already end with any of the suffixes. + * Input: +* string - The string to be appended. +* suffix - The string suffix to append to the end of the string. +* suffixes - Optional - Additional string suffixes that are valid terminators. --- End diff -- Ah, got it. The documentation you point at for apache commons PrependIfMission method actually uses signature: ``` CharSequence... suffixes ``` so it is a varargs usage. You can do the same with Stellar, or we have found it better to allow a list of strings instead (which you would explode into the calling varargs in the java code). Using a list instead of varargs makes it much easier to extend the function later if we choose. It's up to you whether to do this now. We are trying to close a RC, so there is some time pressure. Maybe you should just open a jira to enhance this later with a list arg. The current implementation of your call to PrependIfMissing is such that the "additionalsuffix" description you suggest would be correct. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r111010577
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
--- End diff --
I think you need
```
if (strings == null || strings.size() == 0 || strings.get(0) == null ||
strings.get(0).length == 0) { ...
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110995412
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
--- End diff --
Do you think adding a check for null as follows, would be the right way?
```
if(strings == null || strings.size() == 0) {
throw new IllegalArgumentException("[CHOP] missing argument: string to
be chopped");
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/516#discussion_r110992116 --- Diff: metron-platform/metron-common/README.md --- @@ -167,6 +172,14 @@ The `!=` operator is the negation of the above. | [ `WEEK_OF_YEAR`](#week_of_year) | | [ `YEAR`](#year) | +### `APPEND_IF_MISSING` + * Description: Appends the suffix to the end of the string if the string does not already end with any of the suffixes. + * Input: +* string - The string to be appended. +* suffix - The string suffix to append to the end of the string. +* suffixes - Optional - Additional string suffixes that are valid terminators. --- End diff -- Hi @mattf-horton , I was trying to following the same wording as the [apache commons documentation](https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/StringUtils.html) for PrependIfMissing, which uses the plural form. Let me know if it would be better if I changed the text as follows: `* suffix - The string suffix to append to the end of the string.` `* additionalsuffix - Optional - An additional string suffix that is a valid terminator.` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/516#discussion_r110964577 --- Diff: metron-platform/metron-common/README.md --- @@ -434,6 +466,14 @@ The `!=` operator is the negation of the above. * dateTime - The datetime as a long representing the milliseconds since unix epoch * Returns: The current month (0-based). +### `PREPEND_IF_MISSING` + * Description: Prepends the prefix to the start of the string if the string does not already start with any of the prefixes. + * Input: +* string - The string to be prepended. +* prefix - The string prefix to prepend to the start of the string. +* prefixes - Optional - Additional string prefixes that are valid. --- End diff -- ditto --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/516#discussion_r110964390 --- Diff: metron-platform/metron-common/README.md --- @@ -167,6 +172,14 @@ The `!=` operator is the negation of the above. | [ `WEEK_OF_YEAR`](#week_of_year) | | [ `YEAR`](#year) | +### `APPEND_IF_MISSING` + * Description: Appends the suffix to the end of the string if the string does not already end with any of the suffixes. + * Input: +* string - The string to be appended. +* suffix - The string suffix to append to the end of the string. +* suffixes - Optional - Additional string suffixes that are valid terminators. --- End diff -- Please state the format of "suffixes". Is it a string, a regex, an array of strings, or what? If it is a single string, how does it represent plural suffixes? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110965812
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,120 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOMP"
+ , description = "Removes one newline from end of a String if
it's there, otherwise leave it alone. A newline is \"\\n\", \"\\r\", or
\"\\r\\n\""
+ , params = { "the String to chomp a newline from, may be null"}
+ , returns = "String without newline, null if null String input"
+ )
+ public static class Chomp extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOMP] missing argument:
string to chomp a newline from");
+ }
+
+ String chomp = StringUtils.chomp((String) strings.get(0));
+ return chomp;
+}
+ }
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class Chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPEND_IF_MISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = {
+ "str - The string."
+ , "prefix - The string prefix to prepend to the start of the
string"
+ , "prefixes - Optional - Additional string prefixes that are
valid"
+ }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class PrependIfMissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String prefixed;
+ switch (strings.size()) {
+case 2: prefixed = StringUtils.prependIfMissing((String)
strings.get(0), (String) strings.get(1));
+ break;
+case 3: prefixed = StringUtils.prependIfMissing((String)
strings.get(0), (String) strings.get(1), (String) strings.get(2));
+ break;
+default: throw new IllegalArgumentException("[PREPEND_IF_MISSING]
incorrect arguments: " + strings.toString() + "\nUsage: PREPEND_IF_MISSING
[...]");
+ }
+ return prefixed;
+}
+ }
+
+ @Stellar( name = "APPEND_IF_MISSING"
+ , description = "Appends the suffix to the end of the string if
the string does not already end with any of the suffixes"
+ , params = {
+ "str - The string."
+ , "suffix - The string suffix to append to the end of the string"
+ , "suffixes - Optional - Additional string suffixes that are
valid terminators"
--- End diff --
Same doc concern as in the .md file: Is "suffixes" a string, a regex, an
array of strings, or what? If it is a single string, how does it represent
plural suffixes?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110965525
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
--- End diff --
Any thoughts about why treat nulls and empty strings differently?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110672618
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
--- End diff --
Okay, thanks. I stand corrected about the URI/URL.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110629490
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
--- End diff --
That is not to say the CHOP cannot be useful however
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110629309
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
--- End diff --
If we are going to do URL/URI work, we should create functions off of the
URI / URL classes and use their specific parsing for uri named component parts
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110584819
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
--- End diff --
IMHO the `chop` function would be helpful when fine-tuning string
extraction operations through stellar (E.g. when extracting specific variations
of hostnames or URLs).
Aside of that, I went ahead and added the `chomp` string function as well
since I agree that it would be of use.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110582745
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
--- End diff --
Sure, done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110582764
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
--- End diff --
Sure, changed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110582728
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class prependifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String prefixed;
+ switch (strings.size()) {
+case 2: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1));
--- End diff --
Changed the import statement to use lang3.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110581823
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class prependifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String prefixed;
+ switch (strings.size()) {
+case 2: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1));
+ break;
+case 3: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1), (String) strings.get(2));
+ break;
+default: throw new IllegalArgumentException("[PREPENDIFMISSING]
incorrect arguments. Usage: PREPENDIFMISSING [...]");
--- End diff --
Now included the arguments provided by user as well, apart from Usage info.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110581688
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
---
@@ -207,4 +207,78 @@ public void testFormatWithNoArguments() throws
Exception {
public void testFormatWithMissingArguments() throws Exception {
run("FORMAT('missing arg: %d')", Collections.emptyMap());
}
+
+
+ /**
+ * CHOP StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testChop() throws Exception {
+Assert.assertEquals("ab", run("CHOP('abc')", new HashedMap()));
+Assert.assertEquals(null, run("CHOP(null)", new HashedMap()));
+Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));
+Assert.assertEquals("", run("CHOP(msg)", ImmutableMap.of("msg",
"\n")));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testChopWithMissingArguments() throws Exception {
+run("CHOP()", Collections.emptyMap());
+ }
+
+ /**
+ * PREPENDIFMISSING StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testPrependIfMissing() throws Exception {
+Assert.assertEquals("xyzabc", run("PREPENDIFMISSING('abc',
'xyz')", new HashedMap()));
+Assert.assertEquals("xyzXYZabc", run("PREPENDIFMISSING('XYZabc',
'xyz', 'mno')", new HashedMap()));
--- End diff --
Done, added.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110581713
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class prependifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String prefixed;
+ switch (strings.size()) {
+case 2: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1));
+ break;
+case 3: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1), (String) strings.get(2));
+ break;
+default: throw new IllegalArgumentException("[PREPENDIFMISSING]
incorrect arguments. Usage: PREPENDIFMISSING [...]");
+ }
+ return prefixed;
+}
+ }
+
+ @Stellar( name = "APPENDIFMISSING"
+ , description = "Appends the suffix to the end of the string if
the string does not already end with any of the suffixes"
+ , params = { "str - The string.", "suffix - The suffix to append
to the end of the string", "suffixes - Additional suffixes that are valid
terminators" }
+ , returns = "A new String if suffix was appended, the same
string otherwise."
+ )
+ public static class appendifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String suffixed;
+ switch (strings.size()) {
+case 2:
+ suffixed =
org.apache.commons.lang3.StringUtils.appendIfMissing((String) strings.get(0),
(String) strings.get(1));
+ break;
+case 3:
+ suffixed =
org.apache.commons.lang3.StringUtils.appendIfMissing((String) strings.get(0),
(String) strings.get(1), (String) strings.get(2));
+ break;
+default:
+ throw new IllegalArgumentException("[APPENDIFMISSING] incorrect
arguments. Usage: APPENDIFMISSING [...]");
+ }
+ return suffixed;
+}
+ }
+
+ @Stellar( name = "COUNTMATCHES"
+ , description = "Counts how many times the substring appears in
the larger string"
+ , params = { "str - the CharSequence to check, may be null",
"sub - the substring to count, may be null"}
+ , returns = "the number of occurrences, 0 if either CharSequence
is null"
--- End diff --
Corrected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user anandsubbu commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110581665
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
---
@@ -207,4 +207,78 @@ public void testFormatWithNoArguments() throws
Exception {
public void testFormatWithMissingArguments() throws Exception {
run("FORMAT('missing arg: %d')", Collections.emptyMap());
}
+
+
+ /**
+ * CHOP StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testChop() throws Exception {
+Assert.assertEquals("ab", run("CHOP('abc')", new HashedMap()));
+Assert.assertEquals(null, run("CHOP(null)", new HashedMap()));
+Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));
+Assert.assertEquals("", run("CHOP(msg)", ImmutableMap.of("msg",
"\n")));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testChopWithMissingArguments() throws Exception {
+run("CHOP()", Collections.emptyMap());
+ }
+
+ /**
+ * PREPENDIFMISSING StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testPrependIfMissing() throws Exception {
+Assert.assertEquals("xyzabc", run("PREPENDIFMISSING('abc',
'xyz')", new HashedMap()));
+Assert.assertEquals("xyzXYZabc", run("PREPENDIFMISSING('XYZabc',
'xyz', 'mno')", new HashedMap()));
+Assert.assertEquals(null, run("PREPENDIFMISSING(null, null,
null)", new HashedMap()));
+Assert.assertEquals("xyz",run("PREPENDIFMISSING('', 'xyz',
null)", new HashedMap()));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testPrependIfMissingWithIncorrectArgs() throws Exception {
--- End diff --
Yup, you're right. Added try/catch blocks now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user cestella commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110431257
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
---
@@ -207,4 +207,78 @@ public void testFormatWithNoArguments() throws
Exception {
public void testFormatWithMissingArguments() throws Exception {
run("FORMAT('missing arg: %d')", Collections.emptyMap());
}
+
+
+ /**
+ * CHOP StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testChop() throws Exception {
+Assert.assertEquals("ab", run("CHOP('abc')", new HashedMap()));
+Assert.assertEquals(null, run("CHOP(null)", new HashedMap()));
+Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));
--- End diff --
Because it's coming in through a variable `msg`. The idea being that it
could be existent in a message flowing through the topologies. Stellar can't
represent `\n` and `\r`, but they can get in there honestly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110291263
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
--- End diff --
`PREPEND_IF_MISSING` would be more consistent with usual usage.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110291091
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class prependifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String prefixed;
+ switch (strings.size()) {
+case 2: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1));
+ break;
+case 3: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1), (String) strings.get(2));
+ break;
+default: throw new IllegalArgumentException("[PREPENDIFMISSING]
incorrect arguments. Usage: PREPENDIFMISSING [...]");
--- End diff --
@cestella , isn't there boilerplate in the Stellar executor that
automagically dumps the @Stellar declaration on any error? So one doesn't have
to implement `Usage:` in the operator?
@anandsubbu , but you can still provide useful contextual info, like what
arguments were actually provided, or in this case, at least how many (incorrect
number) of arguments were provided.
The issues raised in PREPENDIFMISSING also apply to APPENDIFMISSING, of
course.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110270759
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
--- End diff --
Seems like if an empty string gives IAE, then so should a null argument.
But I could also support returning the empty string for an empty string, and
null for null.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110294160
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
---
@@ -207,4 +207,78 @@ public void testFormatWithNoArguments() throws
Exception {
public void testFormatWithMissingArguments() throws Exception {
run("FORMAT('missing arg: %d')", Collections.emptyMap());
}
+
+
+ /**
+ * CHOP StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testChop() throws Exception {
+Assert.assertEquals("ab", run("CHOP('abc')", new HashedMap()));
+Assert.assertEquals(null, run("CHOP(null)", new HashedMap()));
+Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));
+Assert.assertEquals("", run("CHOP(msg)", ImmutableMap.of("msg",
"\n")));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testChopWithMissingArguments() throws Exception {
+run("CHOP()", Collections.emptyMap());
+ }
+
+ /**
+ * PREPENDIFMISSING StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testPrependIfMissing() throws Exception {
+Assert.assertEquals("xyzabc", run("PREPENDIFMISSING('abc',
'xyz')", new HashedMap()));
+Assert.assertEquals("xyzXYZabc", run("PREPENDIFMISSING('XYZabc',
'xyz', 'mno')", new HashedMap()));
+Assert.assertEquals(null, run("PREPENDIFMISSING(null, null,
null)", new HashedMap()));
+Assert.assertEquals("xyz",run("PREPENDIFMISSING('', 'xyz',
null)", new HashedMap()));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testPrependIfMissingWithIncorrectArgs() throws Exception {
+run("PREPENDIFMISSING()", Collections.emptyMap());
+run("PREPENDIFMISSING('abc')", Collections.emptyMap());
+run("PREPENDIFMISSING('abc', 'def', 'ghi', 'jkl')",
Collections.emptyMap());
+ }
+
+ /**
+ * APPENDIFMISSING StringFunction
--- End diff --
Same issues here of course, and in COUNTMATCHES.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110290675
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class prependifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String prefixed;
+ switch (strings.size()) {
+case 2: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1));
--- End diff --
It's not a bad thing to give the FQ class name for StringUtils -- I
initially thought line 361 referred to
metron-common/src/main/java/org/apache/metron/common/utils/StringUtils.java,
and this makes clear it doesn't. But why use commons.lang.StringUtils in line
361 and commons.lang3.StringUtils here? Shouldn't they all just use lang3?
And you should *either* import the library and refer to it by short name,
or not import it and refer to it by FQ class name, I think.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110294012
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
---
@@ -207,4 +207,78 @@ public void testFormatWithNoArguments() throws
Exception {
public void testFormatWithMissingArguments() throws Exception {
run("FORMAT('missing arg: %d')", Collections.emptyMap());
}
+
+
+ /**
+ * CHOP StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testChop() throws Exception {
+Assert.assertEquals("ab", run("CHOP('abc')", new HashedMap()));
+Assert.assertEquals(null, run("CHOP(null)", new HashedMap()));
+Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));
+Assert.assertEquals("", run("CHOP(msg)", ImmutableMap.of("msg",
"\n")));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testChopWithMissingArguments() throws Exception {
+run("CHOP()", Collections.emptyMap());
+ }
+
+ /**
+ * PREPENDIFMISSING StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testPrependIfMissing() throws Exception {
+Assert.assertEquals("xyzabc", run("PREPENDIFMISSING('abc',
'xyz')", new HashedMap()));
+Assert.assertEquals("xyzXYZabc", run("PREPENDIFMISSING('XYZabc',
'xyz', 'mno')", new HashedMap()));
+Assert.assertEquals(null, run("PREPENDIFMISSING(null, null,
null)", new HashedMap()));
+Assert.assertEquals("xyz",run("PREPENDIFMISSING('', 'xyz',
null)", new HashedMap()));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testPrependIfMissingWithIncorrectArgs() throws Exception {
--- End diff --
I may be wrong, but I think JUnit expected errors do not auto-continue
inside an errored test case. So, you're only testing the first sub-case.
To fix, you either need 3 test cases, or you need to do try/catch/finally
contexts around each `run()`, where the catch eats the exception but sets a
marker flag, and the finally asserts the marker flag.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110286424
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
--- End diff --
With all multi-param Stellar functions, please format the inputs one per
line following the pattern of, eg,
[here](https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/DataStructureFunctions.java#L57-L59)
Also, if the third argument is optional, please note that parenthetically.
And what is the type of `prefixes`? String with a regex? List of Strings?
Clarifying this will also make clear that `prefixes` is *not* a varargs
sequence. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110273359
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
--- End diff --
What is the expected use of `chop` in Metron? Would `chomp` be more
useful? Would just like to know your thoughts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110293191
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
---
@@ -207,4 +207,78 @@ public void testFormatWithNoArguments() throws
Exception {
public void testFormatWithMissingArguments() throws Exception {
run("FORMAT('missing arg: %d')", Collections.emptyMap());
}
+
+
+ /**
+ * CHOP StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testChop() throws Exception {
+Assert.assertEquals("ab", run("CHOP('abc')", new HashedMap()));
+Assert.assertEquals(null, run("CHOP(null)", new HashedMap()));
+Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));
--- End diff --
Why does this work, when the string "abc\r\n" is never processed by any of
the shell or java string input processors that might convert it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110293542
--- Diff:
metron-platform/metron-common/src/test/java/org/apache/metron/common/dsl/functions/StringFunctionsTest.java
---
@@ -207,4 +207,78 @@ public void testFormatWithNoArguments() throws
Exception {
public void testFormatWithMissingArguments() throws Exception {
run("FORMAT('missing arg: %d')", Collections.emptyMap());
}
+
+
+ /**
+ * CHOP StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testChop() throws Exception {
+Assert.assertEquals("ab", run("CHOP('abc')", new HashedMap()));
+Assert.assertEquals(null, run("CHOP(null)", new HashedMap()));
+Assert.assertEquals("abc", run("CHOP(msg)", ImmutableMap.of("msg",
"abc\r\n")));
+Assert.assertEquals("", run("CHOP(msg)", ImmutableMap.of("msg",
"\n")));
+ }
+
+ @Test(expected = ParseException.class)
+ public void testChopWithMissingArguments() throws Exception {
+run("CHOP()", Collections.emptyMap());
+ }
+
+ /**
+ * PREPENDIFMISSING StringFunction
+ * @throws Exception
+ */
+ @Test
+ public void testPrependIfMissing() throws Exception {
+Assert.assertEquals("xyzabc", run("PREPENDIFMISSING('abc',
'xyz')", new HashedMap()));
+Assert.assertEquals("xyzXYZabc", run("PREPENDIFMISSING('XYZabc',
'xyz', 'mno')", new HashedMap()));
--- End diff --
Please add a case that exercises non-null alt prefix, eg:
```
Assert.assertEquals("mnoXYZabc", run("PREPENDIFMISSING('mnoXYZabc', 'xyz',
'mno')", new HashedMap()));
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user mattf-horton commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110292164
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class prependifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String prefixed;
+ switch (strings.size()) {
+case 2: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1));
+ break;
+case 3: prefixed =
org.apache.commons.lang3.StringUtils.prependIfMissing((String) strings.get(0),
(String) strings.get(1), (String) strings.get(2));
+ break;
+default: throw new IllegalArgumentException("[PREPENDIFMISSING]
incorrect arguments. Usage: PREPENDIFMISSING [...]");
+ }
+ return prefixed;
+}
+ }
+
+ @Stellar( name = "APPENDIFMISSING"
+ , description = "Appends the suffix to the end of the string if
the string does not already end with any of the suffixes"
+ , params = { "str - The string.", "suffix - The suffix to append
to the end of the string", "suffixes - Additional suffixes that are valid
terminators" }
+ , returns = "A new String if suffix was appended, the same
string otherwise."
+ )
+ public static class appendifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ String suffixed;
+ switch (strings.size()) {
+case 2:
+ suffixed =
org.apache.commons.lang3.StringUtils.appendIfMissing((String) strings.get(0),
(String) strings.get(1));
+ break;
+case 3:
+ suffixed =
org.apache.commons.lang3.StringUtils.appendIfMissing((String) strings.get(0),
(String) strings.get(1), (String) strings.get(2));
+ break;
+default:
+ throw new IllegalArgumentException("[APPENDIFMISSING] incorrect
arguments. Usage: APPENDIFMISSING [...]");
+ }
+ return suffixed;
+}
+ }
+
+ @Stellar( name = "COUNTMATCHES"
+ , description = "Counts how many times the substring appears in
the larger string"
+ , params = { "str - the CharSequence to check, may be null",
"sub - the substring to count, may be null"}
+ , returns = "the number of occurrences, 0 if either CharSequence
is null"
--- End diff --
"the number of non-overlapping occurrences..."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/516#discussion_r110292921
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/StringFunctions.java
---
@@ -343,4 +343,89 @@ public Object apply(List args) {
return String.format(format, formatArgs);
}
}
+
+ @Stellar( name="CHOP"
+ , description = "Remove the last character from a String"
+ , params = { "the String to chop last character from, may be
null"}
+ , returns = "String without last character, null if null String
input"
+ )
+ public static class chop extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
+ if(strings.size() == 0) {
+throw new IllegalArgumentException("[CHOP] missing argument:
string to be chopped");
+ }
+
+ String chop = StringUtils.chop((String) strings.get(0));
+ return chop;
+}
+ }
+
+ @Stellar( name = "PREPENDIFMISSING"
+ , description = "Prepends the prefix to the start of the string
if the string does not already start with any of the prefixes"
+ , params = { "str - The string.", "prefix - The prefix to
prepend to the start of the string", "prefixes - Additional prefixes that are
valid" }
+ , returns = "A new String if prefix was prepended, the same
string otherwise."
+ )
+ public static class prependifmissing extends BaseStellarFunction {
+
+@Override
+public Object apply(List strings) {
+
--- End diff --
Invalid Cast exception?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #516: METRON-830 Adding StringFunctions to Ste...
GitHub user anandsubbu opened a pull request: https://github.com/apache/incubator-metron/pull/516 METRON-830 Adding StringFunctions to Stellar - chop, prependifmissing, appendifmissing and countmatches ## Contributor Comments Added stellar functions and unit tests for the following: chop -> Remove the last character from a String. prependifmissing -> Prepends the prefix to the start of the string if the string does not already start with any of the prefixes. appendifmissing -> Appends the suffix to the end of the string if the string does not already end with any of the suffixes. countmatches -> Counts how many times the substring appears in the larger string. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron (Incubating). Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [ ] Have you included steps or a guide to how the change may be verified and tested manually? - [ ] Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via: ``` mvn -q clean integration-test install && build_utils/verify_licenses.sh ``` - [x] Have you written or updated unit tests and or integration tests to verify your changes? - [NA] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book bin/generate-md.sh mvn site:site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommened that [travis-ci](https://travis-ci.org) is set up for your personal repository such that your branches are built there before submitting a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/anandsubbu/incubator-metron METRON-830 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-metron/pull/516.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #516 commit 62d7544e420f0cd067c272c6bacaf53be545b2c9 Author: Anand Subramanian Date: 2017-04-06T19:39:05Z Added string functions and unit tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
