[
https://issues.apache.org/jira/browse/METRON-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145317#comment-16145317
]
ASF GitHub Bot commented on METRON-1128:
----------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/714
> MAAS_GET_ENDPOINT - Unable to resolve function error seen for squid example
> ---------------------------------------------------------------------------
>
> Key: METRON-1128
> URL: https://issues.apache.org/jira/browse/METRON-1128
> Project: Metron
> Issue Type: Bug
> Reporter: Anand Subramanian
> Assignee: Casey Stella
>
> While attempting to run the the DGA MaaS
> [example|https://github.com/apache/metron/tree/master/metron-analytics/metron-maas-service#example]
> with squid, a ParseException is seen saying it could not resolve the
> 'MAAS_GET_ENDPOINT' function. Full exception is pasted at the bottom of the
> description.
> I confirmed that the metron-management jar is present under $METRON_HOME/lib.
> Also, the MAAS functions are listed when I checked on the stellar REPL.
> [Stellar]>>> ?MAAS_
> ?MAAS_GET_ENDPOINT ?MAAS_MODEL_APPLY
> I have tried to kill and start the squid topology, but it did not resolve the
> issue.
> Full exception:
> {code}
> 017-08-23 09:11:01.113 o.a.s.d.executor [ERROR]
> java.lang.IllegalStateException: Unable to process transformation:
> MAP_GET('is_malicious', MAAS_MODEL_APPLY(MAAS_GET_ENDPOINT('dga'), {'host' :
> domain_without_subdomains})) for is_malicious because Unable to resolve
> function named 'MAAS_GET_ENDPOINT'. Valid functions are
> IS_DATE,ZIP,IS_DOMAIN,MONTH,FORMAT,TRIM,SQRT,WEEK_OF_MONTH,JOIN,MAP_GET,LOG10,FLOOR,TO_INTEGER,YEAR,OUTLIER_MAD_SCORE,WEEK_OF_YEAR,PROTOCOL_TO_NAME,GET_FIRST,FILL_LEFT,STATS_MAX,CHOP,DECODE,PROFILE_FIXED,DOMAIN_REMOVE_SUBDOMAINS,STATS_SUM_SQUARES,REGEXP_GROUP_VAL,STATS_INIT,ENRICHMENT_GET,STATS_SD,HLLP_INIT,GEO_GET,GET,STATS_COUNT,IS_INTEGER,DAY_OF_WEEK,MAP,ENCODE,IS_EMPTY,DAY_OF_YEAR,MAP_EXISTS,TO_LOWER,STATS_SKEWNESS,ENRICHMENT_EXISTS,DAY_OF_MONTH,LENGTH,IN_SUBNET,SPLIT,PROFILER_APPLY,ABS,STATS_VARIANCE,STATS_ADD,PROFILER_FLUSH,TO_UPPER,OUTLIER_MAD_STATE_MERGE,TO_EPOCH_TIMESTAMP,BLOOM_EXISTS,GET_SUPPORTED_ENCODINGS,COS,BLOOM_INIT,URL_TO_PATH,OUTLIER_MAD_ADD,URL_TO_PROTOCOL,TO_FLOAT,STATS_SUM_LOGS,IS_IP,TO_LONG,STATS_MEAN,PROFILE_GET,CHOMP,TAN,LN,LIST_ADD,PROFILER_INIT,ENDS_WITH,DOMAIN_TO_TLD,COUNT_MATCHES,BYTEARRAY_MATCHER,TO_STRING,HLLP_MERGE,HLLP_CARDINALITY,APPEND_IF_MISSING,BLOOM_MERGE,STARTS_WITH,STATS_KURTOSIS,DOMAIN_REMOVE_TLD,LOG2,SYSTEM_ENV_GET,CEILING,HLLP_ADD,SIN,STATS_BIN,REDUCE,IS_EMAIL,REGEXP_MATCH,PROFILE_WINDOW,URL_TO_PORT,ROUND,GET_LAST,FILL_RIGHT,STATS_POPULATION_VARIANCE,SYSTEM_PROPERTY_GET,IS_ENCODING,FILTER,BIN,BLOOM_ADD,STATS_QUADRATIC_MEAN,STATS_GEOMETRIC_MEAN,STATS_MIN,PREPEND_IF_MISSING,URL_TO_HOST,ZIP_LONGEST,IS_URL,STATS_SUM,STATS_MERGE,STATS_PERCENTILE,EXP,STRING_ENTROPY,TO_DOUBLE
> at
> org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:61)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.configuration.FieldTransformer.transform(FieldTransformer.java:108)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.configuration.FieldTransformer.transformAndUpdate(FieldTransformer.java:120)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:190)
> [stormjar.jar:?]
> at
> org.apache.storm.daemon.executor$fn__6573$tuple_action_fn__6575.invoke(executor.clj:734)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__6494.invoke(executor.clj:466)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.disruptor$clojure_handler$reify__6007.onEvent(disruptor.clj:40)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.daemon.executor$fn__6573$fn__6586$fn__6639.invoke(executor.clj:853)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
> Caused by: org.apache.metron.stellar.dsl.ParseException: Unable to resolve
> function named 'MAAS_GET_ENDPOINT'. Valid functions are
> IS_DATE,ZIP,IS_DOMAIN,MONTH,FORMAT,TRIM,SQRT,WEEK_OF_MONTH,JOIN,MAP_GET,LOG10,FLOOR,TO_INTEGER,YEAR,OUTLIER_MAD_SCORE,WEEK_OF_YEAR,PROTOCOL_TO_NAME,GET_FIRST,FILL_LEFT,STATS_MAX,CHOP,DECODE,PROFILE_FIXED,DOMAIN_REMOVE_SUBDOMAINS,STATS_SUM_SQUARES,REGEXP_GROUP_VAL,STATS_INIT,ENRICHMENT_GET,STATS_SD,HLLP_INIT,GEO_GET,GET,STATS_COUNT,IS_INTEGER,DAY_OF_WEEK,MAP,ENCODE,IS_EMPTY,DAY_OF_YEAR,MAP_EXISTS,TO_LOWER,STATS_SKEWNESS,ENRICHMENT_EXISTS,DAY_OF_MONTH,LENGTH,IN_SUBNET,SPLIT,PROFILER_APPLY,ABS,STATS_VARIANCE,STATS_ADD,PROFILER_FLUSH,TO_UPPER,OUTLIER_MAD_STATE_MERGE,TO_EPOCH_TIMESTAMP,BLOOM_EXISTS,GET_SUPPORTED_ENCODINGS,COS,BLOOM_INIT,URL_TO_PATH,OUTLIER_MAD_ADD,URL_TO_PROTOCOL,TO_FLOAT,STATS_SUM_LOGS,IS_IP,TO_LONG,STATS_MEAN,PROFILE_GET,CHOMP,TAN,LN,LIST_ADD,PROFILER_INIT,ENDS_WITH,DOMAIN_TO_TLD,COUNT_MATCHES,BYTEARRAY_MATCHER,TO_STRING,HLLP_MERGE,HLLP_CARDINALITY,APPEND_IF_MISSING,BLOOM_MERGE,STARTS_WITH,STATS_KURTOSIS,DOMAIN_REMOVE_TLD,LOG2,SYSTEM_ENV_GET,CEILING,HLLP_ADD,SIN,STATS_BIN,REDUCE,IS_EMAIL,REGEXP_MATCH,PROFILE_WINDOW,URL_TO_PORT,ROUND,GET_LAST,FILL_RIGHT,STATS_POPULATION_VARIANCE,SYSTEM_PROPERTY_GET,IS_ENCODING,FILTER,BIN,BLOOM_ADD,STATS_QUADRATIC_MEAN,STATS_GEOMETRIC_MEAN,STATS_MIN,PREPEND_IF_MISSING,URL_TO_HOST,ZIP_LONGEST,IS_URL,STATS_SUM,STATS_MERGE,STATS_PERCENTILE,EXP,STRING_ENTROPY,TO_DOUBLE
> at
> org.apache.metron.stellar.common.StellarCompiler.resolveFunction(StellarCompiler.java:588)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:551)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:160)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:152)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:49)
> ~[stormjar.jar:?]
> ... 13 more
> Caused by: java.lang.IllegalStateException: Unknown function:
> `MAAS_GET_ENDPOINT`
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:106)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:47)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler.resolveFunction(StellarCompiler.java:583)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:551)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:160)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:152)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:49)
> ~[stormjar.jar:?]
> ... 13 more
> 2017-08-23 09:11:01.186 o.a.s.d.executor [ERROR]
> java.lang.IllegalStateException: Unable to process transformation:
> MAP_GET('is_malicious', MAAS_MODEL_APPLY(MAAS_GET_ENDPOINT('dga'), {'host' :
> domain_without_subdomains})) for is_malicious because Unable to resolve
> function named 'MAAS_GET_ENDPOINT'. Valid functions are
> IS_DATE,ZIP,IS_DOMAIN,MONTH,FORMAT,TRIM,SQRT,WEEK_OF_MONTH,JOIN,MAP_GET,LOG10,FLOOR,TO_INTEGER,YEAR,OUTLIER_MAD_SCORE,WEEK_OF_YEAR,PROTOCOL_TO_NAME,GET_FIRST,FILL_LEFT,STATS_MAX,CHOP,DECODE,PROFILE_FIXED,DOMAIN_REMOVE_SUBDOMAINS,STATS_SUM_SQUARES,REGEXP_GROUP_VAL,STATS_INIT,ENRICHMENT_GET,STATS_SD,HLLP_INIT,GEO_GET,GET,STATS_COUNT,IS_INTEGER,DAY_OF_WEEK,MAP,ENCODE,IS_EMPTY,DAY_OF_YEAR,MAP_EXISTS,TO_LOWER,STATS_SKEWNESS,ENRICHMENT_EXISTS,DAY_OF_MONTH,LENGTH,IN_SUBNET,SPLIT,PROFILER_APPLY,ABS,STATS_VARIANCE,STATS_ADD,PROFILER_FLUSH,TO_UPPER,OUTLIER_MAD_STATE_MERGE,TO_EPOCH_TIMESTAMP,BLOOM_EXISTS,GET_SUPPORTED_ENCODINGS,COS,BLOOM_INIT,URL_TO_PATH,OUTLIER_MAD_ADD,URL_TO_PROTOCOL,TO_FLOAT,STATS_SUM_LOGS,IS_IP,TO_LONG,STATS_MEAN,PROFILE_GET,CHOMP,TAN,LN,LIST_ADD,PROFILER_INIT,ENDS_WITH,DOMAIN_TO_TLD,COUNT_MATCHES,BYTEARRAY_MATCHER,TO_STRING,HLLP_MERGE,HLLP_CARDINALITY,APPEND_IF_MISSING,BLOOM_MERGE,STARTS_WITH,STATS_KURTOSIS,DOMAIN_REMOVE_TLD,LOG2,SYSTEM_ENV_GET,CEILING,HLLP_ADD,SIN,STATS_BIN,REDUCE,IS_EMAIL,REGEXP_MATCH,PROFILE_WINDOW,URL_TO_PORT,ROUND,GET_LAST,FILL_RIGHT,STATS_POPULATION_VARIANCE,SYSTEM_PROPERTY_GET,IS_ENCODING,FILTER,BIN,BLOOM_ADD,STATS_QUADRATIC_MEAN,STATS_GEOMETRIC_MEAN,STATS_MIN,PREPEND_IF_MISSING,URL_TO_HOST,ZIP_LONGEST,IS_URL,STATS_SUM,STATS_MERGE,STATS_PERCENTILE,EXP,STRING_ENTROPY,TO_DOUBLE
> at
> org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:61)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.configuration.FieldTransformer.transform(FieldTransformer.java:108)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.configuration.FieldTransformer.transformAndUpdate(FieldTransformer.java:120)
> ~[stormjar.jar:?]
> at
> org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:190)
> [stormjar.jar:?]
> at
> org.apache.storm.daemon.executor$fn__6573$tuple_action_fn__6575.invoke(executor.clj:734)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.daemon.executor$mk_task_receiver$fn__6494.invoke(executor.clj:466)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.disruptor$clojure_handler$reify__6007.onEvent(disruptor.clj:40)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at
> org.apache.storm.daemon.executor$fn__6573$fn__6586$fn__6639.invoke(executor.clj:853)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484)
> [storm-core-1.0.1.2.5.6.0-40.jar:1.0.1.2.5.6.0-40]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
> Caused by: org.apache.metron.stellar.dsl.ParseException: Unable to resolve
> function named 'MAAS_GET_ENDPOINT'. Valid functions are
> IS_DATE,ZIP,IS_DOMAIN,MONTH,FORMAT,TRIM,SQRT,WEEK_OF_MONTH,JOIN,MAP_GET,LOG10,FLOOR,TO_INTEGER,YEAR,OUTLIER_MAD_SCORE,WEEK_OF_YEAR,PROTOCOL_TO_NAME,GET_FIRST,FILL_LEFT,STATS_MAX,CHOP,DECODE,PROFILE_FIXED,DOMAIN_REMOVE_SUBDOMAINS,STATS_SUM_SQUARES,REGEXP_GROUP_VAL,STATS_INIT,ENRICHMENT_GET,STATS_SD,HLLP_INIT,GEO_GET,GET,STATS_COUNT,IS_INTEGER,DAY_OF_WEEK,MAP,ENCODE,IS_EMPTY,DAY_OF_YEAR,MAP_EXISTS,TO_LOWER,STATS_SKEWNESS,ENRICHMENT_EXISTS,DAY_OF_MONTH,LENGTH,IN_SUBNET,SPLIT,PROFILER_APPLY,ABS,STATS_VARIANCE,STATS_ADD,PROFILER_FLUSH,TO_UPPER,OUTLIER_MAD_STATE_MERGE,TO_EPOCH_TIMESTAMP,BLOOM_EXISTS,GET_SUPPORTED_ENCODINGS,COS,BLOOM_INIT,URL_TO_PATH,OUTLIER_MAD_ADD,URL_TO_PROTOCOL,TO_FLOAT,STATS_SUM_LOGS,IS_IP,TO_LONG,STATS_MEAN,PROFILE_GET,CHOMP,TAN,LN,LIST_ADD,PROFILER_INIT,ENDS_WITH,DOMAIN_TO_TLD,COUNT_MATCHES,BYTEARRAY_MATCHER,TO_STRING,HLLP_MERGE,HLLP_CARDINALITY,APPEND_IF_MISSING,BLOOM_MERGE,STARTS_WITH,STATS_KURTOSIS,DOMAIN_REMOVE_TLD,LOG2,SYSTEM_ENV_GET,CEILING,HLLP_ADD,SIN,STATS_BIN,REDUCE,IS_EMAIL,REGEXP_MATCH,PROFILE_WINDOW,URL_TO_PORT,ROUND,GET_LAST,FILL_RIGHT,STATS_POPULATION_VARIANCE,SYSTEM_PROPERTY_GET,IS_ENCODING,FILTER,BIN,BLOOM_ADD,STATS_QUADRATIC_MEAN,STATS_GEOMETRIC_MEAN,STATS_MIN,PREPEND_IF_MISSING,URL_TO_HOST,ZIP_LONGEST,IS_URL,STATS_SUM,STATS_MERGE,STATS_PERCENTILE,EXP,STRING_ENTROPY,TO_DOUBLE
> at
> org.apache.metron.stellar.common.StellarCompiler.resolveFunction(StellarCompiler.java:588)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:551)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:160)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:152)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:49)
> ~[stormjar.jar:?]
> ... 13 more
> Caused by: java.lang.IllegalStateException: Unknown function:
> `MAAS_GET_ENDPOINT`
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:106)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.dsl.functions.resolver.BaseFunctionResolver.apply(BaseFunctionResolver.java:47)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler.resolveFunction(StellarCompiler.java:583)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:551)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:160)
> ~[stormjar.jar:?]
> at
> org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:152)
> ~[stormjar.jar:?]
> at
> org.apache.metron.common.field.transformation.StellarTransformation.map(StellarTransformation.java:49)
> ~[stormjar.jar:?]
> ... 13 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
