Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/1251
### Testing
#### Create a custom Stellar function
See the following for reference -
https://github.com/apache/metron/blob/master/metron-stellar/stellar-common/3rdPartyStellar.md
1. Setup project dirs
```
mkdir custom-stellar && cd custom-stellar
mkdir -p src/main/java/com/thirdparty/stellar
```
1. Create a pom.xml
```
<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
<modelVersion>4.0.0</modelVersion>
<groupId>com.thirdparty</groupId>
<artifactId>stellar-funcs</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Stellar Functions</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>stellar-common</artifactId>
<version>0.6.1</version>
<!-- NOTE: We will want to depend on the deployed common on the
classpath. -->
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
```
1. Create a custom Stellar function
```
package com.thirdparty.stellar;
import org.apache.metron.stellar.common.utils.ConversionUtils;
import org.apache.metron.stellar.dsl.Context;
import org.apache.metron.stellar.dsl.ParseException;
import org.apache.metron.stellar.dsl.Stellar;
import org.apache.metron.stellar.dsl.StellarFunction;
import java.io.IOException;
import java.util.List;
import org.jboss.netty.util.internal.ConversionUtil;
public class StellarCustom {
@Stellar(name = "NOW",
description = "Simple now time",
params = {
"throwException - whether we should throw an exception on close()"
},
returns = "Time in millis"
)
public static class Now implements StellarFunction {
private boolean isInitialized = false;
private boolean throwException = false;
public Object apply(List<Object> args, Context context) throws
ParseException {
if (args.size() == 1) {
throwException = ConversionUtils.convert(args.get(0),
Boolean.class);
}
return System.currentTimeMillis();
}
public void initialize(Context context) {
try {
System.out.println("Initializing function.");
} finally {
isInitialized = true;
}
}
public boolean isInitialized() {
return isInitialized;
}
public void close() throws IOException {
System.out.println("Test function called close()!");
if (throwException) {
NullPointerException cause = new NullPointerException("Don't
point at me!");
throw new IOException("Something icky happened.", cause);
}
}
}
}
```
1. Build it
`mvn clean install`
1. Ship it
`scp custom-stellar/target/stellar-funcs-1.0-SNAPSHOT.jar
root@node1:/tmp/`
1. Setup HDFS stellar
```
sudo -u hdfs hdfs dfs -mkdir /apps/metron/stellar
sudo -u hdfs hdfs dfs -put /tmp/stellar-funcs-1.0-SNAPSHOT.jar
/apps/metron/stellar
sudo -u hdfs hdfs dfs -chown -R metron:metron /apps/metron/stellar
```
1. Setup global config for Stellar HDFS location
```
$METRON_HOME/bin/zk_load_configs.sh -m PULL -o
$METRON_HOME/config/zookeeper -z $ZOOKEEPER -f
vim $METRON_HOME/config/zookeeper/global.json
# add this -> "stellar.function.paths" :
"hdfs://node1:8020/apps/metron/stellar/.*.jar"
# update global.json in ZK
$METRON_HOME/bin/zk_load_configs.sh -m PUSH -i
$METRON_HOME/config/zookeeper -z $ZOOKEEPER
```
1. Setup a parser or enrichment with the new function. (more specific
details to come)
Don't want an exception thrown
`NOW(false)`
Restart parser or enrichment using this function
Should get data still
Stop the topology - should see a notice about shutdown in the logs.
1. Same as before but throw exception on shutdown. (more specific details
to come)
WANT an exception thrown
`NOW(true)`
Restart parser or enrichment using this function
Should get data still
Stop the topology - should see our exception in the logs.
---
