Hi All,
We’re evaluating a expression evaluator for our use case.
*Example Use Case:*
The expressions needs to contain Java specific code for evaluating once and
running the same for every tuple.
For e.g. a POJO has following definition:
public class POJO {
String firstname; // Firstname
String lastname; // Lastname
Date dob; // Date of birth
}
>From this POJO, we need to generate fullname as concatenation of firstname
& lastname and age which will be derived from dob field.
The expressions for those might look like following:
For full name : ${inp.firstname} + “ “ + ${inp.lastname}
For Age : new Date().getYear() - ${inp.dob}.getYear()
Currently, I have a implementation using Janino library for expression
evaluation. Code (ExpressionEvaluator.java) and Test code (Main.java)
attached.
As performance is an important concern, we chose a custom evaluator using
Janino’s fast script evaluator.
*Design of the custom expression evaluator:*
*ExpressionEvaluator class is used for evaluating expressions which takes
multiple parameter object and the result is returned for that expression.*
*The way to reference a variable in an object is ${placeholder.varname}.*
*The variable will be resolved to its accessible variable or getter method
in order. After this the variable can be used as if its a Java variable.*
*ExpressionEvaluator also allows you to set extra imports that needs to be
added over default is java.lang.**
*ExpressionEvaluator needs to be configured with following configurations
as minimal configuration:*
*1. Mapping of input object place holders to it corresponding types.*
* This can be done with setInputObjectPlaceholders method.*
*2. Return type of of expression eveluation.*
*3. Expression to be evaluated. This is a standard java expression except
for referencing the variable inside object JEL syntax needs to be used i.e.
${objectPlaceHolder.varName}*
*Example Use of custom expression evaluator:*
ExpressionEveluator ee = new ExpressionEvaluator();
// Let expression evaluator know what are the object mappings
present in expressions and their class types.
ee.setInputObjectPlaceholders(new String[]{"input"}, new
Class[]{Test.class});
// Generate expression for finding age from Date object.
String expression = "${input.firstname} + \" \" + ${input.lastname}";
ExpressionEvaluator.DataGetter<String> getter4 =
ee.createGetter(expression, String.class);
inp1.firstname = "ABC";
inp1.lastname = "XYZ";
String fullname = getter4.get(inp1);
System.out.println("Fullname is: " + fullname);
*Output:*
Fullname is: ABC XYZ
Can you please suggest for any improvements in this OR is there a better
option to achieve expression evaluation?
Can this code possibly go into Malhar library?
~ Chinmay.
package com.datatorrent.test;
import java.util.Date;
public class Main
{
public static class Test
{
private int a;
public int b;
private Date d;
public String firstname;
public String lastname;
public Test(int a, int b, Date d)
{
this.a = a;
this.b = b;
this.d = d;
}
public int getA()
{
return a;
}
public void setA(int a)
{
this.a = a;
}
public int getB()
{
return b;
}
public void setB(int b)
{
this.b = b;
}
public Date getD()
{
return d;
}
public void setD(Date d)
{
this.d = d;
}
}
public static void main(String[] arg) throws Exception
{
Test inp1 = new Test(1, 2, new Date(1988-1900, 2, 11));
Test inp2 = new Test(3, 4, new Date(1993-1900, 2, 11));
/**
* Test1
*/
ExpressionEvaluator ee = new ExpressionEvaluator();
// Let expression evaluator know if there are other imports that needs to be added other than java.lang.*
ee.addDefaultImports(new String[]{"java.util.Date"});
// Let expression evaluator know what are the object mappings present in expressions and their class types.
ee.setInputObjectPlaceholders(new String[]{"inpA", "inpB"}, new Class[]{Test.class, Test.class});
// Generate expression for finding age from Date object.
String expression = "new Date().getYear() - ${inpA.d}.getYear()";
ExpressionEvaluator.DataGetter<Integer> getter1 = ee.createGetter(expression, Integer.class);
Integer age = getter1.get(inp1, inp2);
System.out.println("Age is: " + age);
/**
* Test2
*/ ee = new ExpressionEvaluator();
ee.setInputObjectPlaceholders(new String[]{"inpA", "inpB"}, new Class[]{Test.class, Test.class});
// Generate expression conditional expression.
expression = "(${inpA.a} * ${inpA.b}) > 0 ? ${inpB.a} : ${inpB.b}";
ExpressionEvaluator.DataGetter<Integer> getter2 = ee.createGetter(expression, Integer.class);
// Positive result
inp1.setA(1);
Integer computation = getter2.get(inp1, inp2);
System.out.println("Computation is: " + computation);
// Negative result
inp1.setA(-1);
computation = getter2.get(inp1, inp2);
System.out.println("Computation is: " + computation);
/**
* Test3
*/
ee = new ExpressionEvaluator();
// Let expression evaluator know if there are other imports that needs to be added other than java.lang.*
ee.addDefaultImports(new String[]{"java.util.Date"});
// Let expression evaluator know what are the object mappings present in expressions and their class types.
ee.setInputObjectPlaceholders(new String[]{"inp"}, new Class[]{Test.class});
// Generate expression for finding age from Date object.
expression = "${inp.d} == null ? new Date() : ${inp.d}";
ExpressionEvaluator.DataGetter<Date> getter3 = ee.createGetter(expression, Date.class);
Date date = getter3.get(inp1);
System.out.println("Date is: " + date);
inp1.setD(null);
date = getter3.get(inp1);
System.out.println("Date is: " + date);
/**
* Test4
*/
ee = new ExpressionEvaluator();
// Let expression evaluator know what are the object mappings present in expressions and their class types.
ee.setInputObjectPlaceholders(new String[]{"input"}, new Class[]{Test.class});
// Generate expression for finding age from Date object.
expression = "${input.firstname} + \" \" + ${input.lastname}";
ExpressionEvaluator.DataGetter<String> getter4 = ee.createGetter(expression, String.class);
inp1.firstname = "ABC";
inp1.lastname = "XYZ";
String fullname = getter4.get(inp1);
System.out.println("Fullname is: " + fullname);
}
}
package com.datatorrent.test;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.HashMap;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.codehaus.commons.compiler.CompilerFactoryFactory;
import org.codehaus.commons.compiler.IScriptEvaluator;
/**
* This class is used for evaluating expressions which takes multiple parameter object and the result is returned for that expression.
*
* The way to reference a variable in an object is ${placeholder.varname}.
* The variable will be resolved to its accessible variable or getter method in order. After this the variable can be used as if its a Java variable.
*
* ExpressionEvaluator also allows you to set extra imports that needs to be added over default is java.lang.*
*
* ExpressionEvaluator needs to be configured with following configurations as minimal configuration:
* 1. Mapping of input object place holders to it corresponding types.
* This can be done with setInputObjectPlaceholders method.
* 2. Return type of of expression eveluation.
* 3. Expression to be evaluated. This is a standard java expression except for referencing the variable inside object JEL syntax needs to be used i.e. ${objectPlaceHolder.varName}
*/
public class ExpressionEvaluator
{
private static final String INP_OBJECT_FUNC_VAR = "obj";
private static final String GET = "get";
private static final String IS = "is";
private IScriptEvaluator se;
private Map<String, Class> placeholderClassMapping = new HashMap<String, Class>();
private Map<String, Integer> placeholderIndexMapping = new HashMap<String, Integer>();
public ExpressionEvaluator() throws Exception
{
se = CompilerFactoryFactory.getDefaultCompilerFactory().newScriptEvaluator();
}
public interface DataGetter<O>
{
O get(Object... obj);
}
public <O> DataGetter<O> createGetter(String expression, Class<?> returnType)
{
Pattern entry = Pattern.compile("\\$\\{(.*?)\\.(.*?)\\}");
Matcher matcher = entry.matcher(expression);
StringBuffer sb = new StringBuffer();
while (matcher.find()) {
String obj = matcher.group(1);
String var = matcher.group(2);
if (placeholderClassMapping.containsKey(obj)) {
matcher.appendReplacement(sb, getObjectJavaExpression(obj, var));
} else {
throw new RuntimeException("Invalid expression: " + matcher.group());
}
}
matcher.appendTail(sb);
String code = getFinalMethodCode(sb.toString(), returnType);
try {
return (DataGetter)se.createFastEvaluator(code, DataGetter.class, new String[]{INP_OBJECT_FUNC_VAR});
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private String getFinalMethodCode(String code, Class<?> returnType)
{
StringBuilder sb = new StringBuilder();
sb.append("if (")
.append(INP_OBJECT_FUNC_VAR)
.append(".length != ")
.append(placeholderClassMapping.size())
.append(") {")
.append("throw new RuntimeException(\"Incorrect number of parameters passed to DataGetter.\");")
.append("}\n")
.append("return (")
.append(returnType.getName())
.append(")(")
.append(code)
.append(");");
return sb.toString();
}
private String getObjectJavaExpression(String obj, String var)
{
Class classType = this.placeholderClassMapping.get(obj);
String replacement = INP_OBJECT_FUNC_VAR + "[" + this.placeholderIndexMapping.get(obj) + "]";
StringBuilder sb = new StringBuilder();
sb.append("(")
.append("(")
.append(classType.getName().replace("$", "\\$"))
.append(")")
.append("(")
.append(replacement)
.append(")")
.append(")")
.append(".")
.append(getGetterForVariable(var, classType));
return sb.toString();
}
private String getGetterForVariable(String var, Class inputClassType)
{
try {
final Field field = inputClassType.getField(var);
if (Modifier.isPublic(field.getModifiers())) {
return var;
}
System.out.println("Field " + var + " is not publicly accessible. Proceeding to locate a getter method.");
} catch (NoSuchFieldException ex) {
System.out.println(inputClassType.getName() + " does not have field " + var + ". Proceeding to locate a getter method.");
} catch (SecurityException ex) {
System.out.println(inputClassType.getName() + " does not have field " + var + ". Proceeding to locate a getter method.");
}
String methodName = GET + var.substring(0, 1).toUpperCase() + var.substring(1);
try {
Method method = inputClassType.getMethod(methodName);
if (Modifier.isPublic(method.getModifiers())) {
return methodName + "()";
}
System.out.println("Method " + methodName + " of " + inputClassType.getName() + " is not accessible. Proceeding to locate another getter method.");
} catch (NoSuchMethodException ex) {
System.out.println(inputClassType.getName() + " does not have method " + methodName + ". Proceeding to locate another getter method");
} catch (SecurityException ex) {
System.out.println(inputClassType.getName() + " does not have method " + methodName + ". Proceeding to locate another getter method");
}
methodName = IS + var.substring(0, 1).toUpperCase() + var.substring(1);
try {
Method method = inputClassType.getMethod(methodName);
if (Modifier.isPublic(method.getModifiers())) {
return methodName + "()";
}
System.out.println("Method " + methodName + " of " + inputClassType.getName() + " is not accessible. Proceeding to locate another getter method.");
} catch (NoSuchMethodException ex) {
System.out.println(inputClassType.getName() + " does not have method " + methodName + ". Proceeding to locate another getter method");
} catch (SecurityException ex) {
System.out.println(inputClassType.getName() + " does not have method " + methodName + ". Proceeding to locate another getter method");
}
return var;
}
public void addDefaultImports(String[] imports)
{
se.setDefaultImports(imports);
}
public void setInputObjectPlaceholders(String[] inputObjectPlaceholders, Class[] classTypes)
{
for (int i = 0; i < inputObjectPlaceholders.length; i++) {
this.placeholderClassMapping.put(inputObjectPlaceholders[i], classTypes[i]);
this.placeholderIndexMapping.put(inputObjectPlaceholders[i], i);
}
}
}
