Dear Community,
We're creating a transform operator which will allow the apex users to
transform data over a stream using this operator.
Use Case:
----
When the data has been received from input source, one can transform data
using transform operator and then dump it to destination.
Transform means:
----
1. Conversion of fields from one type to another. Eg. int to Integer, epoc
to java.util.Date object, int to String or String to int etc.
2. Deriving new fields from one or more input fields.
For eg. Input contains 2 fields viz. "firstname", "lastname" and
derived field is "fullname" which is combination of firstname and lastname
in some way.
Another example is date of birth present in input tuple, generate age
from it.
3. Change in schema from input to output. For eg. Input schema is subset of
output schema and output schema contains some extra derived fields from
input fields.
Functionality:
----
1. Transform operator will access POJO as input tuple and emit another/same
POJO output tuple.
2. Operator needs to be configured with input tuple schema and output tuple
schema for this. This can be done via TUPLE_CLASS attribute on ports.
3. Generation of output tuple from input tuple can be specified using
simple java based expressions.
For eg.
outfield1 = <expression using fields of input POJO>
outfield2 = <expression using fields of output POJO>
4. If no expression is mentioned for a certain output field, then a
matching field (name and type) will be picked input POJO and contents will
be copied to output POJO.
Otherwise, the output field will be left to default empty value.
Expression Support in PojoUtils:
----
1. For achieving above functionality, I thought about extending PojoUtils
which already support much of what is required. Then this utility can be
used in Transform operator.
2. Couple of minor enhancements/fixing issues required to this utility:
a. Multiple POJO support is not required. Hence will stick to single
POJO.
b. PojoUtils does not work well with resolution of getter methods when
more than one fields are mentioned in expression. Will need to fix that.
c. Following interface can be added to PojoUtils. This will be the
return method that can be called for executing the expression
interface Expression<O>
{
O execute(Object obj);
}
d. Following methods can be added to PojoUtils for addressing
expression support:
// Evaluates given expression
Expression evaluateExpression(String expr, Class inputObjectType,
Class returnType);
// Adds a custom method that can be used in an expression.
// The one who's using this library can utility this method to
provide predefined functionality to user.
void registerCustomMethod(String qualifiedFunc)
Please share your valuable inputs on this.
Thanks,
Chinmay.
