etadelta222 opened a new issue #7606:
URL: https://github.com/apache/pinot/issues/7606
I have a date of birth (dob) column in my source postgres table. I'm
creating a realtime table in Pinot and using datatype:timestamp in my schema
but this ends up defaulting the 'null' dob values to '1970-01-01' which is
incorrect. So instead I'm using datatype:string and using the transformConfigs
in table config to convert it.
Below is what I've tried out.
Error on the 'null' values when trying to use the 'todatetime' function.
`,
"ingestionConfig": {
"transformConfigs": [
{
"columnName": "dob",
"transformFunction": "toDateTime(date_of_birth,'yyyy-MM-dd')"
}
]
},`
Throws this error:
`java.lang.IllegalStateException: Caught exception while invoking method:
public static java.lang.String
org.apache.pinot.common.function.scalar.DateTimeFunctions.toDateTime(long,java.lang.String)
with arguments: [null, yyyy-MM-dd]`
Attempted using Groovy but it does not work with Pinot builtin functions.
` "ingestionConfig": {
"transformConfigs": [
{
"columnName": "dob",
"transformFunction": "Groovy({date_of_birth != 'null' ?
toDateTime(date_of_birth,'yyyy-MM-dd'): '1900-01-01'},date_of_birth)"
}
]
},`
Give this error:
`groovy.lang.MissingMethodException: No signature of method:
Script1.toDateTime() is applicable for argument types: (java.lang.Long,
java.lang.String) values: [304992000000, yyyy-MM-dd]`
Request is to add null support udf.
Pinot version: 0.8.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]