abhishekrb19 commented on issue #16782: URL: https://github.com/apache/druid/issues/16782#issuecomment-2258980712
Hi @Donutellko, thanks for reporting. IIRC struct and array types weren't fully supported with the upstream Delta Kernel library in 3.0.0 when the extension was originally written. Now that we use Kernel 3.2.0, it seems that support has been added. I will look into adding it in the Druid connector. Re the expected behavior you note: > Expected behavior: > * StructType's StructFields are loaded as a set of columns with a common prefix: `MetaData.FieldOne`, `MetaData.FieldTwo`, ...; > * or (at least) StructType is loaded as a JSON string. > * Additionally, I would like to discuss a possibility of loading delta **ArrayType** as a JSON string. I think the Delta input source should just write structs as json and arrays as arrays. For structs, if a user wants to flatten the fields or extract/transform specific fields present in the struct, it should still be possible to do so using the SQL JSON [functions](https://druid.apache.org/docs/latest/querying/sql-json-functions/) that can be used at ingest and/or query time . For example, `JSON_VALUE("MetaData", '$.FieldOne') AS "Metadata.FieldOne"` will extract `FieldOne` as a separate column in whichever way you'd like. Does that sound good to you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
