clintropolis commented on a change in pull request #6360: overhaul
'druid-parquet-extensions' module, promoting from 'contrib' to 'core'
URL: https://github.com/apache/incubator-druid/pull/6360#discussion_r224634435
##########
File path:
extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/AvroFlattenerMaker.java
##########
@@ -50,13 +50,37 @@
private static final EnumSet<Schema.Type> ROOT_TYPES = EnumSet.of(
Schema.Type.STRING,
Schema.Type.BYTES,
- Schema.Type.ARRAY,
Schema.Type.INT,
Schema.Type.LONG,
Schema.Type.FLOAT,
Schema.Type.DOUBLE
);
+ private static boolean isPrimitive(Schema schema)
+ {
+ return ROOT_TYPES.contains(schema.getType());
+ }
+ private static boolean isPrimitiveArray(Schema schema)
+ {
+ return schema.getType().equals(Schema.Type.ARRAY) &&
isPrimitive(schema.getElementType());
+ }
+
+ private static boolean isOptionalPrimitive(Schema schema)
+ {
+ return schema.getType().equals(Schema.Type.UNION) &&
+ schema.getTypes().size() == 2 &&
+ schema.getTypes().get(0).getType().equals(Schema.Type.NULL) &&
Review comment:
Hmm, i'm actually not certain it's guaranteed, from spec
>Unions
Unions, as mentioned above, are represented using JSON arrays. For example,
["null", "string"] declares a schema which may be either a null or string.
>(Note that when a default value is specified for a record field whose type
is a union, the type of the default value must match the first element of the
union. Thus, for unions containing "null", the "null" is usually listed first,
since the default value of such unions is typically null.)
>Unions may not contain more than one schema with the same type, except for
the named types record, fixed and enum. For example, unions containing two
array types or two map types are not permitted, but two types with different
names are permitted. (Names permit efficient resolution when reading and
writing unions.)
>Unions may not immediately contain other unions.
I'm actually not certain what would be best to do in this case, but since we
didn't support nullable fields at all previously afaict maybe it's ok.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
