dmitrykoval commented on code in PR #240:
URL: https://github.com/apache/parquet-format/pull/240#discussion_r1773933397
##########
src/main/thrift/parquet.thrift:
##########
@@ -237,6 +237,98 @@ struct SizeStatistics {
3: optional list<i64> definition_level_histogram;
}
+/**
+ * Interpretation for edges of GEOMETRY logical type, i.e. whether the edge
+ * between points represent a straight cartesian line or the shortest line on
+ * the sphere. It applies to all non-point geometry objects.
+ */
+enum Edges {
+ PLANAR = 0;
+ SPHERICAL = 1;
+}
+
+/**
+ * A custom binary-encoded polygon or multi-polygon to represent a covering of
+ * geometries. For example, it may be a bounding box or an envelope of
geometries
+ * when a bounding box cannot be built (e.g. a geometry has spherical edges,
or if
+ * an edge of geographic coordinates crosses the antimeridian). In addition,
it can
+ * also be used to provide vendor-agnostic coverings like S2 or H3 grids.
+ */
+struct Covering {
+ /**
+ * A type of covering. Currently accepted values: "WKB".
+ */
+ 1: required string kind;
+ /**
+ * A payload specific to kind. Below are the supported values:
+ * - WKB: well-known binary of a POLYGON or MULTI-POLYGON that completely
+ * covers the contents. This will be interpreted according to the same CRS
+ * and edges defined by the logical type.
+ */
+ 2: required binary value;
+}
+
+/**
+ * Bounding box of geometries in the representation of min/max value pair of
+ * coordinates from each axis. Values of Z and M are omitted for 2D geometries.
+ * Filter pushdown on geometries are only safe for planar spatial predicate
+ * but it is recommended that the writer always generates bounding box
statistics,
+ * regardless of whether the geometries are planar or spherical.
+ */
+struct BoundingBox {
Review Comment:
@szehon-ho In our last discussion I suggested that having numeric min/max
values for spherical model is still better than having no stats at all. The
limitations discussed by @desruisseaux and @mkaravel make sense. From the
query engine's perspective, we can still perform pruning, but not only when
specific conditions are met, for example when: stored objects do not change
meridian signs, there are no poles intersections, etc. In these cases, we can
take arc curvature into account when evaluating spatial predicates and skip
processing of files/rowgroups where no ambiguity exists.
To summarize, min/max stats still make sense compared to no stats, but there
are certainly better alternatives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
