arina-ielchiieva commented on a change in pull request #2051: DRILL-7696: EVF v2 scan schema resolution URL: https://github.com/apache/drill/pull/2051#discussion_r407074984
########## File path: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/scan/v3/schema/ScanSchemaTracker.java ########## @@ -0,0 +1,466 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.drill.exec.physical.impl.scan.v3.schema; + +import org.apache.drill.common.exceptions.CustomErrorContext; +import org.apache.drill.exec.physical.resultSet.ResultSetLoader; +import org.apache.drill.exec.physical.resultSet.impl.ProjectionFilter; +import org.apache.drill.exec.record.metadata.ColumnMetadata; +import org.apache.drill.exec.record.metadata.TupleMetadata; + +/** + * Computes <i>scan output schema</i> from a variety of sources. + * <p> + * The scan operator output schema can be <i>defined</i> or <i>dynamic.</i> + * + * <h4>Defined Schema</h4> + * + * The planner computes a defined schema from metadata, as in a typical + * query engine. A defined schema defines the output schema directly: + * the defined schema <b>is</b> the output schema. Drill's planner does not + * yet support a defined schema, but work is in progress to get there for + * some cases. + * <p> + * With a defined schema, the reader is given a fully-defined schema and + * its job is to produce vectors that match the given schema. (The details + * are handled by the {@link ResultSetLoader}.) + * <p> + * At present, since the planner does not actually provide a defined schema, + * we support it in this class, and verify that the defined schema, if provided, + * exactly matches the names in the project list in the same order. + * + * <h4>Dynamic Schema</h4> + * + * A dynamic schema is one defined at run time: the traditional Drill approach. + * A dynamic schema starts with a <i>projection list</i> : a list of column names + * without types. + * This class converts the project list into a dynamic reader schema which is + * a schema in which each column has the type {@code LATE}, which basically means + * "a type to be named later" by the reader. + * + * <h4>Hybrid Schema</h4> + * + * Some readers support a <i>provided schema</i>, which is an concept similar to, + * but distinct from, a defined schema. The provided schema provides <i>hints</i> + * about a schema. At present, it + * is an extra; not used or understood by the planner. Thus, the projection + * list is independent of the provided schema: the lists may be disjoint. + * <p> + * With a provided schema, the project list defines the output schema. If the + * provided schema provides projected columns, then the provided schema for those + * columns flow to the output schema, just as for a defined schema. Similarly, the + * reader is given a defined schema for those columns. + * <p> + * Where a provided schema differs is that the project list can include columns + * not in the provided schema, such columns act like the dynamic case: the reader + * defines the column type. + * + * <h4>Projection Types</h4> + * + * Drill will pass in a project list which is one of three kinds:: Review comment: ```suggestion * Drill will pass in a project list which is one of three kinds: ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
