This is an automated email from the ASF dual-hosted git repository. twalthr pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git.
from 61e7877 [hotfix][tests] Remove Mocking from TaskLocalStateStoreImplTest new ab090d5 [hotfix][table-runtime] Update copyright for some filesystem classes new 44b2756 [hotfix][dist] flink-json and flink-csv are now declared as dependencies in the flink-dist to enforce the reactor order new 158c68c [hotfix][connectors] Every connector now shades the flink-connector-base in its uber jar new 97237d0 [FLINK-24687][table-common] Fix the Table Factory loading mechanism to tolerate NoClassDefFoundError. Added a test and converted FactoryUtil to use assertj. new d12fd3d [FLINK-24687][table-planner] Remove planner dependency on FileSystemConnectorOptions new 2ae04c2 [FLINK-24687][parquet] Copied DecimalDataUtils#is32BitDecimal and DecimalDataUtils#is32BitDecimal in ParquetSchemaConverter to remove the dependency on DecimalDataUtils (from planner) new 6bb0907 [FLINK-24687][table-runtime] Refactored test csv format to be independent of planner (except ScanRuntimeProviderContext.INSTANCE::createDataStructureConverter) and to implement SerializationSchema more than BulkWriterFormatFactory. Moved to a specific package new 9bbadb9 [FLINK-24687][table][connectors] Move FileSystemTableSink, FileSystemTableSource to flink-connector-files and columnar support to flink-table-common The 8 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/content/docs/connectors/table/filesystem.md | 1 + docs/data/sql_connectors.yml | 46 ++-- .../e577412e-8d38-496c-a680-b842112e4b94 | 1 - .../flink-connector-elasticsearch-base/pom.xml | 5 + flink-connectors/flink-connector-files/pom.xml | 38 ++- .../file/table}/AbstractFileSystemTable.java | 9 +- .../flink/connector/file/table}/BinPacking.java | 18 +- .../connector/file/table}/ColumnarRowIterator.java | 6 +- .../file/table}/ContinuousPartitionFetcher.java | 2 +- .../file/table}/DefaultPartTimeExtractor.java | 4 +- .../file/table}/DeserializationSchemaAdapter.java | 6 +- .../file/table}/DynamicPartitionWriter.java | 2 +- .../file/table}/EmptyMetaStoreFactory.java | 2 +- .../connector/file/table}/EnrichedRowData.java | 2 +- .../file/table}/FileInfoExtractorBulkFormat.java | 6 +- .../connector/file/table}/FileSystemCommitter.java | 6 +- .../file/table}/FileSystemConnectorOptions.java | 2 +- .../connector/file/table}/FileSystemFactory.java | 2 +- .../file/table}/FileSystemOutputFormat.java | 2 +- .../file/table}/FileSystemTableFactory.java | 10 +- .../connector/file/table}/FileSystemTableSink.java | 38 +-- .../file/table}/FileSystemTableSource.java | 25 +- .../file/table}/GroupedPartitionWriter.java | 2 +- .../connector/file/table}/LimitableBulkFormat.java | 4 +- .../file/table}/MetastoreCommitPolicy.java | 6 +- .../connector/file/table}/OutputFormatFactory.java | 2 +- .../file/table}/PartitionCommitPolicy.java | 2 +- .../connector/file/table}/PartitionComputer.java | 2 +- .../connector/file/table}/PartitionFetcher.java | 2 +- .../file/table}/PartitionFieldExtractor.java | 4 +- .../connector/file/table}/PartitionLoader.java | 5 +- .../connector/file/table}/PartitionReader.java | 2 +- .../file/table}/PartitionTempFileManager.java | 2 +- .../file/table}/PartitionTimeExtractor.java | 2 +- .../connector/file/table}/PartitionWriter.java | 2 +- .../file/table}/PartitionWriterFactory.java | 4 +- .../file/table}/ProjectingBulkFormat.java | 2 +- .../file/table}/RowDataPartitionComputer.java | 2 +- .../file/table}/RowPartitionComputer.java | 2 +- .../file/table}/SerializationSchemaAdapter.java | 2 +- .../file/table}/SingleDirectoryWriter.java | 2 +- .../file/table}/SuccessFileCommitPolicy.java | 4 +- .../file/table}/TableMetaStoreFactory.java | 2 +- .../table/factories/BulkReaderFormatFactory.java | 7 +- .../table/factories/BulkWriterFormatFactory.java | 4 +- .../table/factories/FileSystemFormatFactory.java | 3 +- .../file/table}/format/BulkDecodingFormat.java | 4 +- .../table}/stream/AbstractStreamingWriter.java | 2 +- .../file/table}/stream/PartitionCommitInfo.java | 5 +- .../table}/stream/PartitionCommitPredicate.java | 11 +- .../file/table}/stream/PartitionCommitTrigger.java | 11 +- .../file/table}/stream/PartitionCommitter.java | 20 +- .../stream/PartitionTimeCommitPredicate.java | 16 +- .../table}/stream/PartitionTimeCommitTrigger.java | 7 +- .../table}/stream/ProcTimeCommitPredicate.java | 6 +- .../file/table}/stream/ProcTimeCommitTrigger.java | 6 +- .../file/table}/stream/StreamingFileWriter.java | 8 +- .../file/table}/stream/StreamingSink.java | 26 +- .../connector/file/table}/stream/TaskTracker.java | 5 +- .../table}/stream/compact/CompactBucketWriter.java | 4 +- .../table}/stream/compact/CompactBulkReader.java | 4 +- .../file/table}/stream/compact/CompactContext.java | 17 +- .../table}/stream/compact/CompactCoordinator.java | 20 +- .../table}/stream/compact/CompactFileWriter.java | 14 +- .../table}/stream/compact/CompactMessages.java | 4 +- .../table}/stream/compact/CompactOperator.java | 25 +- .../file/table}/stream/compact/CompactReader.java | 18 +- .../file/table}/stream/compact/CompactWriter.java | 18 +- .../compact/FileInputFormatCompactReader.java | 4 +- .../org.apache.flink.table.factories.Factory | 2 +- .../connector/file/table}/BinPackingTest.java | 15 +- .../connector/file/table}/EnrichedRowDataTest.java | 2 +- .../file/table}/FileSystemCommitterTest.java | 2 +- .../file/table}/FileSystemOutputFormatTest.java | 2 +- .../file/table}/LimitableBulkFormatTest.java | 2 +- .../connector/file/table}/PartitionWriterTest.java | 4 +- .../file/table}/RowPartitionComputerTest.java | 2 +- .../table}/stream/StreamingFileWriterTest.java | 12 +- .../stream/compact/AbstractCompactTestBase.java | 2 +- .../stream/compact/CompactCoordinatorTest.java | 16 +- .../stream/compact/CompactFileWriterTest.java | 8 +- .../table}/stream/compact/CompactOperatorTest.java | 10 +- .../file/table}/stream/compact/TestByteFormat.java | 3 +- flink-connectors/flink-connector-hive/pom.xml | 16 +- .../hive/ContinuousHiveSplitEnumerator.java | 2 +- .../connectors/hive}/FileSystemLookupFunction.java | 4 +- .../connectors/hive/HadoopFileSystemFactory.java | 2 +- .../connectors/hive/HiveDynamicTableFactory.java | 6 +- .../connectors/hive/HiveLookupTableSource.java | 13 +- .../hive/HiveRowDataPartitionComputer.java | 2 +- .../connectors/hive/HiveRowPartitionComputer.java | 2 +- .../apache/flink/connectors/hive/HiveSource.java | 2 +- .../flink/connectors/hive/HiveSourceBuilder.java | 14 +- .../connectors/hive/HiveTableMetaStoreFactory.java | 2 +- .../flink/connectors/hive/HiveTableSink.java | 22 +- .../flink/connectors/hive/HiveTableSource.java | 8 +- .../hive/read/HiveCompactReaderFactory.java | 6 +- .../hive/read/HiveContinuousPartitionContext.java | 2 +- .../hive/read/HiveContinuousPartitionFetcher.java | 6 +- .../connectors/hive/read/HiveInputFormat.java | 11 +- .../hive/read/HiveInputFormatPartitionReader.java | 2 +- .../connectors/hive/read/HivePartitionContext.java | 2 +- .../hive/read/HivePartitionFetcherContextBase.java | 14 +- .../hive/read/HiveVectorizedOrcSplitReader.java | 2 +- .../read/HiveVectorizedParquetSplitReader.java | 2 +- .../hive/write/HiveOutputFormatFactory.java | 2 +- .../hive/HiveDynamicTableFactoryTest.java | 19 +- .../connectors/hive/HiveLookupJoinITCase.java | 19 +- .../flink/connectors/hive/HiveTableSinkITCase.java | 10 +- .../connectors/hive/HiveTemporalJoinITCase.java | 2 +- .../connectors/hive/PartitionMonitorTest.java | 4 +- .../connectors/hive/TestCustomCommitPolicy.java | 2 +- .../hive/read/HivePartitionFetcherTest.java | 4 +- .../flink-sql-connector-hbase-1.4/pom.xml | 1 + .../flink-sql-connector-hbase-2.2/pom.xml | 1 + flink-connectors/flink-sql-connector-kafka/pom.xml | 3 +- .../flink-sql-connector-kinesis/pom.xml | 1 + .../flink-sql-connector-rabbitmq/pom.xml | 3 +- .../apache/flink/testutils/ClassLoaderUtils.java | 48 +++- flink-dist/pom.xml | 21 ++ flink-dist/src/main/assemblies/bin.xml | 9 +- flink-examples/flink-examples-table/pom.xml | 5 + .../flink-avro-confluent-registry/pom.xml | 12 + flink-formats/flink-avro/pom.xml | 11 +- .../flink/formats/avro/AvroFileFormatFactory.java | 2 +- .../formats/avro/AvroFileSystemFormatFactory.java | 2 +- flink-formats/flink-csv/pom.xml | 11 + .../formats/csv/CsvFileSystemFormatFactory.java | 2 +- flink-formats/flink-json/pom.xml | 11 + flink-formats/flink-orc-nohive/pom.xml | 26 +- .../nohive/OrcNoHiveColumnarRowInputFormat.java | 25 +- .../flink/orc/nohive/OrcNoHiveSplitReaderUtil.java | 4 +- .../orc/nohive/vector/AbstractOrcNoHiveVector.java | 8 +- .../orc/nohive/vector/OrcNoHiveBytesVector.java | 2 +- .../orc/nohive/vector/OrcNoHiveDecimalVector.java | 2 +- .../orc/nohive/vector/OrcNoHiveDoubleVector.java | 4 +- .../orc/nohive/vector/OrcNoHiveLongVector.java | 10 +- .../nohive/vector/OrcNoHiveTimestampVector.java | 2 +- flink-formats/flink-orc/pom.xml | 15 +- .../flink/orc/OrcColumnarRowInputFormat.java | 36 +-- .../flink/orc/OrcColumnarRowSplitReader.java | 4 +- .../org/apache/flink/orc/OrcFileFormatFactory.java | 11 +- .../org/apache/flink/orc/OrcSplitReaderUtil.java | 4 +- .../flink/orc/vector/AbstractOrcColumnVector.java | 8 +- .../flink/orc/vector/ColumnBatchFactory.java | 2 +- .../flink/orc/vector/OrcArrayColumnVector.java | 6 +- .../flink/orc/vector/OrcBytesColumnVector.java | 2 +- .../flink/orc/vector/OrcDecimalColumnVector.java | 2 +- .../flink/orc/vector/OrcDoubleColumnVector.java | 4 +- .../orc/vector/OrcLegacyTimestampColumnVector.java | 2 +- .../flink/orc/vector/OrcLongColumnVector.java | 10 +- .../flink/orc/vector/OrcMapColumnVector.java | 6 +- .../flink/orc/vector/OrcRowColumnVector.java | 8 +- .../flink/orc/vector/OrcTimestampColumnVector.java | 2 +- .../flink/orc/OrcColumnarRowInputFormatTest.java | 9 +- flink-formats/flink-parquet/pom.xml | 14 +- .../parquet/ParquetColumnarRowInputFormat.java | 38 +-- .../formats/parquet/ParquetFileFormatFactory.java | 9 +- .../parquet/ParquetVectorizedInputFormat.java | 6 +- .../formats/parquet/row/ParquetRowDataWriter.java | 6 +- .../parquet/utils/ParquetSchemaConverter.java | 9 + .../formats/parquet/vector/ColumnBatchFactory.java | 4 +- .../vector/ParquetColumnarRowSplitReader.java | 8 +- .../parquet/vector/ParquetDecimalVector.java | 16 +- .../formats/parquet/vector/ParquetDictionary.java | 2 +- .../parquet/vector/ParquetSplitReaderUtil.java | 34 +-- .../vector/reader/AbstractColumnReader.java | 4 +- .../parquet/vector/reader/BooleanColumnReader.java | 4 +- .../parquet/vector/reader/ByteColumnReader.java | 4 +- .../parquet/vector/reader/BytesColumnReader.java | 4 +- .../parquet/vector/reader/ColumnReader.java | 2 +- .../parquet/vector/reader/DoubleColumnReader.java | 4 +- .../vector/reader/FixedLenBytesColumnReader.java | 18 +- .../parquet/vector/reader/FloatColumnReader.java | 4 +- .../parquet/vector/reader/IntColumnReader.java | 2 +- .../parquet/vector/reader/LongColumnReader.java | 4 +- .../parquet/vector/reader/RunLengthDecoder.java | 4 +- .../parquet/vector/reader/ShortColumnReader.java | 4 +- .../vector/reader/TimestampColumnReader.java | 4 +- .../parquet/ParquetColumnarRowInputFormatTest.java | 7 +- .../vector/ParquetColumnarRowSplitReaderTest.java | 4 +- flink-formats/flink-sql-orc/pom.xml | 6 +- flink-formats/flink-sql-parquet/pom.xml | 6 +- .../flink/table/runtime/arrow/ArrowReader.java | 6 +- .../flink/table/runtime/arrow/ArrowUtils.java | 2 +- .../arrow/vectors/ArrowArrayColumnVector.java | 6 +- .../arrow/vectors/ArrowBigIntColumnVector.java | 2 +- .../arrow/vectors/ArrowBooleanColumnVector.java | 2 +- .../arrow/vectors/ArrowDateColumnVector.java | 2 +- .../arrow/vectors/ArrowDecimalColumnVector.java | 2 +- .../arrow/vectors/ArrowDoubleColumnVector.java | 2 +- .../arrow/vectors/ArrowFloatColumnVector.java | 2 +- .../arrow/vectors/ArrowIntColumnVector.java | 2 +- .../arrow/vectors/ArrowRowColumnVector.java | 8 +- .../arrow/vectors/ArrowSmallIntColumnVector.java | 2 +- .../arrow/vectors/ArrowTimeColumnVector.java | 2 +- .../arrow/vectors/ArrowTimestampColumnVector.java | 2 +- .../arrow/vectors/ArrowTinyIntColumnVector.java | 2 +- .../arrow/vectors/ArrowVarBinaryColumnVector.java | 2 +- .../arrow/vectors/ArrowVarCharColumnVector.java | 2 +- .../flink/table/runtime/arrow/ArrowUtilsTest.java | 2 +- .../table/sinks/CsvAppendTableSinkFactory.java | 3 +- .../table/sinks/CsvBatchTableSinkFactory.java | 3 +- .../org/apache/flink/table/sinks/CsvTableSink.java | 3 +- .../flink/table/sinks/CsvTableSinkFactoryBase.java | 3 +- .../table/sources/CsvAppendTableSourceFactory.java | 3 +- .../table/sources/CsvBatchTableSourceFactory.java | 3 +- .../apache/flink/table/sources/CsvTableSource.java | 3 +- .../table/sources/CsvTableSourceFactoryBase.java | 3 +- flink-table/flink-table-common/pom.xml | 6 - .../table/data/columnar}/ColumnarArrayData.java | 54 ++-- .../table/data/columnar}/ColumnarMapData.java | 23 +- .../table/data/columnar}/ColumnarRowData.java | 17 +- .../data/columnar}/vector/ArrayColumnVector.java | 4 +- .../data/columnar}/vector/BooleanColumnVector.java | 5 +- .../data/columnar}/vector/ByteColumnVector.java | 5 +- .../data/columnar}/vector/BytesColumnVector.java | 5 +- .../table/data/columnar}/vector/ColumnVector.java | 5 +- .../data/columnar}/vector/DecimalColumnVector.java | 4 +- .../table/data/columnar}/vector/Dictionary.java | 4 +- .../data/columnar}/vector/DoubleColumnVector.java | 5 +- .../data/columnar}/vector/FloatColumnVector.java | 5 +- .../data/columnar}/vector/IntColumnVector.java | 5 +- .../data/columnar}/vector/LongColumnVector.java | 5 +- .../data/columnar}/vector/MapColumnVector.java | 4 +- .../data/columnar}/vector/RowColumnVector.java | 6 +- .../data/columnar}/vector/ShortColumnVector.java | 5 +- .../columnar}/vector/TimestampColumnVector.java | 4 +- .../columnar}/vector/VectorizedColumnBatch.java | 6 +- .../columnar}/vector/heap/AbstractHeapVector.java | 6 +- .../columnar}/vector/heap/HeapBooleanVector.java | 6 +- .../data/columnar}/vector/heap/HeapByteVector.java | 6 +- .../columnar}/vector/heap/HeapBytesVector.java | 6 +- .../columnar}/vector/heap/HeapDoubleVector.java | 6 +- .../columnar}/vector/heap/HeapFloatVector.java | 6 +- .../data/columnar}/vector/heap/HeapIntVector.java | 6 +- .../data/columnar}/vector/heap/HeapLongVector.java | 6 +- .../columnar}/vector/heap/HeapShortVector.java | 6 +- .../columnar}/vector/heap/HeapTimestampVector.java | 6 +- .../vector/writable/AbstractWritableVector.java | 8 +- .../vector/writable/WritableBooleanVector.java | 6 +- .../vector/writable/WritableByteVector.java | 6 +- .../vector/writable/WritableBytesVector.java | 6 +- .../vector/writable/WritableColumnVector.java | 8 +- .../vector/writable/WritableDoubleVector.java | 6 +- .../vector/writable/WritableFloatVector.java | 6 +- .../vector/writable/WritableIntVector.java | 6 +- .../vector/writable/WritableLongVector.java | 6 +- .../vector/writable/WritableShortVector.java | 6 +- .../vector/writable/WritableTimestampVector.java | 6 +- .../table/descriptors/FileSystemValidator.java | 4 +- .../apache/flink/table/factories/FactoryUtil.java | 33 ++- .../flink/table/factories/ServiceLoaderUtil.java | 100 ++++++++ .../data/columnar}/vector/ColumnVectorTest.java | 36 +-- .../vector/VectorizedColumnBatchTest.java | 22 +- .../flink/table/factories/FactoryUtilTest.java | 275 +++++++++++++-------- flink-table/flink-table-planner/pom.xml | 8 + .../batch/BatchPhysicalLegacySinkRule.scala | 11 +- .../physical/batch/BatchPhysicalSinkRule.scala | 11 +- .../stream/StreamPhysicalLegacySinkRule.scala | 11 +- .../physical/stream/StreamPhysicalSinkRule.scala | 7 +- .../file/table}/FileSystemTableFactoryTest.java | 4 +- .../file/table}/FileSystemTableSinkTest.java | 2 +- .../file/table}/FileSystemTableSourceTest.java | 2 +- .../testcsv}/TestCsvDeserializationSchema.java | 35 ++- .../formats/testcsv/TestCsvFormatFactory.java} | 89 ++----- .../testcsv/TestCsvSerializationSchema.java | 58 +++++ .../table/planner/factories/TestFileFactory.java | 2 +- .../runtime/stream/sql/CompactionITCaseBase.java | 2 +- .../org.apache.flink.table.factories.Factory | 1 + .../file/table}/FileSystemTableSourceTest.xml | 0 .../batch/sql/PartitionableSinkITCase.scala | 2 +- .../runtime/stream/FsStreamingSinkITCaseBase.scala | 4 +- .../runtime/typeutils/ArrayDataSerializer.java | 2 +- .../org.apache.flink.table.factories.Factory | 1 - .../runtime/typeutils/ArrayDataSerializerTest.java | 4 +- flink-table/flink-table-uber/pom.xml | 3 - 277 files changed, 1582 insertions(+), 961 deletions(-) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/AbstractFileSystemTable.java (88%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/util => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/BinPacking.java (82%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/ColumnarRowIterator.java (94%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/ContinuousPartitionFetcher.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/DefaultPartTimeExtractor.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/DeserializationSchemaAdapter.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/DynamicPartitionWriter.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/EmptyMetaStoreFactory.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/EnrichedRowData.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileInfoExtractorBulkFormat.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileSystemCommitter.java (93%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileSystemConnectorOptions.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileSystemFactory.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileSystemOutputFormat.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileSystemTableFactory.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileSystemTableSink.java (93%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/FileSystemTableSource.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/GroupedPartitionWriter.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/LimitableBulkFormat.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/MetastoreCommitPolicy.java (92%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/OutputFormatFactory.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionCommitPolicy.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionComputer.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionFetcher.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionFieldExtractor.java (95%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionLoader.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionReader.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionTempFileManager.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionTimeExtractor.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionWriter.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/PartitionWriterFactory.java (94%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/ProjectingBulkFormat.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/RowDataPartitionComputer.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/RowPartitionComputer.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/SerializationSchemaAdapter.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/SingleDirectoryWriter.java (98%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/SuccessFileCommitPolicy.java (94%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/TableMetaStoreFactory.java (98%) rename {flink-table/flink-table-common/src/main/java/org/apache/flink => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file}/table/factories/BulkReaderFormatFactory.java (84%) rename {flink-table/flink-table-common/src/main/java/org/apache/flink => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file}/table/factories/BulkWriterFormatFactory.java (88%) rename {flink-table/flink-table-common/src/main/java/org/apache/flink => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file}/table/factories/FileSystemFormatFactory.java (97%) rename {flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/format/BulkDecodingFormat.java (89%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/AbstractStreamingWriter.java (99%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/PartitionCommitInfo.java (95%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/PartitionCommitPredicate.java (90%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/PartitionCommitTrigger.java (87%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/PartitionCommitter.java (91%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/PartitionTimeCommitPredicate.java (81%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/PartitionTimeCommitTrigger.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/ProcTimeCommitPredicate.java (87%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/ProcTimeCommitTrigger.java (95%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/StreamingFileWriter.java (94%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/StreamingSink.java (86%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/TaskTracker.java (93%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactBucketWriter.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactBulkReader.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactContext.java (79%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactCoordinator.java (91%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactFileWriter.java (80%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactMessages.java (97%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactOperator.java (93%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactReader.java (64%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/CompactWriter.java (64%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table}/stream/compact/FileInputFormatCompactReader.java (95%) rename {flink-table/flink-table-runtime/src/test => flink-connectors/flink-connector-files/src/main}/resources/META-INF/services/org.apache.flink.table.factories.Factory (92%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/runtime/util => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/BinPackingTest.java (85%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/EnrichedRowDataTest.java (98%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/FileSystemCommitterTest.java (99%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/FileSystemOutputFormatTest.java (99%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/LimitableBulkFormatTest.java (99%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/PartitionWriterTest.java (98%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/RowPartitionComputerTest.java (98%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/stream/StreamingFileWriterTest.java (97%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/stream/compact/AbstractCompactTestBase.java (96%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/stream/compact/CompactCoordinatorTest.java (90%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/stream/compact/CompactFileWriterTest.java (89%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/stream/compact/CompactOperatorTest.java (96%) rename {flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/table}/stream/compact/TestByteFormat.java (96%) rename {flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem => flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive}/FileSystemLookupFunction.java (97%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/ColumnarArrayData.java (77%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/ColumnarMapData.java (69%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/ColumnarRowData.java (91%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/ArrayColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/BooleanColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/ByteColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/BytesColumnVector.java (93%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/ColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/DecimalColumnVector.java (90%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/Dictionary.java (91%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/DoubleColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/FloatColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/IntColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/LongColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/MapColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/RowColumnVector.java (84%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/ShortColumnVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/TimestampColumnVector.java (90%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/VectorizedColumnBatch.java (95%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/AbstractHeapVector.java (94%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapBooleanVector.java (90%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapByteVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapBytesVector.java (95%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapDoubleVector.java (93%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapFloatVector.java (93%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapIntVector.java (94%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapLongVector.java (93%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapShortVector.java (89%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/heap/HeapTimestampVector.java (91%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/AbstractWritableVector.java (87%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableBooleanVector.java (85%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableByteVector.java (85%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableBytesVector.java (87%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableColumnVector.java (88%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableDoubleVector.java (88%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableFloatVector.java (88%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableIntVector.java (90%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableLongVector.java (88%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableShortVector.java (85%) rename flink-table/{flink-table-runtime/src/main/java/org/apache/flink/table/data => flink-table-common/src/main/java/org/apache/flink/table/data/columnar}/vector/writable/WritableTimestampVector.java (86%) create mode 100644 flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/ServiceLoaderUtil.java rename flink-table/{flink-table-runtime/src/test/java/org/apache/flink/table/data => flink-table-common/src/test/java/org/apache/flink/table/data/columnar}/vector/ColumnVectorTest.java (89%) rename flink-table/{flink-table-runtime/src/test/java/org/apache/flink/table/data => flink-table-common/src/test/java/org/apache/flink/table/data/columnar}/vector/VectorizedColumnBatchTest.java (93%) rename flink-table/{flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-table-planner/src/test/java/org/apache/flink/connector/file/table}/FileSystemTableFactoryTest.java (98%) rename flink-table/flink-table-planner/src/test/java/org/apache/flink/{table/filesystem => connector/file/table}/FileSystemTableSinkTest.java (99%) rename flink-table/flink-table-planner/src/test/java/org/apache/flink/{table/filesystem => connector/file/table}/FileSystemTableSourceTest.java (98%) rename flink-table/{flink-table-runtime/src/test/java/org/apache/flink/table/filesystem => flink-table-planner/src/test/java/org/apache/flink/formats/testcsv}/TestCsvDeserializationSchema.java (83%) rename flink-table/{flink-table-runtime/src/test/java/org/apache/flink/table/filesystem/TestCsvFileSystemFormatFactory.java => flink-table-planner/src/test/java/org/apache/flink/formats/testcsv/TestCsvFormatFactory.java} (54%) create mode 100644 flink-table/flink-table-planner/src/test/java/org/apache/flink/formats/testcsv/TestCsvSerializationSchema.java rename flink-table/flink-table-planner/src/test/resources/org/apache/flink/{table/filesystem => connector/file/table}/FileSystemTableSourceTest.xml (100%)