These factories should be enabled to allow users to configure more easily and support more features;
Below I briefly explain the factory: 1. CatalogFactory: a feature with the same target as the user-defined schema, used to obtain the structured information of the table; 2. DeserializationFormatFactory & SerializationFormatFactory: used to support kafka, file and other unstructured connectors to process data in various formats; 3. TableSourceFactory: It is used to support multi-table reading and metadata column features. It is an upper-level extension of source: 4. TableSinkFactory: The same level as TableSourceFactory, but there are currently no supported features; You can review the flowchart in this proposal and refine the entire process: https://github.com/apache/incubator-seatunnel/issues/1608 Best, Zongwen Li JUN GAO <[email protected]> 于2022年8月12日周五 10:24写道： > Should we enable `org.apache.seatunnel.api.table.factory` and optimize the > design ? I think @Lucifer Tyrant can do this. > > > Zongwen Li <[email protected]> 于2022年8月12日周五 10:15写道： > > > The Format feature already exists, the problem is that we haven't enabled > > it yet; > > In fact, all the features in the `org.apache.seatunnel.api.table.factory` > > package under the `seatunnel-api` module are not enabled, they are > mutually > > dependent; > > > > Best, > > Zongwen Li > > > > 范佳 <[email protected]> 于2022年8月11日周四 15:32写道： > > > > > +1 > > > > > > Thanks for open this discussion. > > > > > > ________________________ > > > > > > Jia Fan > > > > > > > 2022年8月11日 14:35，Lucifer Tyrant <[email protected]> 写道： > > > > > > > > Hi seatunnel community, > > > > > > > > According to pr #2392, seatunnel has a new feature to parse schema > > > > information. Next I want to discuss with your guys how to define data > > > > format in source connector configuration. > > > > > > > > In my opinion, the config can be defined as the following: > > > > > > > > schema { > > > > fields {} > > > > format = xxx > > > > delimiter = xxx > > > > } > > > > > > > > We can abstract a new interface to parse format information in > config. > > > This > > > > interface can use *seatunnel-formats* converting upstream from row > data > > > to > > > > a seatunnel row. What do you think about it? > > > > > > > > TyrantLucifer > > > > > > > > > > > > > > > -- > > Best Regards > > ------------ > > EricJoy2048 > [email protected] >
