I need to familiarize with the design first, before I do this work I will discuss with your guys again.
BR, TyrantLucifer Zongwen Li <[email protected]> 于2022年8月12日周五 10:42写道： > These factories should be enabled to allow users to configure more easily > and support more features; > > Below I briefly explain the factory: > 1. CatalogFactory: a feature with the same target as the user-defined > schema, used to obtain the structured information of the table; > 2. DeserializationFormatFactory & SerializationFormatFactory: used to > support kafka, file and other unstructured connectors to process data in > various formats; > 3. TableSourceFactory: It is used to support multi-table reading and > metadata column features. It is an upper-level extension of source: > 4. TableSinkFactory: The same level as TableSourceFactory, but there are > currently no supported features; > > You can review the flowchart in this proposal and refine the entire > process: https://github.com/apache/incubator-seatunnel/issues/1608 > > Best, > Zongwen Li > > JUN GAO <[email protected]> 于2022年8月12日周五 10:24写道： > > > Should we enable `org.apache.seatunnel.api.table.factory` and optimize > the > > design ? I think @Lucifer Tyrant can do this. > > > > > > Zongwen Li <[email protected]> 于2022年8月12日周五 10:15写道： > > > > > The Format feature already exists, the problem is that we haven't > enabled > > > it yet; > > > In fact, all the features in the > `org.apache.seatunnel.api.table.factory` > > > package under the `seatunnel-api` module are not enabled, they are > > mutually > > > dependent; > > > > > > Best, > > > Zongwen Li > > > > > > 范佳 <[email protected]> 于2022年8月11日周四 15:32写道： > > > > > > > +1 > > > > > > > > Thanks for open this discussion. > > > > > > > > ________________________ > > > > > > > > Jia Fan > > > > > > > > > 2022年8月11日 14:35，Lucifer Tyrant <[email protected]> 写道： > > > > > > > > > > Hi seatunnel community, > > > > > > > > > > According to pr #2392, seatunnel has a new feature to parse schema > > > > > information. Next I want to discuss with your guys how to define > data > > > > > format in source connector configuration. > > > > > > > > > > In my opinion, the config can be defined as the following: > > > > > > > > > > schema { > > > > > fields {} > > > > > format = xxx > > > > > delimiter = xxx > > > > > } > > > > > > > > > > We can abstract a new interface to parse format information in > > config. > > > > This > > > > > interface can use *seatunnel-formats* converting upstream from row > > data > > > > to > > > > > a seatunnel row. What do you think about it? > > > > > > > > > > TyrantLucifer > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best Regards > > > > ------------ > > > > EricJoy2048 > > [email protected] > > >
