EricJoy2048 commented on issue #3573:
URL:
https://github.com/apache/incubator-seatunnel/issues/3573#issuecomment-1328909535
> I have some questions:
>
> 1. As you shown, add a new parameter to tell connector get a part of
upstream data. But this requirement only works on json format, for text format,
how to deal with this feature?
> 2. If user want to get more than one parts of upstream data, this feature
how to work?
> 3. If the part that user want to get is only a string, a integer, a double
etc... not a list, or the every item in list also contains objects array,
schema how to work on data?
1. I think the most of SaaS API will return data use json format. It is rare
to use the text format. If it appears, we will deal with it separately.
2. In fact, each `content field` to be read is equivalent to a table. I this
case the `user` can be see a read table, and we can config the `schema` about
the it. Now SeaTunnel only support read one table data in one connector, so we
only support config one `content field` once. If we support reading data from
multiple tables in one connector in the future, we can also support defining
multiple `content field` in one connector.
3. User can use `schema` to define the schema of `content field`. If use
didn't config schema, We can think that users only want to read basic data such
as string/integer/long and the column name same as `content field`.
I suggest we only support basic array type in the case. This is a example.
```
{
"xxx":"xxx",
"users":[
{
"id":1,
"name":"n1",
"int_list": [1,2,3],
"json_list":[{"n1":"v1", "n2":"v2"}]
}
]
}
```
```
schema: {
id: int,
name: string,
int_list: array<int>,
json_lsit: array<string>
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
