[
https://issues.apache.org/jira/browse/FLINK-39235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated FLINK-39235:
---------------------------------
Description:
Investigating whether this change is required.
We have a proprietary json format in IBM that we use that allows literals
(constants) to be specified.
We are looking to move off this code an use the open source connector.
What we require for requests is :
- change 'http.request.additional-body-json' so that it ONLY works with columns
that are defined.
- change 'http.request.additional-body-json' so that it can specify a nested
constant and this is merged with event content
- change 'http.request.additional-body-json' so it will be used in preference
to the event content
What we require for responses
- 'http.response.default-json' . This will be one or more json objects that
define constants that will be used if there is no value for optional fields in
the HTTP response.
- 'http.response.default-json' needs to be able to specify json that will be
merged with event json including nesting
We may need to add more options if the community would like a more general
solution , but the above is what we need to be able to configure.
was:
Investigating whether this change is required.
We have a proprietary json format in IBM that we use that allows literals to be
specified.
We are looking to move off this code an use the open source connector.
Initially I was thinking that we could have a subsequent view that introduced a
constant. This works in most cases. It does not work when there is second a
downstream lookup join when the constant is a boolean object or array. The
reason for this i that a lookup join needs an equality from the table planner
for the lookup join and these types correctly do not result in that.
I propose we have a 'http.response.additional-json' similar to
'http.request.additional-body-json' but for responses. This would hide the
constants from the table planner. With these considerations
- We would allow only fields that were defined in the table to be specified.
- what do we do if the response actually returns this field. I suggest a second
field 'response.honour.additional-json' when true means the config wins.
- We should type check the supplied json in this config.
> Re implement HTTP Connector constants support
> ---------------------------------------------
>
> Key: FLINK-39235
> URL: https://issues.apache.org/jira/browse/FLINK-39235
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / HTTP
> Reporter: David Radley
> Assignee: David Radley
> Priority: Major
>
> Investigating whether this change is required.
>
> We have a proprietary json format in IBM that we use that allows literals
> (constants) to be specified.
> We are looking to move off this code an use the open source connector.
> What we require for requests is :
> - change 'http.request.additional-body-json' so that it ONLY works with
> columns that are defined.
> - change 'http.request.additional-body-json' so that it can specify a nested
> constant and this is merged with event content
> - change 'http.request.additional-body-json' so it will be used in preference
> to the event content
> What we require for responses
> - 'http.response.default-json' . This will be one or more json objects that
> define constants that will be used if there is no value for optional fields
> in the HTTP response.
> - 'http.response.default-json' needs to be able to specify json that will be
> merged with event json including nesting
>
> We may need to add more options if the community would like a more general
> solution , but the above is what we need to be able to configure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
