mdedetrich commented on PR #1246: URL: https://github.com/apache/pekko-connectors/pull/1246#issuecomment-3414005875
> @mdedetrich / @pjfanning - I made this draft PR to demonstrate the problem in order to discuss the path forward for fixing this problem. I'm going to look to see if I can create an integration test tomorrow. > > I'm thinking that we have an allow list of all correct headers and filter out all the headers that start with the `x-amz-` that do not belong to that list? > > I say starts with `x-amz-` as there might exist use cases with S3 compliant stores that have custom headers. This is just pure speculation however. > > The main con here is that if AWS introduces new headers in the future this would require an update. Alternatively we may wish to simply remove `x-amz-storage-class` and create a block list, which would still need to be maintained. > > Open to alternative suggestions if there are any I would say, if we do have an allowlist it should be provided in as configuration in `reference.conf` so people can add more values without needing us to do an additional release. The initial list can be hardcoded, but there can be additional `pekko.connectors.s3.upload-whitelist-headers` configuration where people can also add additional headers in the case that s3 adds something new. @pjfanning @raboof thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
