bulalalaaa commented on PR #57096: URL: https://github.com/apache/doris/pull/57096#issuecomment-6522080239
Hi, thank you for proposing this fix. Could you help clarify whether adding the `AWS_TOKEN` alias might also address an issue with asynchronous EXPORT? We are running Apache Doris `4.1.1-rc01` (commit `b10073ad9ca`), with all FE and BE nodes on the same version. We export to Huawei Cloud OBS through the S3-compatible API using temporary credentials (`s3.access_key`, `s3.secret_key`, and `s3.session_token`). With the same credential settings: - `SELECT... INTO OUTFILE` succeeds. - `EXPORT TABLE... WITH S3 (...)` fails with: `failed to CreateMultipartUpload: InvalidAccessKeyId — The AWS Access Key Id you provided does not exist in our records.` One difference in our initial tests is that OUTFILE filters a single record, while EXPORT uses a broader filter. From reviewing the code, it appears that `ExportJob.convertOutfileProperties()` obtains backend properties, where the session token is named `AWS_TOKEN`. These properties are then passed into the internal OUTFILE execution path. However, in our revision, the token aliases in `OBSProperties` do not include `AWS_TOKEN`. Could this cause the token to be lost during the second round of property parsing? If so, would adding `AWS_TOKEN` to both `OBSProperties` and `S3Properties` be an appropriate fix? I noticed that this PR was closed without being merged. Was it superseded by another fix, or is there a reason this approach should not be used? We have not yet tested the patch, so this is still a hypothesis. Any guidance or links to a related issue or fix would be greatly appreciated. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
