[
https://issues.apache.org/jira/browse/HIVE-24225?focusedWorklogId=496487&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496487
]
ASF GitHub Bot logged work on HIVE-24225:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Oct/20 10:53
Start Date: 07/Oct/20 10:53
Worklog Time Spent: 10m
Work Description: pgaref edited a comment on pull request #1547:
URL: https://github.com/apache/hive/pull/1547#issuecomment-704856290
Hey @steveloughran --- the approach of the above patch was a bit off, one
problem was that the Fs objects were lazility initalized and could end up
throwing exceptions when setting the option eagerly.
The most important issue was that the LLAP IO creates its own FS object (and
the above where only used for output) so the option itself was not properly
propagated.
A solution for all this could be the S3A **openFileWithOptions** call that
adds file options for the open File call instead of on the FS (still needs to
add support for **fadvise** though)
https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L4828
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 496487)
Time Spent: 50m (was: 40m)
> FIX S3A recordReader policy selection
> -------------------------------------
>
> Key: HIVE-24225
> URL: https://issues.apache.org/jira/browse/HIVE-24225
> Project: Hive
> Issue Type: Bug
> Reporter: Panagiotis Garefalakis
> Assignee: Panagiotis Garefalakis
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Dynamic S3A recordReader policy selection can cause issues on lazy
> initialized FS objects
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
