[GitHub] [hadoop] steveloughran commented on pull request #5172: HADOOP-18543. AliyunOSSFileSystem#open(Path path, int bufferSize) use buffer size as its downloadPartSize

steveloughran commented on PR #5172: URL: https://github.com/apache/hadoop/pull/5172#issuecomment-1335114105 > I'd like to make oss also implement openFile() in this pr as s3a does which could also meet our needs. This is exactly what the API was designed for -to let people provide ex

[GitHub] [hadoop] steveloughran commented on pull request #5172: HADOOP-18543. AliyunOSSFileSystem#open(Path path, int bufferSize) use buffer size as its downloadPartSize

steveloughran commented on PR #5172: URL: https://github.com/apache/hadoop/pull/5172#issuecomment-1334109192 (note also that includes letting you declare read policy (whole-file, sequential, random, vectoredthat can be used to change default block size too) -- This is an automated me

[GitHub] [hadoop] steveloughran commented on pull request #5172: HADOOP-18543. AliyunOSSFileSystem#open(Path path, int bufferSize) use buffer size as its downloadPartSize

steveloughran commented on PR #5172: URL: https://github.com/apache/hadoop/pull/5172#issuecomment-1334108437 sorry, but I'm going to say -1 to using the normal IO buffer size as the GET range. The default value of 4k is way too small even for parquet/orc reads, it will break all existing ap