steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-754613375
MR client not compiling; not seeing useful information from yetus.
This is an automated message from the Ap
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-757344464
style
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:210:
FSBuilder propagateOptions(: 'FSBuilder' has inc
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-757344464
style
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/functional/FutureIO.java:210:
FSBuilder propagateOptions(: 'FSBuilder' has inc
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-759582014
I'm thinking we should be more ambitious in read policy than just "fadvise",
because we can then use it as a declaration for the input streams to tune all
their params, eg.
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-760911137
@ThomasMarquardt could you take a look @ this ?
* I've updated the docs as suggested
* proposed making the policy broader than just seek policy, so allowing
stores
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-761561997
1. we've already shipped that
2. it's in createFile too
imagine in future you want to do something more than a hint. Case in point:
the SQL API in AWS S3. there's
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-763994374
correct.
must() == raise an error if not recognised.
opt() entirely optional
note that createFile() has the same api/rules/ common builder codebase. i
hope
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-764587055
Thomas, to clarify a bit more
yes, an FS can choose to ignore an option, -but it must recognise the option
and so make a conscious decision "this doesn't matter"
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-763994374
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-771070179
rebased to fix compile problems; the final patch is the one with changes
since Thomas's last review
* move all the text on options into the fsdatainputstreambuilder.m
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1075138032
checkstyle
```
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:120:import
org.apache.hadoop.fs.s3a.select.InternalSelectConst
steveloughran commented on pull request #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1086028402
* rebase against trunk
* s3a input stream will drain the inner stream asynchronously in
seek/unbuffer related calls if the # of bytes to drain is > a new
config/openFil
steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1087397803
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:29:import
java.util.concurrent.CompletableFuture;:8: Unused import -
java.uti
steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1090308409
really need reviews of this @mukund-thakur @mehakmeet @bibinchundatt
@dannycjones @surendralilhore
This patch needs to go in before any other input stream optimisations so
tha
steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1090315539
testing,
* s3 london, markers keep, scale
* azure cardiff (to make sure I've not broken anything there in the move to
openFile() in distcp)
--
This is an automated messa
steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1091538089
ok, that suggestion from thomas about having checksum fs pass down is wrong,
as it means the opened file is bypassing checks.
i can't see a good way of passing down the openfil
steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1096910060
thanks for the reviews; updated the pr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1097968517
@mehakmeet thanks, yes, sounds like it. file a JIRA 😁
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
steveloughran commented on PR #2584:
URL: https://github.com/apache/hadoop/pull/2584#issuecomment-1113172048
merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsub
19 matches
Mail list logo
