steveloughran commented on PR #5176:
URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1344513110
#5205 is another followup with the logging and a probe through path
capabilities; this allows me to verify that backports are in.
an abfs instance is vulnerable if
```java
steveloughran commented on PR #5176:
URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1344461171
(oh, and on my personal backport I have added a TRACE log in the buffer
manager to record its state; abfsInputStream.toString does it too.
```
private ReadBufferManager() {
steveloughran commented on PR #5176:
URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1344325189
update: full e2e tests through spark shell are happy! i was trying to do
scalatest tests for this but not been able to replicate the test failure
through my test suite (which rebuild
steveloughran commented on PR #5176:
URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1343192257
it's a race condition in the test, which is why you didn't see
it...different machine, network etc.
--
This is an automated message from the Apache Git Service.
To respond to the
steveloughran commented on PR #5176:
URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1343033763
getting a test failure locally, ITestReadBufferManager failing as one of its
asserts isn't valid.
going to reopen the jira
@pranavsaxena-microsoft can you see if you can rep
steveloughran commented on PR #5176:
URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1339526719
clarified the cleanup problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spec
steveloughran commented on PR #5176:
URL: https://github.com/apache/hadoop/pull/5176#issuecomment-1336158644
sorry, should have been clearer: a local spark build and spark-shell process
is ideal for replication and validation -as all splits are processed in
different worker threads in that
