[
https://issues.apache.org/jira/browse/HADOOP-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095504#comment-17095504
]
Steve Loughran commented on HADOOP-17018:
-----------------------------------------
# anyone know why the 404s?
# Do you see the same output on a successful run?
# maybe we could have multiple counters: #of requests , #of successes; #of
failures (or just infer that from requests - successes).
then the test can assert on successes only, and we have some metrics in
production which can measure the failure rate.
At the same time, I don't see the initial and final 404s being responses to the
read() operations, I think one is in setup, and other in teardown (it comes
after a delete from the main junit thread)
one of the log entries is on a different thread
{code}
[ABFS-prefetch-4]: services.AbfsClient (AbfsRestOperation.java:execute(133)) -
First execution of REST operation - ReadFile2020-04-28 15:49:10,646 DEBUG
[ABFS-prefetch-4]: services.AbfsClient
(AbfsRestOperation.java:executeHttpOperation(175)) - Signing request with
shared key2020-04-28 15:49:11,389 DEBUG [ABFS-prefetch-4]: services.AbfsClient
(AbfsRestOperation.java:executeHttpOperation(223)) - HttpRequest:
206,,cid=fe0d1b0b-3f46-41fd-8611-5f8fa7c75564,rid=8026b8f0-a01f-005a-4346-1d2ad8000000,sent=0,recv=9,GET,https://mehakmeetdata.dfs.core.windows.net/abfs-testcontainer-ce1153dd-6c16-402b-91c8-099051333a12/user/mehakmeet.singh/testOneReadWriteOps?timeout=902020-04-28
1
{code}
That seems more likely. It could be some race condition which surfaces on
different people's machines/different test setups, so hard to reproduce.
See if you could come up with a scenario where that prefetch code only happened
some times/at a different speed on different systems. Then once we have a story
as to "why" there are differences, we can just change the assert and add a
comment above it about why it's relaxed about the #of requests
> Intermittent failing of ITestAbfsStreamStatistics in ABFS
> ---------------------------------------------------------
>
> Key: HADOOP-17018
> URL: https://issues.apache.org/jira/browse/HADOOP-17018
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 3.2.1
> Reporter: Mehakmeet Singh
> Assignee: Mehakmeet Singh
> Priority: Major
>
> There are intermittent failures of a test inside ITestAbfsStreamStatistics in
> ABFS.
> Did consecutive runs of the test and failure seemed random. Stack Trace in
> the comments.
> Propose:
> - Change the assertion of the test for it to be passed, Since the production
> code seems fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
