[
https://issues.apache.org/jira/browse/HADOOP-17735?focusedWorklogId=602495&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602495
]
ASF GitHub Bot logged work on HADOOP-17735:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 26/May/21 18:56
Start Date: 26/May/21 18:56
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3057:
URL: https://github.com/apache/hadoop/pull/3057#issuecomment-849037658
OK, full testing.md runbook done (with iostats and audit logging enabled).
All good. For the curious, here are the iostats in the
ITestS3AInputStreamPerformance output
```
2021-05-26 19:53:20,019 [JUnit] INFO s3a.AbstractS3ATestBase
(AbstractS3ATestBase.java:dumpFileSystemIOStatistics(123)) - Aggregate
FileSystem Statistics counters=((action_executor_acquired=1)
(action_http_get_request=15)
(action_http_head_request=24)
(audit_request_execution=66)
(audit_span_creation=50)
(directories_created=8)
(directories_deleted=7)
(fake_directories_deleted=1)
(files_created=1)
(files_deleted=1)
(object_bulk_delete_request=2)
(object_delete_objects=9)
(object_delete_request=7)
(object_list_request=9)
(object_metadata_request=24)
(object_put_bytes=32768)
(object_put_request=9)
(object_put_request_completed=9)
(op_create=1)
(op_delete=8)
(op_get_file_status=9)
(op_mkdirs=8)
(op_open=8)
(s3guard_metadatastore_initialization=8)
(s3guard_metadatastore_put_path_request=17)
(s3guard_metadatastore_record_reads=40)
(s3guard_metadatastore_record_writes=26)
(store_io_request=67)
(stream_aborted=2)
(stream_read_bytes=93473433)
(stream_read_bytes_backwards_on_seek=12713984)
(stream_read_bytes_discarded_in_abort=43889622)
(stream_read_bytes_discarded_in_close=252395)
(stream_read_close_operations=8)
(stream_read_closed=13)
(stream_read_fully_operations=8)
(stream_read_opened=15)
(stream_read_operations=14445)
(stream_read_operations_incomplete=3295)
(stream_read_seek_backward_operations=4)
(stream_read_seek_bytes_discarded=45092691)
(stream_read_seek_bytes_skipped=55054163)
(stream_read_seek_forward_operations=175)
(stream_read_seek_operations=179)
(stream_read_seek_policy_changed=9)
(stream_read_total_bytes=138818519)
(stream_write_block_uploads=1)
(stream_write_bytes=32768)
(stream_write_total_data=65536));
gauges=((stream_write_block_uploads_pending=1));
minimums=((action_executor_acquired.min=0)
(action_http_get_request.min=24)
(action_http_head_request.min=22)
(object_bulk_delete_request.min=49)
(object_delete_request.min=28)
(object_list_request.min=35)
(object_put_request.min=95)
(op_create.min=66)
(op_delete.min=48)
(op_get_file_status.min=157)
(op_mkdirs.min=152));
maximums=((action_executor_acquired.max=0)
(action_http_get_request.max=664)
(action_http_head_request.max=1688)
(object_bulk_delete_request.max=65)
(object_delete_request.max=37)
(object_list_request.max=75)
(object_put_request.max=360)
(op_create.max=66)
(op_delete.max=291)
(op_get_file_status.max=1690)
(op_mkdirs.max=446));
means=((action_executor_acquired.mean=(samples=1, sum=0, mean=0.0000))
(action_http_get_request.mean=(samples=15, sum=2583, mean=172.2000))
(action_http_head_request.mean=(samples=24, sum=8556, mean=356.5000))
(object_bulk_delete_request.mean=(samples=2, sum=114, mean=57.0000))
(object_delete_request.mean=(samples=7, sum=228, mean=32.5714))
(object_list_request.mean=(samples=9, sum=392, mean=43.5556))
(object_put_request.mean=(samples=9, sum=1390, mean=154.4444))
(op_create.mean=(samples=1, sum=66, mean=66.0000))
(op_delete.mean=(samples=8, sum=672, mean=84.0000))
(op_get_file_status.mean=(samples=9, sum=6857, mean=761.8889))
(op_mkdirs.mean=(samples=8, sum=1676, mean=209.5000)));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 602495)
Time Spent: 20m (was: 10m)
> Upgrade aws-java-sdk to 1.11.993 or later
> -----------------------------------------
>
> Key: HADOOP-17735
> URL: https://issues.apache.org/jira/browse/HADOOP-17735
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Upgrade the AWS SDK. Apparently the shaded netty jar has some CVEs, and even
> though the AWS codepaths don't seem vulnerable, it's still causing scan tools
> to warn
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
