[
https://issues.apache.org/jira/browse/HDFS-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632346#comment-14632346
]
Steve Loughran commented on HDFS-8790:
--------------------------------------
based on the object store work, I'd add
# create many files in a directory, then perform directory operations
# create deep directories & manipulate
# many small files and work
# zero byte files & attempt seek, read, positioned read
# stop writes half-way through without closing the connection & see what happens
# create a large file, read partway through, call close() on the input and
verify that the close is timely (and not, say reading the entire file)
# do lots of seeks, forward and back, verify times are not excessive. Ideally
forward seeks < backward seeks
# mix positioned reads with normal read() operations on different threads.
> Add Filesystem level stress tests
> ---------------------------------
>
> Key: HDFS-8790
> URL: https://issues.apache.org/jira/browse/HDFS-8790
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: James Clampffer
> Assignee: James Clampffer
>
> I propose adding stress tests on libhdfs(3) compatibility layer was well as
> the async calls. These can be also used for basic performance metrics and
> inputs to profiling tools to see improvements over time.
> I'd like to make these tests into a seperate executable, or set of them, so
> that they can be used for longer running tests on dedicated clusters that may
> already exist. Each should provide a simple command line interface for
> scripted or manual use.
> Basic tests would be:
> looped open-read-close
> sequential scans
> small random reads
> All tests will be parameterized for number of threads, read size, and upper
> and lower offset bounds for a specified file. This will make it much easier
> to detect and reproduce threading issues and resource leaks as well as
> provide a simple executable (or set of executables) that can be run with
> valgrind to gain a high confidence that the code is operating correctly.
> I'd appreciate suggestions for any other simple stress tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
