[jira] [Commented] (HADOOP-19161) S3A: option "fs.s3a.performance.flags" to take list of performance flags
[ https://issues.apache.org/jira/browse/HADOOP-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17855482#comment-17855482 ] ASF GitHub Bot commented on HADOOP-19161: - virajjasani commented on PR #6789: URL: https://github.com/apache/hadoop/pull/6789#issuecomment-2172302849 Sounds good, i would take a detailed look at ConfigurationHelper and FlagSet soon. > S3A: option "fs.s3a.performance.flags" to take list of performance flags > > > Key: HADOOP-19161 > URL: https://issues.apache.org/jira/browse/HADOOP-19161 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.4.1 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > > HADOOP-19072 shows we want to add more optimisations than that of > HADOOP-18930. > * Extending the new optimisations to the existing option is brittle > * Adding explicit options for each feature gets complext fast. > Proposed > * A new class S3APerformanceFlags keeps all the flags > * it build this from a string[] of values, which can be extracted from > getConf(), > * and it can also support a "*" option to mean "everything" > * this class can also be handed off to hasPathCapability() and do the right > thing. > Proposed optimisations > * create file (we will hook up HADOOP-18930) > * mkdir (HADOOP-19072) > * delete (probe for parent path) > * rename (probe for source path) > We could think of more, with different names, later. > The goal is make it possible to strip out every HTTP request we do for > safety/posix compliance, so applications have the option of turning off what > they don't need. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-19161. S3A: option "fs.s3a.performance.flags" to take list of performance flags [hadoop]
virajjasani commented on PR #6789: URL: https://github.com/apache/hadoop/pull/6789#issuecomment-2172302849 Sounds good, i would take a detailed look at ConfigurationHelper and FlagSet soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[PR] HDFS-17523: Add fine-grained lock held info when the lock holding time reaches the specified threshold [hadoop]
ThinkerLei opened a new pull request, #6890: URL: https://github.com/apache/hadoop/pull/6890 Add fine-grained lock held info when the lock holding time reaches the specified threshold -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17523: Add fine-grained locks metrics in DataSetLockManager [hadoop]
ThinkerLei closed pull request #6886: HDFS-17523: Add fine-grained locks metrics in DataSetLockManager URL: https://github.com/apache/hadoop/pull/6886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17518: Sync the editslog if a file is closed in the lease monitor [hadoop]
hadoop-yetus commented on PR #6809:
URL: https://github.com/apache/hadoop/pull/6809#issuecomment-2171832002
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 18s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | checkstyle | 0m 39s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 1m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 2s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 29s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6809/7/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 125 unchanged
- 1 fixed = 126 total (was 126) |
| +1 :green_heart: | mvnsite | 0m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| +1 :green_heart: | spotbugs | 1m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 205m 36s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 291m 45s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.45 ServerAPI=1.45 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6809/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6809 |
| JIRA Issue | HDFS-17518 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d1d6938c241a 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17
09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1fdb6d153f2c7df0d7fe17effc3119d16028d404 |
| Default Java | Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_412-8u412-ga-1~20.04.1-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6809/7/testReport/ |
| Max. process+thread count | 4028 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6809/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
