[jira] [Work logged] (HDFS-16453) Upgrade okhttp from 2.7.5 to 4.9.3
[ https://issues.apache.org/jira/browse/HDFS-16453?focusedWorklogId=786385&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-786385 ] ASF GitHub Bot logged work on HDFS-16453: - Author: ASF GitHub Bot Created on: 30/Jun/22 06:19 Start Date: 30/Jun/22 06:19 Worklog Time Spent: 10m Work Description: pan3793 commented on PR #4229: URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1170813260 okhttp 3.14.9 is the latest version which does not depend on kotlin Issue Time Tracking --- Worklog Id: (was: 786385) Time Spent: 1h 10m (was: 1h) > Upgrade okhttp from 2.7.5 to 4.9.3 > -- > > Key: HDFS-16453 > URL: https://issues.apache.org/jira/browse/HDFS-16453 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs-client >Affects Versions: 3.3.1 >Reporter: Ivan Viaznikov >Assignee: Ashutosh Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > {{org.apache.hadoop:hadoop-hdfs-client}} comes with > {{com.squareup.okhttp:okhttp:2.7.5}} as a dependency, which is vulnerable to > an information disclosure issue due to how the contents of sensitive headers, > such as the {{Authorization}} header, can be logged when an > {{IllegalArgumentException}} is thrown. > This issue could allow an attacker or malicious user who has access to the > logs to obtain the sensitive contents of the affected headers which could > facilitate further attacks. > Fixed in {{5.0.0-alpha3}} by > [this|https://github.com/square/okhttp/commit/dcc6483b7dc6d9c0b8e03ff7c30c13f3c75264a5] > commit. The fix was cherry-picked and backported into {{4.9.2}} with > [this|https://github.com/square/okhttp/commit/1fd7c0afdc2cee9ba982b07d49662af7f60e1518] > commit. > Requesting you to clarify if this dependency will be updated to a fixed > version in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16644) java.io.IOException Invalid token in javax.security.sasl.qop
Walter Su created HDFS-16644: Summary: java.io.IOException Invalid token in javax.security.sasl.qop Key: HDFS-16644 URL: https://issues.apache.org/jira/browse/HDFS-16644 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 3.2.1 Reporter: Walter Su deployment: server side: kerberos enabled cluster with jdk 1.8 and hdfs-server 3.2.1 client side: I run command hadoop fs -put a test file, with kerberos ticket inited first, and use identical core-site.xml & hdfs-site.xml configuration. using client ver 3.2.1, it succeeds. using client ver 2.8.5, it succeeds. using client ver 2.10.1, it fails. The client side error info is: org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2022-06-27 01:06:15,781 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DataNode{data=FSDataset{dirpath='[/mnt/disk1/hdfs, /mnt/***/hdfs, /mnt/***/hdfs, /mnt/***/hdfs]'}, localName='emr-worker-***.***:9866', datanodeUuid='b1c7f64a-6389-4739-bddf-***', xmitsInProgress=0}:Exception transfering block BP-1187699012-10.-***:blk_1119803380_46080919 to mirror 10.*:9866 java.io.IOException: Invalid token in javax.security.sasl.qop: D at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessage(DataTransferSaslUtil.java:220) Once any client ver 2.10.1 connect to hdfs server, the DataNode no longer accepts any client connection, even client ver 3.2.1 cannot connects to hdfs server. The DataNode rejects any client connection. For a short time, all DataNodes rejects client connections. The problem exists even if I replace DataNode with ver 3.3.0 or replace java with jdk 11. The problem is fixed if I replace DataNode with ver 3.2.0. I guess the problem is related to HDFS-13541 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16642) [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock
[ https://issues.apache.org/jira/browse/HDFS-16642?focusedWorklogId=786215&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-786215 ] ASF GitHub Bot logged work on HDFS-16642: - Author: ASF GitHub Bot Created on: 29/Jun/22 23:19 Start Date: 29/Jun/22 23:19 Worklog Time Spent: 10m Work Description: jojochuang commented on PR #4497: URL: https://github.com/apache/hadoop/pull/4497#issuecomment-1170586655 Looks thread-safe to me. Has this patch been tested in a production environment? Issue Time Tracking --- Worklog Id: (was: 786215) Time Spent: 0.5h (was: 20m) > [SBN read] Moving selecting inputstream from JN in EditlogTailer out of > FSNLock > --- > > Key: HDFS-16642 > URL: https://issues.apache.org/jira/browse/HDFS-16642 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > EditlogTailer cost a long time for selecting InputStreams from Journalnode > while holding the write lock of FSNLock. > During this period, 8020 handlers of Observer NameNode will be blocked by the > FSN Lock. > In theory, selecting inputstreams from JournalNode does not involve changing > memory information in NameNode, so we can move the selecting out of the FSN > Lock, and it can improve the throughput of Observer NameNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16642) [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock
[ https://issues.apache.org/jira/browse/HDFS-16642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-16642: --- Summary: [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock (was: [HDFS] Moving selecting inputstream from JN in EditlogTailer out of FSNLock) > [SBN read] Moving selecting inputstream from JN in EditlogTailer out of > FSNLock > --- > > Key: HDFS-16642 > URL: https://issues.apache.org/jira/browse/HDFS-16642 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > EditlogTailer cost a long time for selecting InputStreams from Journalnode > while holding the write lock of FSNLock. > During this period, 8020 handlers of Observer NameNode will be blocked by the > FSN Lock. > In theory, selecting inputstreams from JournalNode does not involve changing > memory information in NameNode, so we can move the selecting out of the FSN > Lock, and it can improve the throughput of Observer NameNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org