date:20230212

[jira] [Commented] (HADOOP-18501) [ABFS]: Partial Read should add to throttling metric

2023-02-12 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687773#comment-17687773
 ] 

ASF GitHub Bot commented on HADOOP-18501:
-

pranavsaxena-microsoft commented on PR #5109:
URL: https://github.com/apache/hadoop/pull/5109#issuecomment-1427469538

   Workload testing has to be done on the PR. Shall request you for review once 
that is done. Thanks.




> [ABFS]: Partial Read should add to throttling metric
> 
>
> Key: HADOOP-18501
> URL: https://issues.apache.org/jira/browse/HADOOP-18501
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/azure
>Affects Versions: 3.3.4
>Reporter: Pranav Saxena
>Assignee: Pranav Saxena
>Priority: Minor
>  Labels: pull-request-available
>
> Error Description:
> For partial read (due to account backend throttling), the ABFS driver retry 
> but doesn't add up in the throttling metrics.
> In case of partial read with connection-reset exception, ABFS driver retry 
> for the full request and doesn't add up in throttling metrics.
> Mitigation:
> In case of partial read, Abfs Driver should retry for the remaining bytes and 
> it should be added in throttling metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] pranavsaxena-microsoft commented on pull request #5109: HADOOP-18501: ABFS: Partial read should add to throttling data: DRAFT

2023-02-12 Thread via GitHub



pranavsaxena-microsoft commented on PR #5109:
URL: https://github.com/apache/hadoop/pull/5109#issuecomment-1427469538

   Workload testing has to be done on the PR. Shall request you for review once 
that is done. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18547) Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

2023-02-12 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687772#comment-17687772
 ] 

ASF GitHub Bot commented on HADOOP-18547:
-

pranavsaxena-microsoft commented on PR #5177:
URL: https://github.com/apache/hadoop/pull/5177#issuecomment-1427466685

   > LGTM, if its ready for review
   
   @steveloughran , requesting you for your kind review. Thanks.




> Check if config value is not empty string in 
> AbfsConfiguration.getMandatoryPasswordString()
> ---
>
> Key: HADOOP-18547
> URL: https://issues.apache.org/jira/browse/HADOOP-18547
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/azure
>Affects Versions: 3.3.4
>Reporter: Pranav Saxena
>Assignee: Pranav Saxena
>Priority: Minor
>  Labels: pull-request-available
>
> The method `getMandatoryPasswordString` is called in 
> `AbfsConfiguration.getTokenProvider()' to check if following configs are 
> non-null (diff keys applicable for different implementation of 
> AccessTokenProvider):
> 1. fs.azure.account.oauth2.client.endpoint: in ClientCredsTokenProvider
> 2. fs.azure.account.oauth2.client.id: in ClientCredsTokenProvider, 
> MsiTokenProvider, RefreshTokenBasedTokenProvider
> 3. fs.azure.account.oauth2.client.secret: in ClientCredsTokenProvider
> 4. fs.azure.account.oauth2.client.endpoint: in UserPasswordTokenProvider
> 5. fs.azure.account.oauth2.user.name: in UserPasswordTokenProvider
> 6. fs.azure.account.oauth2.user.password: in  UserPasswordTokenProvider
> 7. fs.azure.account.oauth2.msi.tenant: in MsiTokenProvider
> 8. fs.azure.account.oauth2.refresh.token: in RefreshTokenBasedTokenProvider
> Right now, this method checks if its non-null and not non-empty. This task 
> needs to add check on non-empty config values.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] pranavsaxena-microsoft commented on pull request #5177: HADOOP-18547. Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

2023-02-12 Thread via GitHub



pranavsaxena-microsoft commented on PR #5177:
URL: https://github.com/apache/hadoop/pull/5177#issuecomment-1427466685

   > LGTM, if its ready for review
   
   @steveloughran , requesting you for your kind review. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-18547) Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

2023-02-12 Thread Pranav Saxena (Jira)



 [ 
https://issues.apache.org/jira/browse/HADOOP-18547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pranav Saxena updated HADOOP-18547:
---
Description: 
The method `getMandatoryPasswordString` is called in 
`AbfsConfiguration.getTokenProvider()' to check if following configs are 
non-null (diff keys applicable for different implementation of 
AccessTokenProvider):

1. fs.azure.account.oauth2.client.endpoint: in ClientCredsTokenProvider
2. fs.azure.account.oauth2.client.id: in ClientCredsTokenProvider, 
MsiTokenProvider, RefreshTokenBasedTokenProvider
3. fs.azure.account.oauth2.client.secret: in ClientCredsTokenProvider
4. fs.azure.account.oauth2.client.endpoint: in UserPasswordTokenProvider
5. fs.azure.account.oauth2.user.name: in UserPasswordTokenProvider
6. fs.azure.account.oauth2.user.password: in  UserPasswordTokenProvider
7. fs.azure.account.oauth2.msi.tenant: in MsiTokenProvider
8. fs.azure.account.oauth2.refresh.token: in RefreshTokenBasedTokenProvider

Right now, this method checks if its non-null and not non-empty. This task 
needs to add check on non-empty config values.

> Check if config value is not empty string in 
> AbfsConfiguration.getMandatoryPasswordString()
> ---
>
> Key: HADOOP-18547
> URL: https://issues.apache.org/jira/browse/HADOOP-18547
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs/azure
>Affects Versions: 3.3.4
>Reporter: Pranav Saxena
>Assignee: Pranav Saxena
>Priority: Minor
>  Labels: pull-request-available
>
> The method `getMandatoryPasswordString` is called in 
> `AbfsConfiguration.getTokenProvider()' to check if following configs are 
> non-null (diff keys applicable for different implementation of 
> AccessTokenProvider):
> 1. fs.azure.account.oauth2.client.endpoint: in ClientCredsTokenProvider
> 2. fs.azure.account.oauth2.client.id: in ClientCredsTokenProvider, 
> MsiTokenProvider, RefreshTokenBasedTokenProvider
> 3. fs.azure.account.oauth2.client.secret: in ClientCredsTokenProvider
> 4. fs.azure.account.oauth2.client.endpoint: in UserPasswordTokenProvider
> 5. fs.azure.account.oauth2.user.name: in UserPasswordTokenProvider
> 6. fs.azure.account.oauth2.user.password: in  UserPasswordTokenProvider
> 7. fs.azure.account.oauth2.msi.tenant: in MsiTokenProvider
> 8. fs.azure.account.oauth2.refresh.token: in RefreshTokenBasedTokenProvider
> Right now, this method checks if its non-null and not non-empty. This task 
> needs to add check on non-empty config values.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

2023-02-12 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687716#comment-17687716
 ] 

ASF GitHub Bot commented on HADOOP-18524:
-

aajisaka commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1427331696

   Cool. If this PR is merged, I can disable 
https://github.com/aajisaka/hadoop-document




> Deploy Hadoop trunk version website
> ---
>
> Key: HADOOP-18524
> URL: https://issues.apache.org/jira/browse/HADOOP-18524
> Project: Hadoop Common
>  Issue Type: Task
>  Components: documentation
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
>  Labels: pull-request-available
>
> Have a link to the running trunk documentation in the hadoop site.
> A lot of projects like ozone, iceberg have a running documentation for the 
> master branch version.
> Good to have for Hadoop as well



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] aajisaka commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

2023-02-12 Thread via GitHub



aajisaka commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1427331696

   Cool. If this PR is merged, I can disable 
https://github.com/aajisaka/hadoop-document


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] slfan1989 commented on pull request #5244: YARN-11349. [Federation] Router Support DelegationToken With SQL.

2023-02-12 Thread via GitHub



slfan1989 commented on PR #5244:
URL: https://github.com/apache/hadoop/pull/5244#issuecomment-1427244187

   ### SQL Server 2008 R2 Enterprise
   
   
   > select @@version
   ```
   Microsoft SQL Server 2008 R2 (SP3) - 10.50.6000.34 (X64) 
Aug 19 2014 12:21:34 
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows NT 6.3  (Build 9600: )
   ```
   
   > FederationStateStoreDatabase.sql
   ```
   master> IF DB_ID ( '[FederationStateStore]') IS NOT NULL
 DROP DATABASE [FederationStateStore]
   [2023-01-25 17:22:05] completed in 41 ms
   master> CREATE database FederationStateStore
   [2023-01-25 17:22:05] completed in 106 ms
   ```
   
   > FederationStateStoreUser.sql
   ```
   master> USE [FederationStateStore]
   [2023-01-25 17:23:20] [S0001][5701] Changed database context to 
'FederationStateStore'.
   [2023-01-25 17:23:20] completed in 41 ms
   master> CREATE LOGIN FederationUser with password = 'Federation@Password', 
default_database=[FederationStateStore]
   [2023-01-25 17:23:21] completed in 51 ms
   master> CREATE USER FederationUser FOR LOGIN FederationUser WITH 
default_schema=dbo
   [2023-01-25 17:23:21] completed in 41 ms
   master> EXEC sp_addrolemember 'db_owner', 'FederationUser'
   [2023-01-25 17:23:21] completed in 41 ms
   ```
   
   > FederationStateStoreTables.sql
   ```
   
   Table delegationTokens exists, no operation required...
   [2023-01-25 17:24:34] completed in 44 ms
   FederationStateStore> IF NOT EXISTS ( SELECT * FROM 
[FederationStateStore].sys.tables
 WHERE name = 'sequenceTable'
 AND schema_id = SCHEMA_ID('dbo'))
 BEGIN
 PRINT 'Table sequenceTable does not exist, 
create it...'
 
 SET ANSI_NULLS ON
 
 SET QUOTED_IDENTIFIER ON
 
 SET ANSI_PADDING ON
 
 CREATE TABLE [dbo].[sequenceTable](
 sequenceName VARCHAR(255) NOT NULL,
 nextVal bigint NOT NULL
 CONSTRAINT [pk_sequenceName] PRIMARY KEY
 (
 [sequenceName]
 )
 )
 
 SET ANSI_PADDING OFF
 
 PRINT 'Table sequenceTable created.'
 END
 ELSE
 PRINT 'Table sequenceTable exists, no operation 
required...'
   Table sequenceTable does not exist, create it...
   Table sequenceTable created.
   ```
   
   ```
   USE [FederationStateStore]
   GO
   SELECT Name FROM SysObjects Where XType='U' ORDER BY Name
   
   applicationsHomeSubCluster
   delegationTokens
   masterKeys
   membership
   policies
   reservationsHomeSubCluster
   sequenceTable
   ```
   
   > FederationStateStoreStoredProcs.sql
   
   ```
   [2023-01-25 17:53:48] completed in 39 ms
   FederationStateStore> CREATE PROCEDURE [dbo].[sp_deleteDelegationToken]
 @sequenceNum_IN bigint,
 @rowCount_OUT int OUTPUT
 AS BEGIN
 DECLARE @errorMessage nvarchar(4000)
 
 BEGIN TRY
 BEGIN TRAN
 
 DELETE FROM [dbo].[delegationTokens]
 WHERE [sequenceNum] = @sequenceNum_IN;
 SELECT @rowCount_OUT = @@ROWCOUNT;
 
 COMMIT TRAN
 END TRY
 
 BEGIN CATCH
 ROLLBACK TRAN
 
 SET @errorMessage = 
dbo.func_FormatErrorMessage(ERROR_MESSAGE(), ERROR_LINE())
 
 /*  raise error and terminate the execution */
 RAISERROR(@errorMessage, --- Error Message
 1, -- Severity
 -1 -- State
 ) WITH log
 END CATCH
 END;
   [2023-01-25 17:53:48] completed in 44 ms
   ```
   ```
   USE [FederationStateStore]
   GO
   SELECT Name FROM SysObjects Where XType='P' ORDER BY Name
   
   sp_addApplicationHomeSubCluster
   sp_addDelegationToken
   sp_addMasterKey
   sp_addReservationHomeSubCluster

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

2023-02-12 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687568#comment-17687568
 ] 

ASF GitHub Bot commented on HADOOP-18524:
-

ayushtkn commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1427023727

   This is how I plan to link this to our website: **Current (trunk)** shall 
point to the doc generated by this Github Action
   https://user-images.githubusercontent.com/25608848/218311774-395204eb-cf63-473e-9322-95d3a9174b28.png;>
   
   




> Deploy Hadoop trunk version website
> ---
>
> Key: HADOOP-18524
> URL: https://issues.apache.org/jira/browse/HADOOP-18524
> Project: Hadoop Common
>  Issue Type: Task
>  Components: documentation
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
>  Labels: pull-request-available
>
> Have a link to the running trunk documentation in the hadoop site.
> A lot of projects like ozone, iceberg have a running documentation for the 
> master branch version.
> Good to have for Hadoop as well



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ayushtkn commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

2023-02-12 Thread via GitHub



ayushtkn commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1427023727

   This is how I plan to link this to our website: **Current (trunk)** shall 
point to the doc generated by this Github Action
   https://user-images.githubusercontent.com/25608848/218311774-395204eb-cf63-473e-9322-95d3a9174b28.png;>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

2023-02-12 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687554#comment-17687554
 ] 

ASF GitHub Bot commented on HADOOP-18524:
-

hadoop-yetus commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1426996059

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 51s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  yamllint  |   0m  0s |  |  yamllint was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  shadedclient  |  35m 52s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  shadedclient  |  25m 24s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 35s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   |  64m 29s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.42 ServerAPI=1.42 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5386 |
   | Optional Tests | dupname asflicense codespell detsecrets yamllint |
   | uname | Linux f87081add2b2 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 65fb533271fecfcad00429b028621f3a2d89b908 |
   | Max. process+thread count | 536 (vs. ulimit of 5500) |
   | modules | C: . U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/2/console |
   | versions | git=2.25.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   




> Deploy Hadoop trunk version website
> ---
>
> Key: HADOOP-18524
> URL: https://issues.apache.org/jira/browse/HADOOP-18524
> Project: Hadoop Common
>  Issue Type: Task
>  Components: documentation
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
>  Labels: pull-request-available
>
> Have a link to the running trunk documentation in the hadoop site.
> A lot of projects like ozone, iceberg have a running documentation for the 
> master branch version.
> Good to have for Hadoop as well



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

2023-02-12 Thread via GitHub



hadoop-yetus commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1426996059

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 51s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  yamllint  |   0m  0s |  |  yamllint was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  shadedclient  |  35m 52s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  shadedclient  |  25m 24s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 35s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   |  64m 29s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.42 ServerAPI=1.42 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5386 |
   | Optional Tests | dupname asflicense codespell detsecrets yamllint |
   | uname | Linux f87081add2b2 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 65fb533271fecfcad00429b028621f3a2d89b908 |
   | Max. process+thread count | 536 (vs. ulimit of 5500) |
   | modules | C: . U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/2/console |
   | versions | git=2.25.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5329: HDFS-16897. fix abundant Broken pipe exception in BlockSender

2023-02-12 Thread via GitHub



ayushtkn commented on code in PR #5329:
URL: https://github.com/apache/hadoop/pull/5329#discussion_r1103772620


##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java:
##
@@ -655,7 +655,7 @@ private int sendPacket(ByteBuffer pkt, int maxChunks, 
OutputStream out,
   if (ioem.startsWith(EIO_ERROR)) {
 throw new DiskFileCorruptException("A disk IO error occurred", e);
   }
-  if (!ioem.startsWith("Broken pipe")
+  if (!ioem.contains("Broken pipe")
   && !ioem.startsWith("Connection reset")) {

Review Comment:
   If I catch it right, in your use case, the cause had an exception, which 
started with Broken Pipe?
   I would say extract the the cause if not null, then do this entire check. 
With your present approach in your case. I doubt if 
``ioem.startsWith("Connection reset")`` this won't work properly, if that is 
also inside the cause, Second doing a contains doesn't look like very safe to 
me either.
   
   Check if something like this can work
   ```
 String causeMessage = e.getCause() != null ? 
e.getCause().getMessage() : "";
 if (!ioem.startsWith("Broken pipe") && 
!ioem.startsWith("Connection reset") &&
 !causeMessage.startsWith("Broken pipe") && 
!causeMessage.startsWith("Connection reset")) {
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

2023-02-12 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687542#comment-17687542
 ] 

ASF GitHub Bot commented on HADOOP-18524:
-

hadoop-yetus commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1426969184

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 52s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  yamllint  |   0m  0s |  |  yamllint was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  shadedclient  |  35m 59s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/1/artifact/out/blanks-eol.txt)
 |  The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | +1 :green_heart: |  shadedclient  |  25m 28s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 37s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   |  64m 40s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.42 ServerAPI=1.42 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5386 |
   | Optional Tests | dupname asflicense codespell detsecrets yamllint |
   | uname | Linux d1b1f1d62faa 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 87d3ef2b453812e648875a2e6c24b8c45a4c679b |
   | Max. process+thread count | 632 (vs. ulimit of 5500) |
   | modules | C: . U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/1/console |
   | versions | git=2.25.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   




> Deploy Hadoop trunk version website
> ---
>
> Key: HADOOP-18524
> URL: https://issues.apache.org/jira/browse/HADOOP-18524
> Project: Hadoop Common
>  Issue Type: Task
>  Components: documentation
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
>  Labels: pull-request-available
>
> Have a link to the running trunk documentation in the hadoop site.
> A lot of projects like ozone, iceberg have a running documentation for the 
> master branch version.
> Good to have for Hadoop as well



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

2023-02-12 Thread via GitHub



hadoop-yetus commented on PR #5386:
URL: https://github.com/apache/hadoop/pull/5386#issuecomment-1426969184

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 52s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  yamllint  |   0m  0s |  |  yamllint was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  shadedclient  |  35m 59s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/1/artifact/out/blanks-eol.txt)
 |  The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | +1 :green_heart: |  shadedclient  |  25m 28s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 37s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   |  64m 40s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.42 ServerAPI=1.42 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5386 |
   | Optional Tests | dupname asflicense codespell detsecrets yamllint |
   | uname | Linux d1b1f1d62faa 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 87d3ef2b453812e648875a2e6c24b8c45a4c679b |
   | Max. process+thread count | 632 (vs. ulimit of 5500) |
   | modules | C: . U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5386/1/console |
   | versions | git=2.25.1 maven=3.6.3 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-18501) [ABFS]: Partial Read should add to throttling metric

[GitHub] [hadoop] pranavsaxena-microsoft commented on pull request #5109: HADOOP-18501: ABFS: Partial read should add to throttling data: DRAFT

[jira] [Commented] (HADOOP-18547) Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

[GitHub] [hadoop] pranavsaxena-microsoft commented on pull request #5177: HADOOP-18547. Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

[jira] [Updated] (HADOOP-18547) Check if config value is not empty string in AbfsConfiguration.getMandatoryPasswordString()

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

[GitHub] [hadoop] aajisaka commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

[GitHub] [hadoop] slfan1989 commented on pull request #5244: YARN-11349. [Federation] Router Support DelegationToken With SQL.

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

[GitHub] [hadoop] ayushtkn commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

[GitHub] [hadoop] hadoop-yetus commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5329: HDFS-16897. fix abundant Broken pipe exception in BlockSender

[jira] [Commented] (HADOOP-18524) Deploy Hadoop trunk version website

[GitHub] [hadoop] hadoop-yetus commented on pull request #5386: HADOOP-18524. Deploy Hadoop trunk version website.

15 matches

Site Navigation

Mail list logo

Footer information