date:20180626

Re: HADOOP-15124 review

2018-06-26 Thread Andrew Wang

I think it's fine to ask for review help on the dev list, sometimes JIRAs
are missed or get stuck. It also helps sometimes to git blame the files
you're touching and see who is a likely reviewer, and then pinging them on
JIRA to ask for a review.

Igor, hopefully a committer takes interest in your JIRA and helps you get
it integrated.

Best,
Andrew

On Tue, Jun 26, 2018 at 9:27 PM Igor Dvorzhak 
wrote:

> Hi Yiqun,
>
> Thank you for the explanation. I didn't know that this is not appropriate
> and will not do so in future.
>
> Thanks,
> Igor
>
>
> On Tue, Jun 26, 2018 at 7:18 PM Lin,Yiqun(vip.com) <
> yiqun01@vipshop.com> wrote:
>
>> Hi Igor,
>>
>> It’s not appropriate to ask for a review request in dev mailing list. Dev
>> mailing list is mainly used for discussing and answering user’s questions.
>> You can ask for the review under specific JIRA, that will be seen by
>> committers or others. If they have time, they will help take the review.
>>
>> Yiqun
>> Thanks
>>
>> 发件人: Igor Dvorzhak [mailto:i...@google.com.INVALID]
>> 发送时间: 2018年6月26日 23:52
>> 收件人: hdfs-...@hadoop.apache.org; common-dev@hadoop.apache.org
>> 主题: Re: HADOOP-15124 review
>>
>> +common-dev@hadoop.apache.org
>>
>> On Tue, Jun 26, 2018 at 8:49 AM Igor Dvorzhak > i...@google.com>> wrote:
>> Hello,
>>
>> I have a patch that
>> improves FileSystem.Statistics implementation and I would like to commit it.
>>
>> May somebody review it?
>>
>> Best regards,
>> Igor Dvorzhak
>> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人，谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容，或将其用于其他任何目的或向任何人披露。谢谢您的合作！
>> This communication is intended only for the addressee(s) and may contain
>> information that is privileged and confidential. You are hereby notified
>> that, if you are not an intended recipient listed above, or an authorized
>> employee or agent of an addressee of this communication responsible for
>> delivering e-mail messages to an intended recipient, any dissemination,
>> distribution or reproduction of this communication (including any
>> attachments hereto) is strictly prohibited. If you have received this
>> communication in error, please notify us immediately by a reply e-mail
>> addressed to the sender and permanently delete the original e-mail
>> communication and any attachments from all storage devices without making
>> or otherwise retaining a copy.
>>
>

Re: HADOOP-15124 review

2018-06-26 Thread Igor Dvorzhak

Hi Yiqun,

Thank you for the explanation. I didn't know that this is not appropriate
and will not do so in future.

Thanks,
Igor


On Tue, Jun 26, 2018 at 7:18 PM Lin,Yiqun(vip.com) 
wrote:

> Hi Igor,
>
> It’s not appropriate to ask for a review request in dev mailing list. Dev
> mailing list is mainly used for discussing and answering user’s questions.
> You can ask for the review under specific JIRA, that will be seen by
> committers or others. If they have time, they will help take the review.
>
> Yiqun
> Thanks
>
> 发件人: Igor Dvorzhak [mailto:i...@google.com.INVALID]
> 发送时间: 2018年6月26日 23:52
> 收件人: hdfs-...@hadoop.apache.org; common-dev@hadoop.apache.org
> 主题: Re: HADOOP-15124 review
>
> +common-dev@hadoop.apache.org
>
> On Tue, Jun 26, 2018 at 8:49 AM Igor Dvorzhak  i...@google.com>> wrote:
> Hello,
>
> I have a patch that
> improves FileSystem.Statistics implementation and I would like to commit it.
>
> May somebody review it?
>
> Best regards,
> Igor Dvorzhak
> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人，谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容，或将其用于其他任何目的或向任何人披露。谢谢您的合作！
> This communication is intended only for the addressee(s) and may contain
> information that is privileged and confidential. You are hereby notified
> that, if you are not an intended recipient listed above, or an authorized
> employee or agent of an addressee of this communication responsible for
> delivering e-mail messages to an intended recipient, any dissemination,
> distribution or reproduction of this communication (including any
> attachments hereto) is strictly prohibited. If you have received this
> communication in error, please notify us immediately by a reply e-mail
> addressed to the sender and permanently delete the original e-mail
> communication and any attachments from all storage devices without making
> or otherwise retaining a copy.
>


smime.p7s
Description: S/MIME Cryptographic Signature

[jira] [Created] (HADOOP-15565) ViewFileSystem.close doesn't close child filesystems and causes FileSystem objects leak.

2018-06-26 Thread Jinglun (JIRA)

Jinglun created HADOOP-15565:


 Summary: ViewFileSystem.close doesn't close child filesystems and 
causes FileSystem objects leak.
 Key: HADOOP-15565
 URL: https://issues.apache.org/jira/browse/HADOOP-15565
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jinglun


When we create a ViewFileSystem, all it's child filesystems will be cached by 
FileSystem.CACHE. Unless we close these child filesystems, they will stay in 
FileSystem.CACHE forever.
I think we should let FileSystem.CACHE cache ViewFileSystem only, and let 
ViewFileSystem cache all it's child filesystems. So we can close ViewFileSystem 
without leak and won't affect other ViewFileSystems.
I find this problem because i need to re-login my kerberos and renew 
ViewFileSystem periodically. Because FileSystem.CACHE.Key is based on 
UserGroupInformation, which changes everytime i re-login, I can't use the 
cached child filesystems when i new a ViewFileSystem. And because 
ViewFileSystem.close does nothing but remove itself from cache, i leak all it's 
child filesystems in cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

答复: HADOOP-15124 review

2018-06-26 Thread Lin,Yiqun(vip.com)

Hi Igor,

It’s not appropriate to ask for a review request in dev mailing list. Dev 
mailing list is mainly used for discussing and answering user’s questions. You 
can ask for the review under specific JIRA, that will be seen by committers or 
others. If they have time, they will help take the review.

Yiqun
Thanks

发件人: Igor Dvorzhak [mailto:i...@google.com.INVALID]
发送时间: 2018年6月26日 23:52
收件人: hdfs-...@hadoop.apache.org; common-dev@hadoop.apache.org
主题: Re: HADOOP-15124 review

+common-dev@hadoop.apache.org

On Tue, Jun 26, 2018 at 8:49 AM Igor Dvorzhak 
mailto:i...@google.com>> wrote:
Hello,

I have a patch that 
improves FileSystem.Statistics implementation and I would like to commit it.

May somebody review it?

Best regards,
Igor Dvorzhak
本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人，谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容，或将其用于其他任何目的或向任何人披露。谢谢您的合作！
 This communication is intended only for the addressee(s) and may contain 
information that is privileged and confidential. You are hereby notified that, 
if you are not an intended recipient listed above, or an authorized employee or 
agent of an addressee of this communication responsible for delivering e-mail 
messages to an intended recipient, any dissemination, distribution or 
reproduction of this communication (including any attachments hereto) is 
strictly prohibited. If you have received this communication in error, please 
notify us immediately by a reply e-mail addressed to the sender and permanently 
delete the original e-mail communication and any attachments from all storage 
devices without making or otherwise retaining a copy.

[jira] [Created] (HADOOP-15564) Classloading Shell should not run a subprocess

2018-06-26 Thread Todd Lipcon (JIRA)

Todd Lipcon created HADOOP-15564:


 Summary: Classloading Shell should not run a subprocess
 Key: HADOOP-15564
 URL: https://issues.apache.org/jira/browse/HADOOP-15564
 Project: Hadoop Common
  Issue Type: Improvement
  Components: util
Affects Versions: 3.0.0
Reporter: Todd Lipcon


The 'Shell' class has a static member isSetsidSupported which, in order to 
initialize, forks out a subprocess. Various other parts of the code reference 
Shell.WINDOWS. For example, the StringUtils class has such a reference. This 
means that, during startup, a seemingly fast call like 
Configuration.getBoolean() ends up class-loading StringUtils, which class-loads 
Shell, which forks out a subprocess. I couldn't measure any big improvement by 
fixing this, but seemed surprising to say the least.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15563) s3guard init and set-capacity to support DDB autoscaling

2018-06-26 Thread Steve Loughran (JIRA)

Steve Loughran created HADOOP-15563:
---

 Summary: s3guard init and set-capacity to support DDB autoscaling
 Key: HADOOP-15563
 URL: https://issues.apache.org/jira/browse/HADOOP-15563
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3
Affects Versions: 3.1.0
 Environment: To keep costs down on DDB, autoscaling is a key feature: 
you set the max values and when idle, you don't get billed, *at the cost of 
delayed scale time and risk of not getting the max value when AWS is busy*

It can be done from the AWS web UI, but not in the s3guard init and 
set-capacity calls

It can be done [through the 
API|https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/AutoScaling.HowTo.SDK.html]

Usual issues then: wiring up, CLI params, testing. It'll be hard to test.
Reporter: Steve Loughran






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15562) Add Software Load Balancing support for Hadoop/Yarn Process

2018-06-26 Thread Giovanni Matteo Fumarola (JIRA)

Giovanni Matteo Fumarola created HADOOP-15562:
-

 Summary: Add Software Load Balancing support for Hadoop/Yarn 
Process
 Key: HADOOP-15562
 URL: https://issues.apache.org/jira/browse/HADOOP-15562
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Giovanni Matteo Fumarola






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: HADOOP-15124 review

2018-06-26 Thread Igor Dvorzhak

+common-dev@hadoop.apache.org

On Tue, Jun 26, 2018 at 8:49 AM Igor Dvorzhak  wrote:

> Hello,
>
> I have a patch  that
> improves FileSystem.Statistics implementation and I would like to commit it.
>
> May somebody review it?
>
> Best regards,
> Igor Dvorzhak
>


smime.p7s
Description: S/MIME Cryptographic Signature

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-06-26 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/

[Jun 25, 2018 1:38:57 PM] (stevel) HADOOP-14396. Add builder interface to 
FileContext. Contributed by  Lei
[Jun 25, 2018 4:50:27 PM] (inigoiri) HADOOP-15458. 
TestLocalFileSystem#testFSOutputStreamBuilder fails on
[Jun 25, 2018 5:38:03 PM] (rohithsharmaks) YARN-8457. Compilation is broken 
with -Pyarn-ui.
[Jun 25, 2018 8:05:22 PM] (aengineer) HDDS-191. Queue SCMCommands via 
EventQueue in SCM. Contributed by Elek,
[Jun 25, 2018 8:59:41 PM] (mackrorysd) HADOOP-15423. Merge fileCache and 
dirCache into ine single cache in
[Jun 25, 2018 10:36:45 PM] (todd) HADOOP-15550. Avoid static initialization of 
ObjectMappers
[Jun 25, 2018 10:47:54 PM] (miklos.szegedi) YARN-8438. 
TestContainer.testKillOnNew flaky on trunk. Contributed by




-1 overall


The following subsystems voted -1:
asflicense findbugs pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.server.datanode.TestDirectoryScanner 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   
hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities 
   
hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity 
   hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun 
   hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps 
   
hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction
 
   
hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema 
   
hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain 
   
hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.mapred.TestMRTimelineEventHandling 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/diff-compile-javac-root.txt
  [352K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/diff-checkstyle-root.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/diff-patch-shelldocs.txt
  [16K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/whitespace-eol.txt
  [9.4M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/whitespace-tabs.txt
  [1.1M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/xml.txt
  [4.0K]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [48K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [56K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [24K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/branch-findbugs-hadoop-ozone_tools.txt
  [4.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/823/artifact/out/diff-javadoc-javadoc-root.txt
  [760K]

   unit:

   
h

[jira] [Created] (HADOOP-15561) Property Azure Account Key Provider Key requires blob.core.windows.net

2018-06-26 Thread Santiago Velasco (JIRA)

Santiago Velasco created HADOOP-15561:
-

 Summary: Property Azure Account Key Provider Key requires 
blob.core.windows.net
 Key: HADOOP-15561
 URL: https://issues.apache.org/jira/browse/HADOOP-15561
 Project: Hadoop Common
  Issue Type: Improvement
  Components: fs/azure
Affects Versions: 3.1.0
Reporter: Santiago Velasco


The documentation for Hadoop Azure Support: Azure Blob Storage

[http://hadoop.apache.org/docs/r3.1.0/hadoop-azure/index.html]

The properties under Protecting the Azure Credentials for WASB within an 
Encrypted File

The property fs.azure.account.keyprovider.youraccount requires 
_.blob.core.windows.net_

Documentation reads:

 
{code:java}

  fs.azure.account.keyprovider.youraccount
  org.apache.hadoop.fs.azure.ShellDecryptionKeyProvider

{code}
Should read:

 
{code:java}

  fs.azure.account.keyprovider.youraccount.blob.core.windows.net
  org.apache.hadoop.fs.azure.ShellDecryptionKeyProvider
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: HADOOP-15124 review

Re: HADOOP-15124 review

[jira] [Created] (HADOOP-15565) ViewFileSystem.close doesn't close child filesystems and causes FileSystem objects leak.

答复: HADOOP-15124 review

[jira] [Created] (HADOOP-15564) Classloading Shell should not run a subprocess

[jira] [Created] (HADOOP-15563) s3guard init and set-capacity to support DDB autoscaling

[jira] [Created] (HADOOP-15562) Add Software Load Balancing support for Hadoop/Yarn Process

Re: HADOOP-15124 review

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[jira] [Created] (HADOOP-15561) Property Azure Account Key Provider Key requires blob.core.windows.net

10 matches

Site Navigation

Mail list logo

Footer information