[jira] [Created] (HADOOP-8647) BlockCompressionStream won't work with BlockDecompressionStream when there are several write

[clockfly (JIRA)]

clockfly created HADOOP-8647:


 Summary: BlockCompressionStream won't work with 
BlockDecompressionStream when there are several write
 Key: HADOOP-8647
 URL: https://issues.apache.org/jira/browse/HADOOP-8647
 Project: Hadoop Common
  Issue Type: Bug
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: clockfly
Priority: Minor
 Attachments: TestBlockDecompressorStream.java

BlockDecompressionStream can not read compressed data using 
BlockCompressionStream when there are multiple writes to BlockCompressionStream.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8646) Avoid logging "Cannot run program getconf" on Windows

[Bikas Saha (JIRA)]

Bikas Saha created HADOOP-8646:
--

 Summary: Avoid logging "Cannot run program getconf" on Windows
 Key: HADOOP-8646
 URL: https://issues.apache.org/jira/browse/HADOOP-8646
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Bikas Saha
Assignee: Bikas Saha


ProcfsbasesProcessTree logs error messages when it cannot run getconf to 
determine system attributes on linux. this causes a lot of log spew on windows. 
need to fix this code because linux is not longer the only OS supported for 
hadoop.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8645) Stabilize branch-1-win

[Bikas Saha (JIRA)]

Bikas Saha created HADOOP-8645:
--

 Summary: Stabilize branch-1-win 
 Key: HADOOP-8645
 URL: https://issues.apache.org/jira/browse/HADOOP-8645
 Project: Hadoop Common
  Issue Type: Task
Reporter: Bikas Saha


Most of the code changes to make Hadoop branch 1 work natively on Windows are 
done. This jira in intended to track the work needed to achieve 100% test pass 
for the dev tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HADOOP-8641) handleConnectionFailure(..) in Client.java should properly handle interrupted exception

[Tsz Wo (Nicholas), SZE (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo (Nicholas), SZE resolved HADOOP-8641.


Resolution: Duplicate

Agree.  Let's resolve this as a duplicate of HADOOP-6221.

> handleConnectionFailure(..) in Client.java should properly handle interrupted 
> exception
> ---
>
> Key: HADOOP-8641
> URL: https://issues.apache.org/jira/browse/HADOOP-8641
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: suja s
>
> If connection retries are happening and thread is interrupted the 
> interruption is not happening and retries will continue till max number of 
> retries configured.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8644) AuthenticatedURL should be able to use SSLFactory

[Alejandro Abdelnur (JIRA)]

Alejandro Abdelnur created HADOOP-8644:
--

 Summary: AuthenticatedURL should be able to use SSLFactory
 Key: HADOOP-8644
 URL: https://issues.apache.org/jira/browse/HADOOP-8644
 Project: Hadoop Common
  Issue Type: New Feature
  Components: security
Affects Versions: 2.2.0-alpha
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Priority: Critical
 Fix For: 2.2.0-alpha


This is required to enable the use of HTTPS with SPNEGO using Hadoop configured 
keystores. This is required by HADOOP-8581.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8643) hadoop-client should exclude hadoop-annotations from hadoop-common dependency

[Alejandro Abdelnur (JIRA)]

Alejandro Abdelnur created HADOOP-8643:
--

 Summary: hadoop-client should exclude hadoop-annotations from 
hadoop-common dependency
 Key: HADOOP-8643
 URL: https://issues.apache.org/jira/browse/HADOOP-8643
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 2.2.0-alpha
Reporter: Alejandro Abdelnur
 Fix For: 2.2.0-alpha


When reviewing HADOOP-8370 I've missed that changing the scope to compile for 
hadoop-annotations in hadoop-common it would make hadoop-annotations to bubble 
up in hadoop-client. Because of this we need to explicitly exclude it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8642) io.native.lib.available only controls zlib

[Eli Collins (JIRA)]

Eli Collins created HADOOP-8642:
---

 Summary: io.native.lib.available only controls zlib
 Key: HADOOP-8642
 URL: https://issues.apache.org/jira/browse/HADOOP-8642
 Project: Hadoop Common
  Issue Type: Bug
  Components: native
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins


Per core-default.xml {{io.native.lib.available}} indicates "Should native 
hadoop libraries, if present, be used" however it looks like it only affects 
zlib. Since we always load the native library this means we may use native 
libraries even if io.native.lib.available is set to false.

Let's make the flag to work as advertised - rather than always load the native 
hadoop library we only attempt to load the library (and report that native is 
available) if this flag is set. Since io.native.lib.available defaults to true 
the default behavior should remain unchanged (except that now we wont actually 
try to load the library if this flag is disabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8641) handleConnectionFailure(..) in Client.java should properly handle interrupted exception

[suja s (JIRA)]

suja s created HADOOP-8641:
--

 Summary: handleConnectionFailure(..) in Client.java should 
properly handle interrupted exception
 Key: HADOOP-8641
 URL: https://issues.apache.org/jira/browse/HADOOP-8641
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 2.0.1-alpha
Reporter: suja s


If connection retries are happening and thread is interrupted the interruption 
is not happening and retries will continue till max number of retries 
configured.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hadoop-Common-trunk #491

[Apache Jenkins Server]

See 

Changes:

[tucu] HDFS-3724. add InterfaceAudience annotations to HttpFS classes and 
making inner enum static. (tucu)

[tucu] HDFS-3513. HttpFS should cache filesystems. (tucu)

[tgraves] MAPREDUCE-4504. SortValidator writes to wrong directory (Robert Evans 
via tgraves)

[eli] HADOOP-8480. The native build should honor -DskipTests. Contributed by 
Colin Patrick McCabe

[eli] HADOOP-8620. Add -Drequire.fuse and -Drequire.snappy. Contributed by 
Colin Patrick McCabe

[tgraves] HADOOP-8633. Interrupted FsShell copies may leave tmp files (Daryn 
Sharp via tgraves)

--
[...truncated 26545 lines...]
[DEBUG]   (s) debug = false
[DEBUG]   (s) effort = Default
[DEBUG]   (s) failOnError = true
[DEBUG]   (s) findbugsXmlOutput = false
[DEBUG]   (s) findbugsXmlOutputDirectory = 

[DEBUG]   (s) fork = true
[DEBUG]   (s) includeTests = false
[DEBUG]   (s) localRepository =id: local
  url: file:///home/jenkins/.m2/repository/
   layout: none

[DEBUG]   (s) maxHeap = 512
[DEBUG]   (s) nested = false
[DEBUG]   (s) outputDirectory = 

[DEBUG]   (s) outputEncoding = UTF-8
[DEBUG]   (s) pluginArtifacts = 
[org.codehaus.mojo:findbugs-maven-plugin:maven-plugin:2.3.2:, 
com.google.code.findbugs:bcel:jar:1.3.9:compile, 
org.codehaus.gmaven:gmaven-mojo:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-api:jar:1.3:compile, 
org.codehaus.gmaven.feature:gmaven-feature-api:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-1.5:jar:1.3:compile, 
org.codehaus.gmaven.feature:gmaven-feature-support:jar:1.3:compile, 
org.codehaus.groovy:groovy-all-minimal:jar:1.5.8:compile, 
org.apache.ant:ant:jar:1.7.1:compile, 
org.apache.ant:ant-launcher:jar:1.7.1:compile, jline:jline:jar:0.9.94:compile, 
org.codehaus.plexus:plexus-interpolation:jar:1.1:compile, 
org.codehaus.gmaven:gmaven-plugin:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-loader:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-support:jar:1.3:compile, 
org.sonatype.gshell:gshell-io:jar:2.0:compile, 
com.thoughtworks.qdox:qdox:jar:1.10:compile, 
org.apache.maven.shared:file-management:jar:1.2.1:compile, 
org.apache.maven.shared:maven-shared-io:jar:1.1:compile, 
commons-lang:commons-lang:jar:2.4:compile, 
org.slf4j:slf4j-api:jar:1.5.10:compile, 
org.sonatype.gossip:gossip:jar:1.2:compile, 
org.apache.maven.reporting:maven-reporting-impl:jar:2.1:compile, 
commons-validator:commons-validator:jar:1.2.0:compile, 
commons-beanutils:commons-beanutils:jar:1.7.0:compile, 
commons-digester:commons-digester:jar:1.6:compile, 
commons-logging:commons-logging:jar:1.0.4:compile, oro:oro:jar:2.0.8:compile, 
xml-apis:xml-apis:jar:1.0.b2:compile, 
org.codehaus.groovy:groovy-all:jar:1.7.4:compile, 
org.apache.maven.reporting:maven-reporting-api:jar:3.0:compile, 
org.apache.maven.doxia:doxia-core:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-logging-api:jar:1.1.3:compile, 
xerces:xercesImpl:jar:2.9.1:compile, 
commons-httpclient:commons-httpclient:jar:3.1:compile, 
commons-codec:commons-codec:jar:1.2:compile, 
org.apache.maven.doxia:doxia-sink-api:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-decoration-model:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-site-renderer:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-module-xhtml:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-module-fml:jar:1.1.3:compile, 
org.codehaus.plexus:plexus-i18n:jar:1.0-beta-7:compile, 
org.codehaus.plexus:plexus-velocity:jar:1.1.7:compile, 
org.apache.velocity:velocity:jar:1.5:compile, 
commons-collections:commons-collections:jar:3.2:compile, 
org.apache.maven.shared:maven-doxia-tools:jar:1.2.1:compile, 
commons-io:commons-io:jar:1.4:compile, 
com.google.code.findbugs:findbugs-ant:jar:1.3.9:compile, 
com.google.code.findbugs:findbugs:jar:1.3.9:compile, 
com.google.code.findbugs:jsr305:jar:1.3.9:compile, 
com.google.code.findbugs:jFormatString:jar:1.3.9:compile, 
com.google.code.findbugs:annotations:jar:1.3.9:compile, 
dom4j:dom4j:jar:1.6.1:compile, jaxen:jaxen:jar:1.1.1:compile, 
jdom:jdom:jar:1.0:compile, xom:xom:jar:1.0:compile, 
xerces:xmlParserAPIs:jar:2.6.2:compile, xalan:xalan:jar:2.6.0:compile, 
com.ibm.icu:icu4j:jar:2.6.1:compile, asm:asm:jar:3.1:compile, 
asm:asm-analysis:jar:3.1:compile, asm:asm-commons:jar:3.1:compile, 
asm:asm-util:jar:3.1:compile, asm:asm-tree:jar:3.1:compile, 
asm:asm-xml:jar:3.1:compile, jgoodies:plastic:jar:1.2.0:compile, 
org.codehaus.plexus:plexus-resources:jar:1.0-alpha-4:compile, 
org.codehaus.plexus:plexus-utils:jar:1.5.1:compile]
[DEBUG]   (s) project = MavenProject: 
org.apache.hadoop:hadoop-common-project:3.0.0-SNAPSHOT @ 

[DEBUG]   (s) relaxe

[jira] [Created] (HADOOP-8640) DU thread transient failures propagate to callers

[Todd Lipcon (JIRA)]

Todd Lipcon created HADOOP-8640:
---

 Summary: DU thread transient failures propagate to callers
 Key: HADOOP-8640
 URL: https://issues.apache.org/jira/browse/HADOOP-8640
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, io
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon


When running some stress tests, I saw a failure where the DURefreshThread 
failed due to the filesystem changing underneath it:

{code}
org.apache.hadoop.util.Shell$ExitCodeException: du: cannot access 
`/data/4/dfs/dn/current/BP-1928785663-172.20.90.20-1343880685858/current/rbw/blk_4637779214690837894':
 No such file or directory
{code}
(the block was probably finalized while the du process was running, which 
caused it to fail)

The next block write, then, called {{getUsed()}}, and the exception got 
propagated causing the write to fail. Since it was a pseudo-distributed 
cluster, the client was unable to pick a different node to write to and failed.

The current behavior of propagating the exception to the next (and only the 
next) caller doesn't seem well-thought-out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira