test failures - newbie help needed
Hi Please excuse me. I am new to building Hadoop. I am attempting to build and run tests for the trunk and encountering a series of test failures which for sure indicates that I am doing something wrong. I have scoured the wiki and can't see what I'm missing. Can anybody offer any advice? I am running the tests in Ubuntu 14.04 hosted in VirtualBox on Windows 8.1 host. Using this command: "mvn clean test" List of failed tests below. Many thanks Simon Failed tests: TestHDFSFileSystemContract>FileSystemContractBaseTest.testRenameDirectoryAsExistingDirectory:536 Renamed nested file1 exists TestTrashWithEncryptionZones.testDeleteEZWithMultipleUsers:164 Non-admin could delete an encryption zone with multiple users : /zones/zone1 expected:<1> but was:<0> TestDataNodeVolumeFailure.testFailedVolumeBeingRemovedFromDataNode:247 expected:<1> but was:<2> TestFileContextXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed TestFileContextXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown TestFileContextXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException TestFileContextXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException TestNameNodeXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed TestNameNodeXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown TestNameNodeXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException TestNameNodeXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException TestNamenodeCapacityReport.testXceiverCount:198->testXceiverCountInternal:344 Test resulted in an unexpected exit TestEncryptionZonesWithKMS>TestEncryptionZones.testGetEZAsNonSuperUser:581 expected AccessControlException TestEncryptionZonesWithKMS>TestEncryptionZones.testBasicOperations:441 createEncryptionZone is superuser-only operation TestReservedRawPaths.testAdminAccessOnly:262 access to /.reserved/raw is superuser-only operation TestDFSShell.testSetXAttrPermission:2992 Returned should be 1 expected:<1> but was:<0> TestWebHDFSXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed TestWebHDFSXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown TestWebHDFSXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException TestWebHDFSXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException TestWebHdfsFileSystemContract>FileSystemContractBaseTest.testRenameDirectoryAsExistingDirectory:536 Renamed nested file1 exists TestEncryptionZones.testGetEZAsNonSuperUser:581 expected AccessControlException TestEncryptionZones.testBasicOperations:441 createEncryptionZone is superuser-only operation Tests in error: TestRBWBlockInvalidation.testBlockInvalidationWhenRBWReplicaMissedInDN:122 > ... TestDataNodeVolumeFailure.tearDown:142->Object.wait:-2 > test timed out after... TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 > test timed ... TestDataNodeVolumeFailureReporting.tearDown:109 > test timed out after 12... TestDataNodeVolumeFailureReporting.tearDown:109 > test timed out after 12... TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 > test timed ... TestFileContextXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Remote N... TestNameNodeMetadataConsistency.testGenerationStampInFuture:109->waitForNumBytes:161 > Timeout TestNameNodeXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Remote No m... TestFSImage.testCompression:96->setCompressCodec:102->testPersistHelper:108 > IO TestDiskBalancerCommand.testRunMultipleCommandsUnderOneSetup > Remote File /sy... TestDiskBalancer.testBalanceDataBetweenMultiplePairsOfVolumes:187 > IllegalArgument TestDiskBalancerRPC.testMoveBlockAcrossVolume > Remote File /tmp.txt could onl... TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.setup:91 > DiskError TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testFileSmallerThanOneStripe2:132->TestDFSStripedOutputStream.testOneFile:187 > Timeout TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testFileMoreThanABlockGroup3:176->TestDFSStripedOutputStream.testOneFile:186 > IO TestParallelRead.setupCluster:37->TestParallelReadUtil.setupCluster:71 > IO Ca... TestParallelRead.teardownCluster:42->TestParallelReadUtil.teardownCluster:394 > NullPointer TestSaslDataTransfer.t
[jira] [Created] (HADOOP-14157) FsUrlStreamHandlerFactory "Illegal character in path" parsing file URL on Windows
Simon Scott created HADOOP-14157:
Summary: FsUrlStreamHandlerFactory "Illegal character in path"
parsing file URL on Windows
Key: HADOOP-14157
URL: https://issues.apache.org/jira/browse/HADOOP-14157
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 3.0.0-alpha2, 2.6.5, 2.7.3
Environment: Windows
Reporter: Simon Scott
Priority: Minor
After registering the FsUrlStreamHandlerFactory with the JVM, subsequent calls
to convert a "file" URL to a URI can fail with "Illegal character in path"
where the illegal character is a backslash.
For example:
{code}
URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
File file = new File("C:/Users");
final URL url = new URL("file:///" + file.getAbsolutePath());
{code}
gives stack trace:
{noformat}
java.net.URISyntaxException: Illegal character in path at index 8:
file:/C:\Users
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.checkChars(URI.java:3021)
at java.net.URI$Parser.parseHierarchical(URI.java:3105)
at java.net.URI$Parser.parse(URI.java:3053)
at java.net.URI.(URI.java:588)
at java.net.URL.toURI(URL.java:946)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
RE: FsUrlStreamHandlerFactory and Windows file URLs
Yes it is. But the registration of the FsUrlStreamHandlerFactory has surely
influenced the JDK in some manner?
-Original Message-
From: Steve Loughran [mailto:ste...@hortonworks.com]
Sent: 03 March 2017 21:49
Cc: common-dev@hadoop.apache.org
Subject: Re: FsUrlStreamHandlerFactory and Windows file URLs
That stack trace is coming from the java code before it his Hadoop
> On 3 Mar 2017, at 13:30, Simon Scott wrote:
>
> Apologies if this an old topic, however the following test program:
>
> package com.viavi;
>
> import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;
>
> import java.io.File;
> import java.net.MalformedURLException; import java.net.URI; import
> java.net.URISyntaxException; import java.net.URL;
>
> public class Main {
>
>public static void main(String[] args) {
>System.out.println("URI is " + makeURI());
>URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
>System.out.println("URI is " + makeURI());
>}
>
>private static URI makeURI() {
>try {
>File file = new File("C:/Users");
>final URL url = new URL("file:///" + file.getAbsolutePath());
>return url.toURI(); ** HERE**
Ty creating the URI via File.toURI();
>} catch (MalformedURLException x) {
>x.printStackTrace();
>return null;
>} catch (URISyntaxException x) {
>x.printStackTrace();
>return null;
>}
>}
> }
>
> gives the following output:
>
> URI is file:/C:/Users
> URI is null
> java.net.URISyntaxException: Illegal character in path at index 8:
> file:/C:\Users
>at java.net.URI$Parser.fail(URI.java:2848)
>at java.net.URI$Parser.checkChars(URI.java:3021)
>at java.net.URI$Parser.parseHierarchical(URI.java:3105)
>at java.net.URI$Parser.parse(URI.java:3053)
>at java.net.URI.(URI.java:588)
>at java.net.URL.toURI(URL.java:946)
>at com.viavi.Main.makeURI(Main.java:23)
>at com.viavi.Main.main(Main.java:16)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>at java.lang.reflect.Method.invoke(Method.java:498)
>at
> com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
>
> That is, it seems that registering the FsUrlStreamHandlerFactory disrupts the
> subsequent creation of Windows "file" URLs.
> I encountered this on 2.6.5, and have reproduced in 3.0.0 - alpha2.
>
> I believe I have a fair understanding of the root of the issue. Is this of
> interest to anybody?
>
> Thanks
> Simon
>
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
FsUrlStreamHandlerFactory and Windows file URLs
Apologies if this an old topic, however the following test program:
package com.viavi;
import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
public class Main {
public static void main(String[] args) {
System.out.println("URI is " + makeURI());
URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
System.out.println("URI is " + makeURI());
}
private static URI makeURI() {
try {
File file = new File("C:/Users");
final URL url = new URL("file:///" + file.getAbsolutePath());
return url.toURI();
} catch (MalformedURLException x) {
x.printStackTrace();
return null;
} catch (URISyntaxException x) {
x.printStackTrace();
return null;
}
}
}
gives the following output:
URI is file:/C:/Users
URI is null
java.net.URISyntaxException: Illegal character in path at index 8:
file:/C:\Users
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.checkChars(URI.java:3021)
at java.net.URI$Parser.parseHierarchical(URI.java:3105)
at java.net.URI$Parser.parse(URI.java:3053)
at java.net.URI.(URI.java:588)
at java.net.URL.toURI(URL.java:946)
at com.viavi.Main.makeURI(Main.java:23)
at com.viavi.Main.main(Main.java:16)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
That is, it seems that registering the FsUrlStreamHandlerFactory disrupts the
subsequent creation of Windows "file" URLs.
I encountered this on 2.6.5, and have reproduced in 3.0.0 - alpha2.
I believe I have a fair understanding of the root of the issue. Is this of
interest to anybody?
Thanks
Simon
