[
https://issues.apache.org/jira/browse/HADOOP-14598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16065397#comment-16065397
]
Steve Loughran edited comment on HADOOP-14598 at 6/28/17 11:20 AM:
-------------------------------------------------------------------
not related to the latest SDK update; happens if I switch to the previous one.
Looking @ the code, it comes down to this: the URI to use
{code}
builder = new UriQueryBuilder();
final URL resourceUrl = builder.addToURI(uri).toURL();
retConnection = (HttpURLConnection) resourceUrl.openConnection(); // *HERE*
{code}
The implication is that (a) hadoop has patched in a URL factory and (b) its
returning the wrong thing, or (c) I'm handing in the wrong URL to the FS.
Which, if I instrument my code better, doesn't appear to be the case
{code}
FileGenerator *** FAILED ***
java.io.IOException: Failed to delete
wasb://contr...@contender.blob.core.windows.net/cloud-integration/DELAY_LISTING_ME/AzureFileGeneratorSuite/filegenerator
on org.apache.hadoop.fs.azure.NativeAzureFileSystem@6fe77a97
org.apache.hadoop.fs.azure.AzureException:
com.microsoft.azure.storage.StorageException:
org.apache.hadoop.fs.FsUrlConnection cannot be cast to
java.net.HttpURLConnection
at
com.hortonworks.spark.cloud.ObjectStoreOperations$class.rm(ObjectStoreOperations.scala:415)
at
com.hortonworks.spark.cloud.operations.CloudFileGenerator.rm(CloudFileGenerator.scala:33)
at
com.hortonworks.spark.cloud.operations.CloudFileGenerator.action(CloudFileGenerator.scala:81)
at
com.hortonworks.spark.cloud.ObjectStoreExample$class.action(ObjectStoreExample.scala:70)
at
com.hortonworks.spark.cloud.operations.CloudFileGenerator.action(CloudFileGenerator.scala:33)
at
com.hortonworks.spark.cloud.common.FileGeneratorTests.generate(FileGeneratorTests.scala:80)
at
com.hortonworks.spark.cloud.common.FileGeneratorTests$$anonfun$2.apply$mcV$sp(FileGeneratorTests.scala:40)
at
com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply$mcV$sp(CloudSuiteTrait.scala:69)
at
com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply(CloudSuiteTrait.scala:67)
at
com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply(CloudSuiteTrait.scala:67)
...
Cause: org.apache.hadoop.fs.azure.AzureException: com.microsoft.azure.st
{code}
was (Author: ste...@apache.org):
not related to the latest SDK update; happens if I switch to the previous one.
Looking @ the code, it comes down to this: the URI to use
{code}
builder = new UriQueryBuilder();
final URL resourceUrl = builder.addToURI(uri).toURL();
retConnection = (HttpURLConnection) resourceUrl.openConnection(); // *HERE*
{code}
The implication is that (a) hadoop has patched in a URL factory and (b) its
returning the wrong thing, or (c) I'm handing in the wrong URL to the FS.
Which, if I instrument my code better, doesn't appear to be the case
{code}
FileGenerator *** FAILED ***
java.io.IOException: Failed to delete
wasb://contr...@contender.blob.core.windows.net/cloud-integration/DELAY_LISTING_ME/AzureFileGeneratorSuite/filegenerator
on org.apache.hadoop.fs.azure.NativeAzureFileSystem@6fe77a97
org.apache.hadoop.fs.azure.AzureException:
com.microsoft.azure.storage.StorageException:
org.apache.hadoop.fs.FsUrlConnection cannot be cast to
java.net.HttpURLConnection
at
com.hortonworks.spark.cloud.ObjectStoreOperations$class.rm(ObjectStoreOperations.scala:415)
at
com.hortonworks.spark.cloud.operations.CloudFileGenerator.rm(CloudFileGenerator.scala:33)
at
com.hortonworks.spark.cloud.operations.CloudFileGenerator.action(CloudFileGenerator.scala:81)
at
com.hortonworks.spark.cloud.ObjectStoreExample$class.action(ObjectStoreExample.scala:70)
at
com.hortonworks.spark.cloud.operations.CloudFileGenerator.action(CloudFileGenerator.scala:33)
at
com.hortonworks.spark.cloud.common.FileGeneratorTests.generate(FileGeneratorTests.scala:80)
at
com.hortonworks.spark.cloud.common.FileGeneratorTests$$anonfun$2.apply$mcV$sp(FileGeneratorTests.scala:40)
at
com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply$mcV$sp(CloudSuiteTrait.scala:69)
at
com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply(CloudSuiteTrait.scala:67)
at
com.hortonworks.spark.cloud.CloudSuiteTrait$$anonfun$ctest$1.apply(CloudSuiteTrait.scala:67)
...
Cause: org.apache.hadoop.fs.azure.AzureException: com.microsoft.azure.st
{code]
> azure wasb failing: FsUrlConnection cannot be cast to HttpURLConnection
> -----------------------------------------------------------------------
>
> Key: HADOOP-14598
> URL: https://issues.apache.org/jira/browse/HADOOP-14598
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, test
> Affects Versions: 3.0.0-beta1
> Reporter: Steve Loughran
>
> my downstream-of-spark cloud integration tests (where I haven't been running
> the azure ones for a while) now have a few of the tests failing
> {code}
> org.apache.hadoop.fs.azure.AzureException:
> com.microsoft.azure.storage.StorageException:
> org.apache.hadoop.fs.FsUrlConnection cannot be cast to
> java.net.HttpURLConnection
> {code}
> No obvious cause, and it's only apparently happening in some of the
> (scalatest) tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
