It seems an issue with the azure, there was a discussion over here https://azure.microsoft.com/en-in/documentation/articles/hdinsight-hadoop-spark-install/
Thanks Best Regards On Thu, Jul 9, 2015 at 9:42 PM, Daniel Haviv < daniel.ha...@veracity-group.com> wrote: > Hi, > I'm running Spark 1.4 on Azure. > DataFrame's insertInto fails, but when saveAsTable works. > It seems like some issue with accessing Azure's blob storage but that > doesn't explain why one type of write works and the other doesn't. > > This is the stack trace: > > Caused by: org.apache.hadoop.fs.azure.AzureException: > org.apache.hadoop.fs.azure.KeyProviderException: Unable to load key > provider class. > > at > org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createAzureStorageSession(AzureNativeFileSystemStore.java:938) > > at > org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.initialize(AzureNativeFileSystemStore.java:438) > > at > org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1048) > > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596) > > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) > > at > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) > > at > org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:2618) > > at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:417) > > at > org.apache.hadoop.hive.shims.Hadoop23Shims.getNonCachedFileSystem(Hadoop23Shims.java:574) > > at > org.apache.hadoop.hive.ql.exec.Utilities.createDirsWithPermission(Utilities.java:3424) > > at > org.apache.hadoop.hive.ql.exec.Utilities.createDirsWithPermission(Utilities.java:3396) > > at > org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:214) > > ... 59 more > > Caused by: org.apache.hadoop.fs.azure.KeyProviderException: Unable to load > key provider class. > > at > org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.getAccountKeyFromConfiguration(AzureNativeFileSystemStore.java:829) > > at > org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createAzureStorageSession(AzureNativeFileSystemStore.java:917) > > ... 70 more > > Caused by: java.lang.ClassNotFoundException: Class > org.apache.hadoop.fs.azure.ShellDecryptionKeyProvider not found > > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1980) > > at > org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.getAccountKeyFromConfiguration(AzureNativeFileSystemStore.java:826) > > ... 71 more > > > Thanks, > > Daniel >
