John, What version of the JDK are you running, and what is your extension jars environment variable set to. Do you have the java cryptology jar included (Oracle JDK usually has this, I don't know if Open JDK does).
"Algorithm HmacSHA1 not available" is usually thrown when Java cannot find the java crypto jar used to calculate the given hash. -Michael On Thu, Mar 12, 2015 at 9:06 AM, John Reynolds <jreyno...@vpicu.net> wrote: > Hi Lewis, > using the latest docker buggtb/oodt image, which i assume is .8 > here’s the command i’m running to test the upload > > filemgr-client --url http://localhost:9000 --operation --ingestProduct > --productName test --productStructure Flat --productTypeName GenericFile > --metadataFile file:///root/test.txt.met --refs file:///root/test.txt > > i verified that i can upload to the path using the s3 tools on the box / > with same credentials i put in the properties file > > here’s the full exception returned: > > rg.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException: > org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException: > Failed to upload product reference /root/test.txt to S3 at > usr/src/oodt/data/archive/test/test.txt > at > org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:768) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.xmlrpc.Invoker.execute(Invoker.java:130) > at > org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84) > at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146) > at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139) > at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125) > at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761) > at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642) > at java.lang.Thread.run(Thread.java:745) > Caused by: > org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException: > Failed to upload product reference /root/test.txt to S3 at > usr/src/oodt/data/archive/test/test.txt > at > org.apache.oodt.cas.filemgr.datatransfer.S3DataTransferer.transferProduct(S3DataTransferer.java:78) > at > org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:752) > ... 12 more > Caused by: com.amazonaws.AmazonClientException: Unable to calculate a > request signature: Unable to calculate a request signature: Algorithm > HmacSHA1 not available > at > com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:71) > at > com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:57) > at > com.amazonaws.services.s3.internal.S3Signer.sign(S3Signer.java:128) > at > com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:330) > at > com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232) > at > com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528) > at > com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1393) > at > org.apache.oodt.cas.filemgr.datatransfer.S3DataTransferer.transferProduct(S3DataTransferer.java:76) > ... 13 more > Caused by: com.amazonaws.AmazonClientException: Unable to calculate a > request signature: Algorithm HmacSHA1 not available > at > com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:90) > at > com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:68) > ... 20 more > Caused by: java.security.NoSuchAlgorithmException: Algorithm HmacSHA1 not > available > at javax.crypto.Mac.getInstance(Mac.java:176) > at > com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:86) > ... 21 more > org.apache.xmlrpc.XmlRpcException: java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product [org.apache.oodt.cas.filemgr.structs.Product@6454bbe1] > : org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException: > Failed to upload product reference /root/test.txt to S3 at > usr/src/oodt/data/archive/test/test.txt > at > org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104) > at > org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71) > at > org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) > at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) > at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) > at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) > at > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1198) > at > org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:112) > at > org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331) > at > org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187) > at > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1350) > java.lang.Exception: Failed to ingest product > [org.apache.oodt.cas.filemgr.structs.Product@7f25ab41] : > java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product [org.apache.oodt.cas.filemgr.structs.Product@6454bbe1] > : org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException: > Failed to upload product reference /root/test.txt to S3 at > usr/src/oodt/data/archive/test/test.txt > at > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1303) > at > org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:112) > at > org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331) > at > org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187) > at > org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1350) > ERROR: Failed to ingest product 'test' : Failed to ingest product > [org.apache.oodt.cas.filemgr.structs.Product@7f25ab41] : > java.lang.Exception: > org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error > ingesting product [org.apache.oodt.cas.filemgr.structs.Product@6454bbe1] > : org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException: > Failed to upload product reference /root/test.txt to S3 at > usr/src/oodt/data/archive/test/test.txt > > > > On Mar 11, 2015, at 6:21 PM, Lewis John Mcgibbney < > lewis.mcgibb...@gmail.com> wrote: > > > > Hi John, > > What beau on of OODT are you using? > > The file types should be encapsulated by the factory implementation I > would > > guess. It should be. Case of mike this the default filemgr store then off > > you go. > > Can you provide a paste of how your arriving at your exception please? > > Lewia > > > > On Wednesday, March 11, 2015, John Reynolds <jreyno...@vpicu.net> wrote: > > > >> Should S3DataTransferer work with the generic file type from the get go > or > >> is there anything i need to change > >> i was using the radix docker but getting 'Caused by: > >> java.security.NoSuchAlgorithmException: Algorithm HmacSHA1 not > available’ > >> i tried downloading the jce but no dice > >> i might try just building on my local machine and trying again > >> > >> Thanks > >> > >>> On Mar 10, 2015, at 6:19 PM, Lewis John Mcgibbney < > >> lewis.mcgibb...@gmail.com <javascript:;>> wrote: > >>> > >>> Hey John, > >>> You're right, you've nearly answered everything here ;) > >>> OK, so your properties can be located here > >>> > >> > https://github.com/apache/oodt/blob/trunk/filemgr/src/main/resources/filemgr.properties#L114-L124 > >>> You'll see the usual key, value pairs in there which you can ad before > >>> build the project and packaging filemgr.properties along with your > >> compiled > >>> code. > >>> Regarding the following, > >>> > >>> On Tue, Mar 10, 2015 at 4:31 PM, John Reynolds <jreyno...@vpicu.net > >> <javascript:;>> wrote: > >>> > >>>> > >>>> in the near future, i will want to pass in a security token as well to > >> the > >>>> S3 client. to do so i would extend or modify the s3datatransfer class > >>>> (which expires after an hour), what’s the best practice to read in > this > >>>> transient data from the command line (or wherever else i would > initiate > >> a > >>>> job from) since this wouldnt go in a properties file > >>> > >>> > >>> Do you mean that your security token expires hourly? As oppose to your > s3 > >>> client? > >>> Lets try and defined exactly what you are after here. > >>> Thanks > >>> Lewis > >> > >> > > > > -- > > *Lewis* > >
