Thank you Christopher,
I was able to determine that the ssl settings in core-site.xml are being
picked up and used. In fact when accumulo init is run, accumulo is able
to create the /accumulo directory in HDFS. What is weird is that when
the FileSKVWriter that is used when createMetadataFile is called it
throws an exception when close is called.
I get:
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
/accumulo/tables/!0/table_info/0_1.rf could only be written to the 0 of the
1 minReplication nodes. There are 1 datanode(s) running and 1 node(s) are
excluded in this operation.
at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1720)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3389)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:683)
at
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.addBlock(AuthorizationProviderProxyClientProtocol.java:214)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:495)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2211)
I don't get this error when I disable ssl on hadoop.
Any insight would be greatly appreciated.
Thanks,
On Tue, Dec 14, 2021 at 2:23 PM Vincent Russell <vincent.russ...@gmail.com>
wrote:
> Thanks Chris.
>
> Yes I do get an error (I can't remember now because it's on a separate
> computer) during the init and I get a MagicNumber exception on the datanode
> during this process which says something like maybe encryption isn't turned
> on.
>
> but let me make sure that the core-default.xml and core-site.xml are on
> the classpath. They may not be.
>
> Thanks again.
>
> On Tue, Dec 14, 2021 at 2:13 PM Christopher <ctubb...@apache.org> wrote:
>
>> I have not personally tested HDFS configured for SSL/TLS, but `new
>> Configuration()` will load the core-default.xml and core-site.xml
>> files it finds on the class path. So, it looks like it should work.
>> Have you tried it? Did you get an error?
>>
>>
>> On Tue, Dec 14, 2021 at 1:54 PM Vincent Russell
>> <vincent.russ...@gmail.com> wrote:
>> >
>> > Thank you Mike,
>> >
>> > but it appears that accumulo uses those settings to connect accumulo,
>> but
>> > not to connect to hdfs.
>> >
>> > For instance the VolumeManagementImpl just does this:
>> >
>> > VolumeConfiguration.create(new Path(volumeUriOrDir), hadoopConf));
>> >
>> > where the hadoopConf is just instantiated in the Initialize class:
>> >
>> > Configuration hadoopConfig = new Configuration();
>> > VolumeManager fs = VolumeManagerImpl.get(siteConfig, hadoopConfig);
>> >
>> > Thanks,
>> > Vincent
>> >
>> > On Tue, Dec 14, 2021 at 12:18 PM Mike Miller <mmil...@apache.org>
>> wrote:
>> >
>> > > Checkout the accumulo client properties that start with the "ssl"
>> prefix.
>> > > https://accumulo.apache.org/docs/2.x/configuration/client-properties
>> > > This blog post from a few years ago may help:
>> > >
>> > >
>> https://accumulo.apache.org/blog/2014/09/02/generating-keystores-for-configuring-accumulo-with-ssl.html
>> > >
>> > > On Tue, Dec 14, 2021 at 9:58 AM Vincent Russell <
>> vincent.russ...@gmail.com
>> > > >
>> > > wrote:
>> > >
>> > > > Hello,
>> > > >
>> > > > I am trying to init a test accumulo instance with an hdfs running
>> with
>> > > > SSL. Is this possible? I am looking at the code and it doesn't
>> look
>> > > > like this is possible.
>> > > >
>> > > > The Initialize class just instantiates a Hadoop config and passes
>> that
>> > > into
>> > > > the VolumeManager without sending over any hadoop configs from the
>> > > core.xml
>> > > > file.
>> > > >
>> > > > Am I missing something?
>> > > >
>> > > > Thanks in advance for your help,
>> > > > Vincent
>> > > >
>> > >
>>
>
