[
https://issues.apache.org/jira/browse/HDFS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239227#comment-14239227
]
Hari Sekhon commented on HDFS-7488:
-----------------------------------
Collin, I agree with a lot of what you said, feedback was as expected. I'm
aware CIFS is non-trivial having spent a decade with Samba - I actually have
successfully layered Samba over a MapR-FS NFS loopback mount point, but I don't
want to get backed in to a corner on Hadoop platform choices just on this point.
The NFS gateway has kerberos support on the back end for accessing kerberized
HDFS which I've used for loopback mount point and that works fine. I believe
the gateway would need to add NFSv4 support for Kerberos?
I've just created HDFS-7499 to track the idea of Kerberos for client
authentication and whatever extensions that will require.
Andrew, thanks very much for the WebDav solution. Would that be a performant
enough given users want to bulk load data via the Windows mapped drive as well
as run existing C# programs on data contained in HDFS?
I think we should leave this ticket open for native CIFS support somehow in
future, either via a dedicated gateway the equivalent of the NFS gateway or
finding a way for Samba to successfully layer over the existing NFS gateway and
offload the CIFS specific bits to Samba.
> HDFS Windows CIFS Gateway
> -------------------------
>
> Key: HDFS-7488
> URL: https://issues.apache.org/jira/browse/HDFS-7488
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 2.4.0
> Environment: HDP 2.1
> Reporter: Hari Sekhon
>
> Stakeholders are pressuring for native Windows file share access to our
> Hadoop clusters.
> I've used NFS gateway several times and while it's theoretically viable for
> users now UID mapping is implemented in 2.5... insecure NFS makes our fully
> Kerberized clusters security pointless.
> We really need CIFS gateway access to enforce authentication which NFSv3
> doesn't (NFSv4?).
> I've even tried Samba over NFS gateway loopback mount point (don't laugh -
> they want it that badly), and enabled hdfs atime precision to an hour to
> prevent FSNamesystem.setTimes() java exceptions in gw logs, but the NFS
> server still doesn't like the Windows CIFS client actions:
> {code}2014-12-08 16:31:38,053 ERROR nfs3.RpcProgramNfs3
> (RpcProgramNfs3.java:setattr(346)) - Setting file size is not supported when
> setattr, fileId: 25597
> 2014-12-08 16:31:38,065 INFO nfs3.WriteManager
> (WriteManager.java:handleWrite(136)) - No opened stream for fileId:25597
> 2014-12-08 16:31:38,122 INFO nfs3.OpenFileCtx
> (OpenFileCtx.java:receivedNewWriteInternal(624)) - Have to change stable
> write to unstable write:FILE_SYNC
> {code}
> A debug of the Samba server shows it's trying to set metadata timestamps
> which hangs indefinitely, resulting in the creation of a zero byte file when
> trying to copy a file in to HDFS /tmp via the Windows mapped drive.
> {code}
> ...
> smb_set_file_time: setting utimes to modified values.
> file_ntime: actime: Thu Jan 1 01:00:00 1970
> file_ntime: modtime: Mon Dec 8 16:31:38 2014
> file_ntime: ctime: Thu Jan 1 01:00:00 1970
> file_ntime: createtime: Thu Jan 1 01:00:00 1970
> {code}
> This is the traceback from NFS gw log when hdfs precision was set to 0:
> {code}org.apache.hadoop.ipc.RemoteException(java.io.IOException): Access time
> for hdfs is not configured. Please set dfs.namenode.accesstime.precision
> configuration parameter.
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setTimes(FSNamesystem.java:1960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setTimes(NameNodeRpcServer.java:950)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setTimes(ClientNamenodeProtocolServerSideTranslatorPB.java:833)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> ...
> {code}
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
