Hi,
I would like to create file transformation pipelines by downloading
input files from remote HDFS and storing outputs file on the same remote
HDFS. Kerberos is used to connect to my hadoop cluster.
On my local machine, I did the following steps:
- install kerberos and obtained a key,
- set the environment variables (`JAVA_HOME`, `HADOOP_CONF_DIR` and
`HADOOP_OPTS`).
- install hadoop client and test read/write on remote HDFS with CLI
command: `hdfs dfs -ls hdfs://clusterhadoop/user` => success
- add environment variables in `hop/config/hop-config.json`
- add hadoop plugins `hadoop-common-3.1.1.jar` and
`hadoop-hdfs-3.1.1.jar` files to the `hdfs/lib` directory created in
`hop/plugins/tech/`
- add hdfs in plugin explorer, with main class
`[org.apache.hadoop.fs.hdfs.HdfsFileSystem]` and path to jar librairies
above.
After all that, I still can't see the contents of my HDFS from the CSV
File Input or TextFIle Input. I get this error:
```
Error browsing to location: 'hdfs://clusterhadoop/user/toto.csv'
FileNotFolderException: Could not list the contents of
"file:///home/ubuntu/hop/hdfs:/clusterhadoop/user" because it is not a
folder.
Root cause: FileNotFolderException: Could not list the contents of
"file:///home/ubuntu/hop/hdfs:/clusterhadoop/user" because it is not a
folder.
org.apache.commons.vfs2.FileNotFolderException: Could not list the
contents of "file:///home/ubuntu/hop/hdfs:/clusterhadoop/user" because
it is not a folder.
at
org.apache.commons.vfs2.provider.AbstractFileObject.getChildren(AbstractFileObject.java:1107)
at
org.apache.hop.ui.core.vfs.HopVfsFileDialog.populateFolder(HopVfsFileDialog.java:941)
at
org.apache.hop.ui.core.vfs.HopVfsFileDialog.refreshBrowser(HopVfsFileDialog.java:898)
at
org.apache.hop.ui.core.vfs.HopVfsFileDialog.refreshAll(HopVfsFileDialog.java:1278)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
org.apache.hop.ui.core.gui.BaseGuiWidgets.lambda$getListener$1(BaseGuiWidgets.java:213)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:91)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5857)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1617)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:5067)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4519)
at
org.apache.hop.ui.core.vfs.HopVfsFileDialog.open(HopVfsFileDialog.java:584)
at
org.apache.hop.ui.core.dialog.BaseDialog.presentFileDialog(BaseDialog.java:278)
at
org.apache.hop.ui.core.dialog.BaseDialog.presentFileDialog(BaseDialog.java:172)
at
org.apache.hop.pipeline.transforms.fileinput.text.TextFileInputDialog.lambda$open$8(TextFileInputDialog.java:484)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:91)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5857)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1617)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:5067)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4519)
at
org.apache.hop.ui.core.dialog.BaseDialog.defaultShellHandling(BaseDialog.java:528)
at
org.apache.hop.pipeline.transforms.fileinput.text.TextFileInputDialog.open(TextFileInputDialog.java:501)
at
org.apache.hop.ui.hopgui.file.pipeline.delegates.HopGuiPipelineTransformDelegate.editTransform(HopGuiPipelineTransformDelegate.java:195)
at
org.apache.hop.ui.hopgui.file.pipeline.HopGuiPipelineGraph.editTransform(HopGuiPipelineGraph.java:3368)
at
org.apache.hop.ui.hopgui.file.pipeline.HopGuiPipelineGraph.editTransform(HopGuiPipelineGraph.java:2210)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
org.apache.hop.core.gui.plugin.action.GuiActionLambdaBuilder.lambda$createLambda$0(GuiActionLambdaBuilder.java:97)
at
org.apache.hop.ui.hopgui.context.GuiContextUtil.lambda$handleActionSelection$0(GuiContextUtil.java:182)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
at
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:5042)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4522)
at org.apache.hop.ui.hopgui.HopGui.open(HopGui.java:478)
at org.apache.hop.ui.hopgui.HopGui.main(HopGui.java:356)
```
Can anyone help me? Thanks in advance for your time.
