http://apex.apache.org/docs/apex/apex_cli/
Thanks,
Thomas
On Mon, Sep 26, 2016 at 5:04 PM, David Yan <[email protected]>
wrote:
Added back users@ to the thread.
On Mon, Sep 26, 2016 at 5:03 PM, David Yan <[email protected]>
wrote:
Can you try one of the released hadoop versions, like 2.7.3 or 2.7.2
or 2.6.4?
We will check that sha commit on our side as well.
David
On Mon, Sep 26, 2016 at 4:40 PM, <[email protected]> wrote:
On 2016-09-26 16:02, David Yan wrote:
From your first email, you said you're using hadoop 2.7.4, but that
hadoop version has not been released.
Did you build it yourself? If so, can you provide a github tag or
something similar so we can build that ourselves so that we can
reproduce the issue?
Yes. Built from apache git repo, branch branch-2.7, date Aug29,
sha is:
32a86f199cd8e7f32c264af55e3459e4b4751963
I changed 1 file:
diff --git a/hadoop-project/pom.xml b/hadoop-project/pom.xml
index 1765363..c35df02 100644
--- a/hadoop-project/pom.xml
+++ b/hadoop-project/pom.xml
@@ -74,3 +74,3 @@
<!-- define the protobuf JAR version
-->
- <protobuf.version>2.5.0</protobuf.version>
+ <protobuf.version>2.6.1</protobuf.version>
<protoc.path>${env.HADOOP_PROTOC_PATH}</protoc.path>
@@ -95,3 +95,3 @@
<!-- Plugin versions and config -->
- <maven-surefire-plugin.argLine>-Xmx4096m -XX:MaxPermSize=768m
-XX:+HeapDumpOnOutOfMemoryError</maven-surefire-plugin.argLine>
+ <maven-surefire-plugin.argLine>-Xmx4096m
-XX:+HeapDumpOnOutOfMemoryError</maven-surefire-plugin.argLine>
<maven-surefire-plugin.version>2.17</maven-surefire-plugin.version>
Also, what Apex or DT RTS version are you using?
3.4.0, also built from Apache source, using tar file:
apex-3.4.0-source-release.tar.gz
Likewise for malhar, using
apache-apex-malhar-3.4.0-source-release.tar.gz
I had to change one thing to get it to build.
This disables the maven check against deploying from a snapshot.
apache-apex-malhar-3.4.0> diff -p -C1 contrib/pom.xml
contrib/pom.xml.orig
*** contrib/pom.xml Tue Sep 13 12:02:27 2016
--- contrib/pom.xml.orig Fri May 20 00:19:42 2016
***************
*** 199,201 ****
</plugin>
- <!-- SOFTIRON patch
<plugin>
--- 199,200 ----
***************
*** 214,216 ****
</plugin>
! -->
<plugin>
--- 213,215 ----
</plugin>
!
<plugin>
Note: I also tried malhar v3.5 on top of apex3.4, and got the same
results.
I think that covers everything.
thanks again,
-david
Thanks!
David
On Mon, Sep 26, 2016 at 3:15 PM, <[email protected]> wrote:
On 2016-09-26 14:52, David Yan wrote:
Also, one of the first steps in the installation wizard is to
enter
the hadoop location (the same screen as the DFS directory).
Can you please double check the hadoop location points to the
correct
hadoop binary in your system?
That, I've already done. It's definitely correct
(/opt/hadoop/bin/hadoop)
-dbs
David
On Mon, Sep 26, 2016 at 2:45 PM, David Yan <[email protected]>
wrote:
Do you see any exception stacktrace in the log when this error
occurred:
Wrong FS: hdfs://namenode:9000/user/dtadmin/datatorrent, expected:
file:///
David
On Mon, Sep 26, 2016 at 2:39 PM, <[email protected]> wrote:
From David Yan <[email protected]>
Date Sat 00:12
Can you please provide any exception stacktrace in the dtgateway.log
file when that happens?
I reran the dtgateway installation wizard, which failed (same as
before) with error msg:
| DFS directory cannot be written to with error message "Mkdirs
failed to create /user/dtadmin/datatorrent (exists=false,
cwd=file:/opt/datatorrent/releases/3.4.0)"
The corresponding exception in dtgateway.log is:
| 2016-09-26 13:43:54,767 ERROR com.datatorrent.gateway.I: DFS
Directory cannot be written to with exception:
| java.io.IOException: Mkdirs failed to create
/user/dtadmin/datatorrent (exists=false,
cwd=file:/opt/datatorrent/releases/3.4.0)
| at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:455)
| at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440)
| at
org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911)
| at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892)
| at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789)
| at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:778)
| at
com.datatorrent.stram.client.FSAgent.createFile(FSAgent.java:77)
| at com.datatorrent.gateway.I.h(gc:324)
| at com.datatorrent.gateway.I.h(gc:284)
| at
com.datatorrent.gateway.resources.ws.v2.ConfigResource.h(fc:136)
| at
com.datatorrent.gateway.resources.ws.v2.ConfigResource.h(fc:171)
| at
com.datatorrent.gateway.resources.ws.v2.ConfigResource.setConfigProperty(fc:34)
| at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
| at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
| at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
| at java.lang.reflect.Method.invoke(Method.java:498)
| at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
| at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
| at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
| at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
| at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
| at
com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134)
| at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
| at
com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134)
| at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
| at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
| at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
| at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
| at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
| at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
| at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
| at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
| at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
| at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
| at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
| at
javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
| at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
| at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:457)
| at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
| at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
| at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
| at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
| at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
| at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
| at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
| at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
| at org.eclipse.jetty.server.Server.handle(Server.java:368)
| at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
| at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
| at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
| at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
| at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
| at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
| at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
| at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
| at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
| at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
| at java.lang.Thread.run(Thread.java:745)
| 2016-09-26 13:43:54,772 INFO
com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
in processing web service:
com.datatorrent.stram.client.DTConfiguration$ConfigException: DFS
directory cannot be written to with error message "Mkdirs failed to
create /user/dtadmin/datatorrent (exists=false,
cwd=file:/opt/datatorrent/releases/3.4.0)"
Also, when I shutdown the gateway, I get a different exception in
dtgateway.log:
| 2016-09-26 13:40:44,366 INFO com.datatorrent.gateway.DTGateway:
Shutting down
| 2016-09-26 13:40:44,427 ERROR com.datatorrent.gateway.I: DFS
Directory cannot be written to with exception:
| java.io.IOException: Filesystem closed
| at
org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808)
| at
org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:2041)
| at
org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:707)
| at
org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:703)
| at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
| at
org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:714)
| at
com.datatorrent.stram.client.FSAgent.deleteFile(FSAgent.java:94)
| at com.datatorrent.gateway.I.h(gc:30)
| at com.datatorrent.gateway.l.K(pc:158)
| at com.datatorrent.gateway.l.h(pc:65)
| at com.datatorrent.gateway.K.run(yc:156)
| at java.lang.Thread.run(Thread.java:745)
| 2016-09-26 13:40:44,444 INFO com.datatorrent.gateway.DTGateway:
Shutdown complete
Thanks much.
-david
On Tue, Sep 13, 2016 at 4:06 PM, <[email protected]> wrote:
I have the dtgateway running (community edition) with
apex/malhar 3.4 and hadoop 2.7.4 on SuSE SLES 12.1 on an ARM64
server (aarch64).
In the Installation Wizard, I set the DFS location to:
hdfs://namenode:9000/user/dtadmin/datatorrent
and the gateway saves the hadoop configuration and restarts
successfully, but I get an error:
Wrong FS: hdfs://namenode:9000/user/dtadmin/datatorrent,
expected: file:///
the 'service dtgateway status' command says it's running.
The log file has the same error as above, but nothing else
useful.
Is there a manual way to move the demo apps into the gateway?
If I ignore this error and try to upload or import a demo app,
I get the same error.
What's odd is that the gateway does write into the HDFS
directory:
Found 6 items
drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35
/user/dtadmin/datatorrent/appPackages
drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35
/user/dtadmin/datatorrent/apps
drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35
/user/dtadmin/datatorrent/conf
drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35
/user/dtadmin/datatorrent/dashboards
drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35
/user/dtadmin/datatorrent/licenses
drwxrwxrwt - dtadmin hadoop 0 2016-09-13 15:35
/user/dtadmin/datatorrent/systemAlerts
I've tried playing with different ways to configure the DFS
location, but nothing works.
If I use an actual local filesystem directory, I get a
different errro from the installation wizard (Non-DFS file system
is
used: org.apache.hadoop.fs.LocalFileSystem), which makes sense. But
if I ignore this error, the app wizard successsfully writes packages
into the local directory, but I can't launch them (no surprise
there).
Anybody have any ideas what I can do?
thanks much.
-david
