Github user davidyan74 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/294#discussion_r58595340
--- Diff:
engine/src/main/java/com/datatorrent/stram/client/StramAppLauncher.java ---
@@ -210,14 +239,67 @@ public StramAppLauncher(FileSystem fs, Path path,
Configuration conf) throws Exc
public StramAppLauncher(String name, Configuration conf) throws Exception
{
this.propertiesBuilder = new LogicalPlanConfiguration(conf);
+ this.conf = conf;
init(name);
}
+ public StramAppLauncher(FileSystem fs, Configuration conf) throws
Exception
+ {
+ this.propertiesBuilder = new LogicalPlanConfiguration(conf);
+ this.fs = fs;
+ this.conf = conf;
+ init();
+ }
+
public String getMvnBuildClasspathOutput()
{
return mvnBuildClasspathOutput.toString();
}
+ /**
+ * This is for recovering an app without specifying apa or appjar file
+ * @throws Exception
+ */
+ private void init() throws Exception
+ {
+ String originalAppId = propertiesBuilder.conf.get(ORIGINAL_APP_ID);
+ if (originalAppId == null) {
+ throw new AssertionError("Need original app id if launching without
apa or appjar");
+ }
+ Path appsBasePath = new Path(StramClientUtils.getDTDFSRootDir(fs,
conf), StramClientUtils.SUBDIR_APPS);
+ Path origAppPath = new Path(appsBasePath, originalAppId);
+ StringWriter writer = new StringWriter();
+ try (FSDataInputStream in = fs.open(new Path(origAppPath,
"meta.json"))) {
+ IOUtils.copy(in, writer);
+ }
+ JSONObject metaJson = new JSONObject(writer.toString());
+ String originalLibJars = null;
+ try {
+ JSONObject attributes = metaJson.getJSONObject("attributes");
+ originalLibJars =
attributes.getString(LogicalPlan.LIBRARY_JARS.getSimpleName());
+ recoveryAppName =
attributes.getString(Context.DAGContext.APPLICATION_NAME.getSimpleName());
+ } catch (JSONException ex) {
+ // ignore
+ }
+
+ LinkedHashSet<URL> clUrls = new LinkedHashSet<>();
--- End diff --
That was what I thought too. But StramClient.copyInitialState itself needs
all the operator classes to be loaded before we can even launch. Here's the
exception stack trace if you remove all that code:
{code}
com.esotericsoftware.kryo.KryoException: Unable to find class:
com.datatorrent.lib.testbench.RandomEventGenerator
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752)
at
com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.java:193)
at
com.datatorrent.stram.plan.logical.LogicalPlan$OperatorMeta.readObject(LogicalPlan.java:857)
...
...
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at
com.datatorrent.stram.FSRecoveryHandler.restore(FSRecoveryHandler.java:247)
at
com.datatorrent.stram.StramClient.copyInitialState(StramClient.java:275)
at
com.datatorrent.stram.StramClient.startApplication(StramClient.java:513)
at
com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:616)
at
com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:2067)
at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1454)
{code}
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
