[
https://issues.apache.org/jira/browse/STORM-942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630291#comment-14630291
]
ASF GitHub Bot commented on STORM-942:
--------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/637#discussion_r34835109
--- Diff:
external/flux/flux-core/src/main/java/org/apache/storm/flux/parser/FluxParser.java
---
@@ -44,40 +44,47 @@ private FluxParser(){}
// TODO refactor input stream processing (see parseResource() method).
public static TopologyDef parseFile(String inputFile, boolean
dumpYaml, boolean processIncludes,
String propertiesFile, boolean
envSub) throws IOException {
- Yaml yaml = yaml();
+
FileInputStream in = new FileInputStream(inputFile);
- // TODO process properties, etc.
- TopologyDef topology = loadYaml(yaml, in, propertiesFile, envSub);
+ TopologyDef topology = parseInputStream(in, dumpYaml,
processIncludes, propertiesFile, envSub);
in.close();
- if(dumpYaml){
- dumpYaml(topology, yaml);
- }
- if(processIncludes) {
- return processIncludes(yaml, topology, propertiesFile, envSub);
- } else {
- return topology;
- }
+
+ return topology;
}
public static TopologyDef parseResource(String resource, boolean
dumpYaml, boolean processIncludes,
String propertiesFile, boolean
envSub) throws IOException {
- Yaml yaml = yaml();
+
InputStream in = FluxParser.class.getResourceAsStream(resource);
- if(in == null){
- LOG.error("Unable to load classpath resource: " + resource);
- System.exit(1);
- }
- TopologyDef topology = loadYaml(yaml, in, propertiesFile, envSub);
+ TopologyDef topology = parseInputStream(in, dumpYaml,
processIncludes, propertiesFile, envSub);
in.close();
- if(dumpYaml){
- dumpYaml(topology, yaml);
- }
- if(processIncludes) {
- return processIncludes(yaml, topology, propertiesFile, envSub);
- } else {
- return topology;
- }
+
+ return topology;
}
+
+ public static TopologyDef parseInputStream(InputStream inputStream,
boolean dumpYaml, boolean processIncludes,
+ String propertiesFile, boolean envSub) throws IOException {
+
+ Yaml yaml = yaml();
+
+ if (inputStream == null) {
+ LOG.error("Unable to load input stream");
+ System.exit(1);
+ }
+
+ TopologyDef topology = loadYaml(yaml, inputStream,
propertiesFile, envSub);
+ inputStream.close();
--- End diff --
@lyonbrw
IMO it isn't a good way to close InputStream which is passed to a
parameter. Caller can take care of it, and parseFile / parseResource already do
it.
> Add FluxParser method parseInputStream() to eliminate disk usage.
> -----------------------------------------------------------------
>
> Key: STORM-942
> URL: https://issues.apache.org/jira/browse/STORM-942
> Project: Apache Storm
> Issue Type: Improvement
> Affects Versions: 0.10.0
> Reporter: Brendan W Lyon
> Fix For: 0.11.0
>
>
> Included reusable parseInputStream for use by parseFile and parseResource.
> Also allows a path for programatically creating topologies without the need
> to write to / read from disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
