abdullah alamoudi has posted comments on this change. Change subject: Introduced Local Filesystem Feed Adapter ......................................................................
Patch Set 3: (14 comments) https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-app/src/test/resources/runtimets/testsuite.xml File asterix-app/src/test/resources/runtimets/testsuite.xml: Line 25: <test-group name="external"> > Nit: spaces instead of tabs would be good :) Ah. the xml tabs comment that I get with almost every change. It is a good thing that I figured out how to configure the xml formatter :-) https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/LocalFileSystemFeedAdapterFactory.java File asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/LocalFileSystemFeedAdapterFactory.java: Line 70: this.expression = (String) configuration.get(AsterixTupleParserFactory.KEY_EXPRESSION); > Unnecessary cast. Done Line 71: String[] splits = ((String) configuration.get(AsterixTupleParserFactory.KEY_PATH)).split(","); > Unnecessary cast. Done Line 77: private void configureFileSplits(String[] splits) throws AsterixException { > Suggestion: This method could be factored out with NCFileSystemAdapterFacto Somewhat done. I didn't want to move this method to their super class since it is not applicable for other streambased factories. I however have made this as a static method in NCFileSystemAdapterFactory and called it. https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/NCFileSystemAdapterFactory.java File asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/NCFileSystemAdapterFactory.java: Line 75: this.expression = (String) configuration.get(AsterixTupleParserFactory.KEY_EXPRESSION); > Unnecessary cast. Done Line 76: String[] splits = ((String) configuration.get(AsterixTupleParserFactory.KEY_PATH)).split(","); > Unnecessary cast. Done https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/FileSystemWatcher.java File asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/FileSystemWatcher.java: Line 39: import org.apache.log4j.Level; > Use Java Logger. I prefer log4j. for some reason, java logger just doesn't work on my machine :-\ Should we have a discussion about what logger to use in the dev list? Line 159: if (expression != null || Pattern.matches(expression, child.toString())) { > This should be (expression == null ...). Good catch. Done :-) Line 165: LOGGER.warn(e.getMessage() + ":" + e.getStackTrace()); > just use e.printStackTrace. I don't think e.getStackTrace().toString() woul Done Line 246: return 1 + in.skip(n - 1); > A comment here would really help :) Skip actually is not logical here in this adapter (at least for our use). I fixed this one. https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LocalFileSystemFeedAdapter.java File asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/LocalFileSystemFeedAdapter.java: Line 120: } > Is it really the case, that no matter what exception we may eat here, we st Mmm not really. I have fixed this and now we will continue only if the exception is an AsterixException -> "missing required record" or HyracksDataException ->"malformed value that the parser couldn't parse". https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/NCFileSystemAdapter.java File asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/NCFileSystemAdapter.java: Line 92: */ > Suggestion: you are using recursion here and walkFileTree in FileSystemWatc Done https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java File asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java: Line 319: String[] builtInAdapterClassNames = new String[] { > +1 ;) But there's lots of other places in the code that are equally guilty Unfortunately, we are aware of this imitation but for this to be fixed, we will need a lot of restructuring and moving things around. The main problem is that asterix-metadata doesn't depend on asterix-external-data nor asterix-tools (This would create a dependency cycle). https://asterix-gerrit.ics.uci.edu/#/c/376/3/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/file/AsterixTupleParserFactory.java File asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/file/AsterixTupleParserFactory.java: Line 68: public static final Object KEY_EXPRESSION = "expression"; It is not wrong since the actual object is a string :p Fixed. -- To view, visit https://asterix-gerrit.ics.uci.edu/376 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I707756e3b4c9ffca4b55ec9817a08e5c16333010 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudi <[email protected]> Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Steven Jacobs <[email protected]> Gerrit-Reviewer: abdullah alamoudi <[email protected]> Gerrit-HasComments: Yes
