> On Sept. 23, 2015, 3:26 a.m., Sowmya Ramesh wrote: > > common/src/main/java/org/apache/falcon/entity/FeedHelper.java, line 786 > > <https://reviews.apache.org/r/38465/diff/2/?file=1077184#file1077184line786> > > > > Can getSource return null or does xsd validation fail if not passed? > > Same for getTableName and getExtract and others
getSource(), getExtract() and getTableName() are all mandatory and validated by the XSD. Only optional tag is import (can be null). isImportEnabled() handles it. > On Sept. 23, 2015, 3:26 a.m., Sowmya Ramesh wrote: > > common/src/main/java/org/apache/falcon/entity/FeedHelper.java, line 830 > > <https://reviews.apache.org/r/38465/diff/2/?file=1077184#file1077184line830> > > > > same as above mergepolicy tag is mandatory so will not get NPE. > On Sept. 23, 2015, 3:26 a.m., Sowmya Ramesh wrote: > > common/src/main/java/org/apache/falcon/entity/FeedHelper.java, line 761 > > <https://reviews.apache.org/r/38465/diff/2/?file=1077184#file1077184line761> > > > > Can't find DatasourceHelper file. Uploaded updated patch and the files are present. > On Sept. 23, 2015, 3:26 a.m., Sowmya Ramesh wrote: > > common/src/main/java/org/apache/falcon/entity/FeedHelper.java, line 771 > > <https://reviews.apache.org/r/38465/diff/2/?file=1077184#file1077184line771> > > > > Since these are related to data source it shoudl be moved to > > DatasourceHelper. This actually checks if the feed XML definition has an IMPORT tag. > On Sept. 23, 2015, 3:26 a.m., Sowmya Ramesh wrote: > > common/src/main/java/org/apache/falcon/entity/FeedHelper.java, line 816 > > <https://reviews.apache.org/r/38465/diff/2/?file=1077184#file1077184line816> > > > > Can getsource, getextract return null? getSource(), getExtract() and getTableName() are all mandatory and validated by the XSD. Only optional tag is import (can be null). isImportEnabled() handles it. > On Sept. 23, 2015, 3:26 a.m., Sowmya Ramesh wrote: > > common/src/main/java/org/apache/falcon/entity/FeedHelper.java, line 846 > > <https://reviews.apache.org/r/38465/diff/2/?file=1077184#file1077184line846> > > > > Does the time zone matter here? Date() itself does not have a timezone associated with it. Timezone is required when formatting it. All of the other code format's them to UTC timezone when creating oozie artifacts. > On Sept. 23, 2015, 3:26 a.m., Sowmya Ramesh wrote: > > common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java, > > line 49 > > <https://reviews.apache.org/r/38465/diff/2/?file=1077185#file1077185line49> > > > > can't find DatasourceEntityParser All the files are present in the new uploaded patch. - Venkatesan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38465/#review100036 ----------------------------------------------------------- On Sept. 23, 2015, 11:58 p.m., Venkatesan Ramachandran wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38465/ > ----------------------------------------------------------- > > (Updated Sept. 23, 2015, 11:58 p.m.) > > > Review request for Falcon, Ajay Yadava, Balu Vellanki, Peeyush Bishnoi, > Sowmya Ramesh, and Venkat Ranganathan. > > > Repository: falcon-git > > > Description > ------- > > FALCON-1459 : Ability to import from database > > > Diffs > ----- > > client/src/main/java/org/apache/falcon/LifeCycle.java 58a2a6c > client/src/main/java/org/apache/falcon/Tag.java beeb812 > client/src/main/java/org/apache/falcon/entity/v0/EntityType.java 0657124 > client/src/main/java/org/apache/falcon/metadata/RelationshipType.java > f034772 > client/src/main/resources/datasource-0.1.xsd PRE-CREATION > client/src/main/resources/feed-0.1.xsd 2af28d2 > client/src/main/resources/jaxb-binding.xjb 6f1d6c7 > client/src/main/resources/mysql_database.xml PRE-CREATION > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java > PRE-CREATION > common/src/main/java/org/apache/falcon/entity/EntityUtil.java ad41674 > common/src/main/java/org/apache/falcon/entity/FeedHelper.java 572923b > > common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java > PRE-CREATION > > common/src/main/java/org/apache/falcon/entity/parser/EntityParserFactory.java > 5a33201 > common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java > 4f5599e > common/src/main/java/org/apache/falcon/entity/store/ConfigurationStore.java > e27187b > common/src/main/java/org/apache/falcon/entity/v0/EntityGraph.java bd4c6cf > > common/src/main/java/org/apache/falcon/entity/v0/EntityIntegrityChecker.java > bd32852 > > common/src/main/java/org/apache/falcon/metadata/EntityRelationshipGraphBuilder.java > 8c3876c > common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java > PRE-CREATION > > common/src/main/java/org/apache/falcon/workflow/WorkflowExecutionContext.java > 4454239 > common/src/test/java/org/apache/falcon/entity/AbstractTestBase.java e9946c4 > common/src/test/java/org/apache/falcon/entity/EntityTypeTest.java 640e87d > common/src/test/java/org/apache/falcon/entity/FeedHelperTest.java c70cfcc > > common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java > PRE-CREATION > > common/src/test/java/org/apache/falcon/entity/parser/FeedEntityParserTest.java > b6fdb13 > common/src/test/java/org/apache/falcon/entity/v0/EntityGraphTest.java > 3863b11 > common/src/test/resources/config/datasource/datasource-0.1.xml PRE-CREATION > common/src/test/resources/config/datasource/datasource-invalid-0.1.xml > PRE-CREATION > common/src/test/resources/config/feed/feed-import-0.1.xml PRE-CREATION > common/src/test/resources/config/feed/feed-import-invalid-0.1.xml > PRE-CREATION > docs/src/site/twiki/EntitySpecification.twiki d4f4140 > > falcon-regression/merlin-core/src/main/java/org/apache/falcon/regression/core/util/HiveAssert.java > 2a934b5 > > oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java > PRE-CREATION > > oozie/src/main/java/org/apache/falcon/oozie/FeedImportCoordinatorBuilder.java > PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java > PRE-CREATION > oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java > a04ae95 > > oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java > 3213a70 > oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java > b819dee > oozie/src/main/resources/action/feed/import-sqoop-database-action.xml > PRE-CREATION > pom.xml 646de69 > webapp/pom.xml 828f7f5 > webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java > PRE-CREATION > webapp/src/test/java/org/apache/falcon/resource/TestContext.java d067dee > webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java > PRE-CREATION > webapp/src/test/resources/datasource-template.xml PRE-CREATION > webapp/src/test/resources/feed-template3.xml PRE-CREATION > > Diff: https://reviews.apache.org/r/38465/diff/ > > > Testing > ------- > > * Unit tests > * Integration tests > * Manual tests > * Setup MySQL, create table and populate > * Create datasource and feed entity with import policy in Falcon > * Made sure the data lands up in the HDFS. > > > Thanks, > > Venkatesan Ramachandran > >
