Dear all,
seems that I figured out what the problem was. The documentation https://hop.apache.org/manual/latest/pipeline/pipeline-unit-testing.html does not mention how to reply to the dialog that is attached as a screenshot. After creating the datasets, this dialogs asks the user whether the data set shall be used as input or golden set. Anticipating the role of this data set in the final test case already, I made the "correct decision" here and assigned one created file to the input and the second to the golden test set, which makes them appear in the pipeline as shown in the documenetation (a bit further down, right before "RUN THE UNIT TEST"). But once this assignment has been done, the "Write rows to data set" no longer works as expected (probably because we are in 'test run mode' already). If I remove the assignment to input/golden sets (and the labels disappear), the "Write rows to data set" operates as expected. So the documentation should put some emphasis on this dialog and advise the first time user to select "Cancel". All the best, Frank On 16.11.21 22:21, FH wrote:
Dear all, thank you all for your efforts on delivering HoP as an Apache project, great work! I have difficulties with a first unit test, however. Starting with a simple pipeline (even simpler as in the example https://hop.apache.org/manual/latest/pipeline/pipeline-unit-testing.html), I managed to define the test case input and output files (they appear in the metadata), but do not get them populated with data as described. When _detaching_ the unit test, the pipeline runs again as expected and the input/output counters go up (in the transform metrics). When re-activating the unit test and starting the pipeline to "write rows to data set", everything still looks alright, but the file does not get filled with data (the time stamp gets updated, though) and all record counters remain at 0. This is somewhat unexpected. When adding a test result (golden set), Hop additionally complains on the console that it cannot "find the sort field 'a' in transform" (which is one column from my example input csv file), which may simply happen because both (test input and output files) remain empty (apart from the header line): 2021/11/16 21:44:01 - Hop - Pipeline opened. 2021/11/16 21:44:01 - Hop - Launching pipeline [first]... 2021/11/16 21:44:01 - Hop - Started the pipeline execution. 2021/11/16 21:44:01 - first - Executing this pipeline using the Local Pipeline Engine with run configuration 'local' 2021/11/16 21:44:01 - first - Execution started for pipeline [first] 2021/11/16 21:44:01 - first - ERROR: Unable to validate unit test/golden rows 2021/11/16 21:44:01 - first - ERROR: org.apache.hop.core.exception.HopException: 2021/11/16 21:44:01 - first - Unable to find sort field 'a' in transform results : [] 2021/11/16 21:44:01 - first - 2021/11/16 21:44:01 - first - at org.apache.hop.testing.util.DataSetConst.validateTransformResultAgainstUnitTest(DataSetConst.java:220) 2021/11/16 21:44:01 - first - at org.apache.hop.testing.xp.ValidatePipelineUnitTestExtensionPoint.callExtensionPoint(ValidatePipelineUnitTestExtensionPoint.java:87) 2021/11/16 21:44:01 - first - at org.apache.hop.testing.xp.ValidatePipelineUnitTestExtensionPoint.callExtensionPoint(ValidatePipelineUnitTestExtensionPoint.java:41) 2021/11/16 21:44:01 - first - at org.apache.hop.core.extension.ExtensionPointMap.callExtensionPoint(ExtensionPointMap.java:149) 2021/11/16 21:44:01 - first - at org.apache.hop.core.extension.ExtensionPointHandler.callExtensionPoint(ExtensionPointHandler.java:41) 2021/11/16 21:44:01 - first - at org.apache.hop.pipeline.Pipeline.lambda$startThreads$1(Pipeline.java:1166) 2021/11/16 21:44:01 - first - at org.apache.hop.pipeline.Pipeline.firePipelineExecutionFinishedListeners(Pipeline.java:1268) 2021/11/16 21:44:01 - first - at org.apache.hop.pipeline.Pipeline.lambda$startThreads$0(Pipeline.java:1083) 2021/11/16 21:44:01 - first - at org.apache.hop.pipeline.transform.BaseTransform.fireTransformFinishedListeners(BaseTransform.java:2729) 2021/11/16 21:44:01 - first - at org.apache.hop.pipeline.transform.BaseTransform.markStop(BaseTransform.java:2718) 2021/11/16 21:44:01 - first - at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:146) 2021/11/16 21:44:01 - first - at java.lang.Thread.run(Thread.java:748) 2021/11/16 21:44:01 - first - Pipeline duration : 0.097 seconds [ 0.097" ] 2021/11/16 21:44:01 - first - Execution finished on a local pipeline engine with run configuration 'local' What I am doing wrong? Any hint is appreciated. All the best, Frank PS: Is there really no shortcut (like pressing Z when hovering over a pipeline) to open the configuration dialog of a step? (I am constantly trying to double click them as I get used to with kettle/pdi ;-) ).
