Drill Test Framework : Planning Tests

2015-10-26 Thread rahul challapalli
Drillers, We have a handful of tests in the drill test framework (Refer to [1] for some examples) which compare the text version of the plan to an expected result. The goal is to compare the structure of the text plan and also selected attributes of some operators ( Eg : numFiles, list of scanned

Re: Drill Test Framework : Planning Tests

2015-10-27 Thread Jacques Nadeau
We should use the structured response (and do structural comparisons) for plan verification. The text plan should be cleaned up so that it is more useful and user friendly. We definitely shouldn't depend on what is effectively a random toString method for functional verification (I know we do now).

Re: Drill Test Framework : Planning Tests

2015-10-27 Thread rahul challapalli
With the json tree matching, we still want to control what determines an equality between 2 json tree's at the level of each operator. We should be able to ignore the attributes/child-nodes which represent any costing information or moving parts. With this requirement don't we still need to come up

Re: Drill Test Framework : Planning Tests

2015-10-27 Thread Jacques Nadeau
In terms of matching, it seems like people should be able to express the portion of nodes that they match. If they want to match the file list length but not the order, they should be able to express that. The important part of a framework is to not require people to overly constrain the output as

Re: Drill Test Framework : Planning Tests

2015-10-28 Thread rahul challapalli
Jacques, If my understanding is correct you are alluding towards declarative style expected values. Can you give an example? Below are my thoughts around implementing this 1. Construct Physical Operator tree's based on the expected value json and actual drill output json strings 2. For each P