Amit, The message out of the test framework tries to provide enough information to debug even if the issues isn't reproducible in your environment. Can you think of any reason why it might be giving the different results shown in the message if the order of the batches changed?
If you need to change the order yourself there are two hacky approaches you could do. Try changing the names or saving the files in a different order to make the FS give them back to you in a different order. You also could just combine together the files and adjust the batch cutoff number used in the json reader, with various ordering of the records in different versions of the dataset. As I write this I realize that combining the files will change the behavior of the read. with the first batch giving a single type and later ones giving a union type. As opposed to the multiple files approach which would produce a bunch of different individual types and make the sort operation generate the union type. To test this properly we may just need a test harness to produce batches explicitly and feed them into an operator, rather than relying on the JSON reader. - Jason On Tue, Dec 15, 2015 at 2:31 PM, Amit Hadke <amit.ha...@gmail.com> wrote: > Hey Guys, > > I'm not able to reproduce same issue and test doesn't seem to be doing > anything. > > Can someone run "mvn -Dtest=TestTopNSchemaChanges#testMissingColumn test" > and see if it fails? > > On Mon, Dec 14, 2015 at 11:51 PM, Amit Hadke <amit.ha...@gmail.com> wrote: > > > This seems like a bug in topn code than test. > > We are expecting sorted by kl2 (descending) so that non null values come > > up on top. > > Results seems to be have nulls on top. > > > > ~ Amit. > > > > On Mon, Dec 14, 2015 at 11:27 PM, Jason Altekruse < > > altekruseja...@gmail.com> wrote: > > > >> Seems weird that the results would be different based on reading order, > as > >> the queries themselves contain an order by. Do we return different types > >> out of the sort depending on which schema we get first? Is this > >> intentional? > >> > >> - Jason > >> > >> On Mon, Dec 14, 2015 at 6:06 PM, Steven Phillips <ste...@dremio.com> > >> wrote: > >> > >> > I just did a build a linux box, and didn't see this failure. My guess > is > >> > that it fails depending on which order the files are read. > >> > > >> > On Mon, Dec 14, 2015 at 5:38 PM, Venki Korukanti < > >> > venki.koruka...@gmail.com> > >> > wrote: > >> > > >> > > Is anyone else seeing below failure on latest master? I am running > it > >> on > >> > > Linux. > >> > > > >> > > > >> > > > >> > > >> > testMissingColumn(org.apache.drill.exec.physical.impl.TopN.TestTopNSchemaChanges) > >> > > Time elapsed: 2.537 sec <<< ERROR! > >> > > java.lang.Exception: unexpected null at position 0 column '`vl2`' > >> should > >> > > have been: 299 > >> > > > >> > > Expected Records near verification failure: > >> > > Record Number: 0 { `kl1` : null,`kl2` : 299,`vl2` : 299,`vl1` : > >> > null,`vl` : > >> > > null,`kl` : null, } > >> > > Record Number: 1 { `kl1` : null,`kl2` : 298,`vl2` : 298,`vl1` : > >> > null,`vl` : > >> > > null,`kl` : null, } > >> > > Record Number: 2 { `kl1` : null,`kl2` : 297,`vl2` : 297,`vl1` : > >> > null,`vl` : > >> > > null,`kl` : null, } > >> > > > >> > > > >> > > Actual Records near verification failure: > >> > > Record Number: 0 { `kl1` : null,`vl2` : null,`kl2` : null,`vl1` : > >> > null,`vl` > >> > > : 100.0,`kl` : 100.0, } > >> > > Record Number: 1 { `kl1` : null,`vl2` : null,`kl2` : null,`vl1` : > >> > null,`vl` > >> > > : 101.0,`kl` : 101.0, } > >> > > Record Number: 2 { `kl1` : null,`vl2` : null,`kl2` : null,`vl1` : > >> > null,`vl` > >> > > : 102.0,`kl` : 102.0, } > >> > > > >> > > For query: select kl, vl, kl1, vl1, kl2, vl2 from > >> > > > >> > > > >> > > >> > dfs_test.`/root/drill/exec/java-exec/target/1450142361702-0/topn-schemachanges` > >> > > order by kl2 desc limit 3 > >> > > at > >> > > > >> > > > >> > > >> > org.apache.drill.DrillTestWrapper.compareValuesErrorOnMismatch(DrillTestWrapper.java:512) > >> > > at > >> > > > >> > > > >> > > >> > org.apache.drill.DrillTestWrapper.compareMergedVectors(DrillTestWrapper.java:170) > >> > > at > >> > > > >> > > > >> > > >> > org.apache.drill.DrillTestWrapper.compareMergedOnHeapVectors(DrillTestWrapper.java:397) > >> > > at > >> > > > >> > > > >> > > >> > org.apache.drill.DrillTestWrapper.compareOrderedResults(DrillTestWrapper.java:352) > >> > > at > >> > org.apache.drill.DrillTestWrapper.run(DrillTestWrapper.java:124) > >> > > at org.apache.drill.TestBuilder.go(TestBuilder.java:129) > >> > > at > >> > > > >> > > > >> > > >> > org.apache.drill.exec.physical.impl.TopN.TestTopNSchemaChanges.testMissingColumn(TestTopNSchemaChanges.java:206) > >> > > > >> > > > >> > > Results : > >> > > > >> > > Tests in error: > >> > > TestTopNSchemaChanges.testMissingColumn:206 ยป unexpected null at > >> > > position 0 c... > >> > > > >> > > Tests run: 4, Failures: 0, Errors: 1, Skipped: 0 > >> > > > >> > > >> > > > > >