Ok... so I have yet another question relating to the EVF. I'm working on a project to improve (hopefully) the PCAP plugin with the ultimate goal being to include parsed PCAP packet data. In any event, I've run into a snag. In one unit test, I'm getting the error below when I call rowWriter.save(). I suspect what is happening is that the TupleWriter is not full when it starts writing the row, but by the time it is finished writing the fields, the batch is full. Does that even make sense?
Here's a link to the offending code <https://github.com/cgivre/drill/blob/caa69e7f27f68aeedaa28902596a2250ab32cd84/exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapBatchReader.java#L232>: Any suggestions? Thanks! --C [Error Id: bf6446fe-95d5-4ba7-bf7f-32e8a5dd4d1e on 192.168.1.21:31010] (java.lang.IllegalStateException) Unexpected state: FULL_BATCH org.apache.drill.exec.physical.resultSet.impl.ResultSetLoaderImpl.saveRow():530 org.apache.drill.exec.physical.resultSet.impl.RowSetLoaderImpl.save():73 org.apache.drill.exec.store.pcap.PcapBatchReader.addDataToTable():232 org.apache.drill.exec.store.pcap.PcapBatchReader.parsePcapFilesAndPutItToTable():174 org.apache.drill.exec.store.pcap.PcapBatchReader.next():102 org.apache.drill.exec.physical.impl.scan.framework.ShimBatchReader.next():132 org.apache.drill.exec.physical.impl.scan.ReaderState.readBatch():412 org.apache.drill.exec.physical.impl.scan.ReaderState.next():369 org.apache.drill.exec.physical.impl.scan.ScanOperatorExec.nextAction():261 org.apache.drill.exec.physical.impl.scan.ScanOperatorExec.next():232 org.apache.drill.exec.physical.impl.protocol.OperatorDriver.doNext():196 org.apache.drill.exec.physical.impl.protocol.OperatorDriver.start():174 org.apache.drill.exec.physical.impl.protocol.OperatorDriver.next():124 org.apache.drill.exec.physical.impl.protocol.OperatorRecordBatch.next():148 org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():237 org.apache.drill.exec.record.AbstractRecordBatch.next():126 org.apache.drill.exec.record.AbstractRecordBatch.next():116 org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():141 org.apache.drill.exec.record.AbstractRecordBatch.next():186 org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():237 org.apache.drill.exec.physical.impl.BaseRootExec.next():104 org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():83 org.apache.drill.exec.physical.impl.BaseRootExec.next():94 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():296 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():283 java.security.AccessController.doPrivileged():-2 javax.security.auth.Subject.doAs():422 org.apache.hadoop.security.UserGroupInformation.doAs():1746 org.apache.drill.exec.work.fragment.FragmentExecutor.run():283 org.apache.drill.common.SelfCleaningRunnable.run():38 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 at org.apache.drill.exec.rpc.RpcException.mapException(RpcException.java:60) ~[classes/:na] at org.apache.drill.exec.client.DrillClient$ListHoldingResultsListener.getResults(DrillClient.java:881) ~[classes/:na] at org.apache.drill.exec.client.DrillClient.runQuery(DrillClient.java:583) ~[classes/:na] at org.apache.drill.test.BaseTestQuery.testRunAndReturn(BaseTestQuery.java:340) ~[test-classes/:na] at org.apache.drill.test.BaseTestQuery.testSqlWithResults(BaseTestQuery.java:321) ~[test-classes/:na] at org.apache.drill.exec.store.pcap.TestPcapRecordReader.runSQLWithResults(TestPcapRecordReader.java:90) ~[test-classes/:na] at org.apache.drill.exec.store.pcap.TestPcapRecordReader.runSQLVerifyCount(TestPcapRecordReader.java:85) ~[test-classes/:na] at org.apache.drill.exec.store.pcap.TestPcapRecordReader.testCorruptPCAPQuery(TestPcapRecordReader.java:47) ~[test-classes/:na] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_211] Caused by: org.apache.drill.common.exceptions.UserRemoteException: EXECUTION_ERROR ERROR: Unexpected state: FULL_BATCH Read failed for reader PcapBatchReader Fragment 0:0 [Error Id: bf6446fe-95d5-4ba7-bf7f-32e8a5dd4d1e on 192.168.1.21:31010] (java.lang.IllegalStateException) Unexpected state: FULL_BATCH org.apache.drill.exec.physical.resultSet.impl.ResultSetLoaderImpl.saveRow():530 org.apache.drill.exec.physical.resultSet.impl.RowSetLoaderImpl.save():73 org.apache.drill.exec.store.pcap.PcapBatchReader.addDataToTable():232 org.apache.drill.exec.store.pcap.PcapBatchReader.parsePcapFilesAndPutItToTable():174 org.apache.drill.exec.store.pcap.PcapBatchReader.next():102 org.apache.drill.exec.physical.impl.scan.framework.ShimBatchReader.next():132 org.apache.drill.exec.physical.impl.scan.ReaderState.readBatch():412 org.apache.drill.exec.physical.impl.scan.ReaderState.next():369 org.apache.drill.exec.physical.impl.scan.ScanOperatorExec.nextAction():261 org.apache.drill.exec.physical.impl.scan.ScanOperatorExec.next():232 org.apache.drill.exec.physical.impl.protocol.OperatorDriver.doNext():196 org.apache.drill.exec.physical.impl.protocol.OperatorDriver.start():174 org.apache.drill.exec.physical.impl.protocol.OperatorDriver.next():124 org.apache.drill.exec.physical.impl.protocol.OperatorRecordBatch.next():148 org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():237 org.apache.drill.exec.record.AbstractRecordBatch.next():126 org.apache.drill.exec.record.AbstractRecordBatch.next():116 org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext():63 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():141 org.apache.drill.exec.record.AbstractRecordBatch.next():186 org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():237 org.apache.drill.exec.physical.impl.BaseRootExec.next():104 org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():83 org.apache.drill.exec.physical.impl.BaseRootExec.next():94 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():296 org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():283 java.security.AccessController.doPrivileged():-2 javax.security.auth.Subject.doAs():422 org.apache.hadoop.security.UserGroupInformation.doAs():1746 org.apache.drill.exec.work.fragment.FragmentExecutor.run():283 org.apache.drill.common.SelfCleaningRunnable.run():38 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748
