[jira] [Updated] (DERBY-6940) Enhance derby statistics for more accurate selectivity estimates.
[ https://issues.apache.org/jira/browse/DERBY-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Pendleton updated DERBY-6940: --- Attachment: EOFException.txt EOFException_derby.log I see the EOFException, too, when I run that test, with your DERBY-6940_3.diff applied. I attached the terminal output of my test command, and also the derby.log file, so you can look at them in more detail and verify that I'm seeing the same behavior that you are seeing. Looking back over my notes in DERBY-3219, it seems like my concern at the time was that the special-case handling for String objects in FormatIdOutputStream.writeExternal and FormatIdInputStream.readInternal was fragile, because it was ambiguous. However, Mike made some very good points in DERBY-3219 about how it would be a substantial challenge to try to change that behavior at this time. This is a challenge; it isn't obvious to me what we can do here. It seems like the hack that I did for MinMaxAggregator in DERBY-3219 won't work here, because the new StatisticsImpl object has both a Min and a Max field added to it, and we can't arrange for BOTH of those fields to be the last field, unfortunately (at most one of them can be the last one). However, it does seem like the case of having a string of length 0 be the minimum and maximum value for a column will be a quite common special case when we compute statistics on an entirely empty table; perhaps we can decree some other way of recording those statistics (for example, if numRows is 0, then we don't record a string of length 0, instead we record NULL). That is, trying to restate the idea: does this problem only arise when numRows is 0? If so, does that give us a way to construct a workaround? > Enhance derby statistics for more accurate selectivity estimates. > - > > Key: DERBY-6940 > URL: https://issues.apache.org/jira/browse/DERBY-6940 > Project: Derby > Issue Type: Sub-task > Components: SQL >Reporter: Harshvardhan Gupta >Assignee: Harshvardhan Gupta >Priority: Minor > Attachments: DERBY-6940_2.diff, DERBY-6940_3.diff, derby-6940.diff, > EOFException_derby.log, EOFException.txt > > > Derby should collect extra statistics during index build time, statistics > refresh time which will help optimizer make more precise selectivity > estimates and chose better execution paths. > We eventually want to utilize the new statistics to make better selectivity > estimates / cost estimates that will help find the best query plan. Currently > Derby keeps two type of stats - the total row count and the number of unique > values. > We are initially extending the stats to include null count, the minimum value > and maximum value associated with each of the columns of an index. This would > be useful in selectivity estimates for operators such as [ IS NULL, <, <=, >, > >= ] , all of which currently rely on hardwired selectivity estimates. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[Java DB - testing] Failure nightly trunk (rev 1799586)
Java DB testing and reporting infrastructure. Failure nightly trunk (rev 1799586) There were 1 failures.
[jira] [Commented] (DERBY-6940) Enhance derby statistics for more accurate selectivity estimates.
[ https://issues.apache.org/jira/browse/DERBY-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060091#comment-16060091 ] Harshvardhan Gupta commented on DERBY-6940: --- While debugging the failed test cases I seem to have reproduced a previously know issue. Please take a look at my comment on DERBY-3219. > Enhance derby statistics for more accurate selectivity estimates. > - > > Key: DERBY-6940 > URL: https://issues.apache.org/jira/browse/DERBY-6940 > Project: Derby > Issue Type: Sub-task > Components: SQL >Reporter: Harshvardhan Gupta >Assignee: Harshvardhan Gupta >Priority: Minor > Attachments: DERBY-6940_2.diff, DERBY-6940_3.diff, derby-6940.diff > > > Derby should collect extra statistics during index build time, statistics > refresh time which will help optimizer make more precise selectivity > estimates and chose better execution paths. > We eventually want to utilize the new statistics to make better selectivity > estimates / cost estimates that will help find the best query plan. Currently > Derby keeps two type of stats - the total row count and the number of unique > values. > We are initially extending the stats to include null count, the minimum value > and maximum value associated with each of the columns of an index. This would > be useful in selectivity estimates for operators such as [ IS NULL, <, <=, >, > >= ] , all of which currently rely on hardwired selectivity estimates. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (DERBY-3219) Group by query with many aggregate columns and case statements fails with: ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted to read more da
[ https://issues.apache.org/jira/browse/DERBY-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060086#comment-16060086 ] Harshvardhan Gupta commented on DERBY-3219: --- While examining the failed test cases as part of patch for DERBY-6940,I get this error because of exactly the same root cause. i.e value I am trying to store is a SQLVarChar whose value is "" (that is, a string of length 0). ] To reproduce the error with the patch attached with DERBY-6940, run ant -Dderby.junit.testclass=org.apache.derbyTesting.functionTests.tests.lang.OrderByAndSortAvoidance junit-single Since I am trying to filter the min and max values from the database, a string of length 0 turns out to be min value for the test cases in OrderByAndSortAvoidance. > Group by query with many aggregate columns and case statements fails with: > ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted > to read more data than was originally stored > - > > Key: DERBY-3219 > URL: https://issues.apache.org/jira/browse/DERBY-3219 > Project: Derby > Issue Type: Bug > Components: SQL >Affects Versions: 10.3.1.4 >Reporter: Stan Bradbury >Assignee: Bryan Pendleton > Fix For: 10.5.1.1 > > Attachments: maxminPatch.diff, patchWithTest.diff, pivotView.zip, > repro.java, sanityTest.diff, ten_four_patch.diff, testWithMemControls.diff > > > using the attached database (v10.3) - " select * from pivotview " fails with > the stack trace below. A view (pivotview_ok) created on a subset of the > columns in pivotview executes fine. Adding one column back into pivotview_ok > causes failures most of the time. See attached for view definitions. > 2007-11-21 00:58:49.421 GMT Thread[main,5,main] (XID = 2734422), (SESSIONID = > 0), (DATABASE = pivotview), (DRDAID = null), Failed Statement is: select * > from pivotview > ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted > to read more data than was originally stored > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.StreamFileContainer.fetchNext(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.StreamFileContainerHandle.fetchNext(Unknown > Source) > at org.apache.derby.impl.store.access.sort.MergeScan.mergeARow(Unknown > Source) > at org.apache.derby.impl.store.access.sort.MergeScan.init(Unknown > Source) > at > org.apache.derby.impl.store.access.sort.MergeSort.openSortScan(Unknown Source) > at > org.apache.derby.impl.store.access.RAMTransaction.openSortScan(Unknown Source) > at > org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.loadSorter(Unknown > Source) > at > org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.openCore(Unknown > Source) > at > org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown > Source) > at > org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(Unknown Source) > at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown > Source) > at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown > Source) > at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) > at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) > at org.apache.derby.impl.tools.ij.ij.executeImmediate(Unknown Source) > at org.apache.derby.impl.tools.ij.utilMain.doCatch(Unknown Source) > at org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(Unknown Source) > at org.apache.derby.impl.tools.ij.utilMain.go(Unknown Source) > at org.apache.derby.impl.tools.ij.Main.go(Unknown Source) > at org.apache.derby.impl.tools.ij.Main.mainCore(Unknown Source) > at org.apache.derby.impl.tools.ij.Main14.main(Unknown Source) > at org.apache.derby.tools.ij.main(Unknown Source) > Caused by: java.io.EOFException > at java.io.DataInputStream.readBoolean(DataInputStream.java:248) > at > org.apache.derby.impl.sql.execute.MaxMinAggregator.readExternal(Unknown > Source) > at > org.apache.derby.iapi.services.io.FormatIdInputStream.readObject(Unknown > Source) > at org.apache.derby.iapi.types.UserType.readExternal(Unknown Source) > ... 22 more > = begin nested exception, level (1) === > java.io.EOFException > at java.io.DataInputStream.readBoolean(DataInputStream.java:248) > at > org.apache.derby.impl.sql.execute.MaxMinAggregator.readExternal(Unknown > Source) > at > org.apache.derb
[Java DB - testing] Failure continuous trunk (rev 1798340)
Java DB testing and reporting infrastructure. Failure continuous trunk (rev 1798340) There were 16 failures.
[jira] [Commented] (DERBY-6905) Enhance SYS.SYSCOLUMNS to report the CYCLE value for an IDENTITY column
[ https://issues.apache.org/jira/browse/DERBY-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059411#comment-16059411 ] ASF subversion and git services commented on DERBY-6905: Commit 1799586 from [~bryanpendleton] in branch 'code/trunk' [ https://svn.apache.org/r1799586 ] DERBY-6905: Enhance SYS.SYSCOLUMNS to report the CYCLE value for IDENTITY column This patch updates the dblook test output to reflect the expected value for the CYCLE column in the output. > Enhance SYS.SYSCOLUMNS to report the CYCLE value for an IDENTITY column > --- > > Key: DERBY-6905 > URL: https://issues.apache.org/jira/browse/DERBY-6905 > Project: Derby > Issue Type: Sub-task > Components: SQL >Affects Versions: 10.13.1.0 >Reporter: Bryan Pendleton >Assignee: Harshvardhan Gupta >Priority: Minor > Fix For: 10.14.0.0 > > Attachments: dblook_tests.diff > > > DERBY-6542 changed IDENTITY columns to build upon the underlying > SEQUENCE objects. SEQUENCE objects can have a CYCLE option > set for them, and as of DERBY-6852, the CYCLE option controls the > behavior of the IDENTITY column that uses that SEQUENCE. > It would be nice if SYS.SYSCOLUMNS would report the CYCLE option > that is in effect for a IDENTITY column in that column's row in SYSCOLUMNS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
