Re: [Firebird-devel] Crash when caught exception is stored in StaticStatusVector
11.04.2016 22:49, Dimitry Sibiryakov wrote: > Let me show that on example from TempSpace::setupFile(): > >> Firebird::StaticStatusVector status_vector; > . >> try >> { > . >> catch (const Firebird::system_error& ex) >> { >> ex.stuffException(status_vector); > > At this point status_vector contain pointers to dynamic strings allocated by > system_exception object ex. > >> continue; >> } > > At this point the exception object is released and all dynamic strings are > freed. > status_vector now is pointing to freed memory. In v2.5 destructor did not release the strings (because they were not dynamically allocated but used a circular buffer), so it worked. >> Firebird::Arg::Gds status(isc_out_of_temp_space); >> status.append(Firebird::Arg::StatusVector(status_vector.begin())); >> status.raise(); > > Here newly created exception object is trying to copy strings from > status_vector, but > they were already replaced with some garbage and the engine crash. I believe some other kind of status holder should be used there. DynamicStatusVector maybe? Do you know other places where this bug exists too? Dmitry -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! https://ad.doubleclick.net/ddm/clk/302982198;130105516;z Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Crash when caught exception is stored in StaticStatusVector
Let me show that on example from TempSpace::setupFile(): > Firebird::StaticStatusVector status_vector; . > try > { . > catch (const Firebird::system_error& ex) > { > ex.stuffException(status_vector); At this point status_vector contain pointers to dynamic strings allocated by system_exception object ex. > continue; > } At this point the exception object is released and all dynamic strings are freed. status_vector now is pointing to freed memory. > Firebird::Arg::Gds status(isc_out_of_temp_space); > status.append(Firebird::Arg::StatusVector(status_vector.begin())); > status.raise(); Here newly created exception object is trying to copy strings from status_vector, but they were already replaced with some garbage and the engine crash. -- WBR, SD. -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! https://ad.doubleclick.net/ddm/clk/302982198;130105516;z Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5194) Invalid computed by definition generated by isql -x
Invalid computed by definition generated by isql -x --- Key: CORE-5194 URL: http://tracker.firebirdsql.org/browse/CORE-5194 Project: Firebird Core Issue Type: Bug Components: ISQL Affects Versions: 3.0 RC2 Reporter: Carlos H. Cantu Priority: Critical This came from Alexandre B. Smith... isql -x from FB 3 RC2 is generated invalid sql script when computed by fields exists. The test works fine in FB 2.5: create database "c:\sp\test.fdb"; CREATE TABLE T (A TIMESTAMP, B COMPUTED BY (current_timestamp - a)); commit; exit; isql -user sysdba -password masterkey test.fdb -x > test.sql The result is: - SET SQL DIALECT 3; CREATE DATABASE 'test.fdb' PAGE_SIZE 8192 DEFAULT CHARACTER SET NONE; COMMIT WORK; /* Table: T, Owner: SYSDBA */ CREATE TABLE T (A TIMESTAMP, B NUMERIC(0, 9) COMPUTED BY (NULL)); /* Computed fields */ ALTER TABLE T ALTER B TYPE NUMERIC(0, 9) COMPUTED BY (current_timestamp - a); - Note the type of Column B defined as *NUMERIC(0,9)* if you run it on isql you will get: - Use CONNECT or CREATE DATABASE to specify a database SQL> CREATE DATABASE 'test2.fdb' PAGE_SIZE 8192 DEFAULT CHARACTER SET NONE; SQL> SQL> SQL> COMMIT WORK; SQL> SQL> /* Table: T, Owner: SYSDBA */ SQL> CREATE TABLE T (A TIMESTAMP, CON> B NUMERIC(0, 9) COMPUTED BY (NULL)); Statement failed, SQLSTATE = HY104 Dynamic SQL Error -SQL error code = -842 -Precision must be from 1 to 18 SQL> SQL> /* Computed fields */ SQL> SQL> ALTER TABLE T CON> ALTER B TYPE NUMERIC(0, 9) COMPUTED BY (current_timestamp - a); Statement failed, SQLSTATE = HY104 Dynamic SQL Error -SQL error code = -842 -Precision must be from 1 to 18 SQL> -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
On Mon, Apr 11, 2016 at 7:40 AM, Dimitry Sibiryakov wrote: > 11.04.2016 13:28, Dmitry Yemanov wrote: > > But it can be made possible. The question is whether it's worth it. > >While bug with orphan index nodes is living in the engine - index only > scan is > impossible completely. > Without transaction information in the index no purely indexed based scan is possible. Orphans don't make any difference. It doesn't matter whether the record isn't there or has had its key value changed by a transaction that's visible to the current transaction. Transformation numeric->double can lose data. Err. Not necessarily and probably there's a work around. Conversions from numeric to double are precise up to 56 bits. For values greater than 56 bits, one could add the last byte of the value to the end of the mangled double and get full precision. With that, you could drop the special indexes for INT64. > Using integers as a key will disable > altering of numeric columns. May be it worth considering. > Not necessary - Even with dealing with fractional values, decimal of fewer than 57 bits will convert properly in both directions. There may be some slop if you try to compare the values exactly, but as long as the conversion works a slight incompatibility doesn't matter. Transformation string->key by ICU does loose data, no way back. > I don't think that matters either. If your collation is accent or case insensitive, your lookup will also be case and accent insensitive. You've asked to lose that information, so it's loss is of no concern. Cheers, Ann -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
> > > > > > > This optimization can only be done if constant conditions are in the > > SQL, so +0 like tricks would not be affected. > > I already considered that, it seems easy enough to implement. > > > > Worth having? -- Regards, Atri *l'apprenant* -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
On Mon, Apr 11, 2016 at 6:15 PM, Dmitry Yemanov wrote: > 11.04.2016 14:40, Dimitry Sibiryakov wrote: > > > > While bug with orphan index nodes is living in the engine - index only > scan is > > impossible completely. > > Pure index-only scan - maybe. It's impossible without transaction > numbers in indices too. But there may be other alternatives to consider. > > > Transformation numeric->double can loose data. Using integers as a key > will disable > > altering of numeric columns. May be it worth considering. > > Perhaps we could live with a requirement to rebuild an index for > integer<->floating modifications. But I'd like to preserve "opaque" > conversions between different NUMERICs. > > +1 -- Regards, Atri *l'apprenant* -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
11.04.2016 14:40, Dimitry Sibiryakov wrote: > > While bug with orphan index nodes is living in the engine - index only scan is > impossible completely. Pure index-only scan - maybe. It's impossible without transaction numbers in indices too. But there may be other alternatives to consider. > Transformation numeric->double can loose data. Using integers as a key will > disable > altering of numeric columns. May be it worth considering. Perhaps we could live with a requirement to rebuild an index for integer<->floating modifications. But I'd like to preserve "opaque" conversions between different NUMERICs. > Transformation string->key by ICU does loose data, no way back. True for case- and accent-insensitive collations, but probably not always true for simple cases. Anyway, I care less about strings than about integers and dates/times. Dmitry -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5193) Operator precedence problem when use: NOT - in complex boolean expression
Operator precedence problem when use: NOT - in complex boolean expression Key: CORE-5193 URL: http://tracker.firebirdsql.org/browse/CORE-5193 Project: Firebird Core Issue Type: Bug Components: Engine Affects Versions: 4.0 Initial, 3.0 RC2 Reporter: Pavel Zotov 1) select (not false = true) is not unknown from rdb$database; -- 2) select not false = true is not unknown from rdb$database; -- Result of first query is obvious and expected. But result of second is equal to: NOT (false = true is not unknown) It seems that results should be evaluated as: (NOT false) = true is not unknown -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
11.04.2016 13:28, Dmitry Yemanov wrote: > But it can be made possible. The question is whether it's worth it. While bug with orphan index nodes is living in the engine - index only scan is impossible completely. Transformation numeric->double can loose data. Using integers as a key will disable altering of numeric columns. May be it worth considering. Transformation string->key by ICU does loose data, no way back. Date/time is fine. -- WBR, SD. -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
11.04.2016 14:16, Dimitry Sibiryakov wrote: > 11.04.2016 13:08, Molnár Attila wrote: >> It's hard and costly (or even possible) to deconstruct the index format to >> get the >> original column values? > > Yes, in common case it is impossible. But it can be made possible. The question is whether it's worth it. Dmitry -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
11.04.2016 14:08, Molnár Attila wrote: > >> When InterBase was created, there was a lot of academic work on >> optimizing corner cases, with the result that academic databases >> tended to spend more time optimizing than retrieving. We made the >> deliberate choice not to spend optimizer time saving idiots from >> themselves. Thirty years later, maybe we'd choose differently. >> However, lots of programs depend on tricks like +0 and concatenating >> with an empty string to coerce unnatural but effective plans. I'd >> worry about the damage done to those cases. > > This optimization can only be done if constant conditions are in the > SQL, so +0 like tricks would not be affected. I already considered that, it seems easy enough to implement. Dmitry -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
11.04.2016 13:08, Molnár Attila wrote: > It's hard and costly (or even possible) to deconstruct the index format to > get the > original column values? Yes, in common case it is impossible. -- WBR, SD. -- Find and fix application performance issues faster with Applications Manager Applications Manager provides deep performance insights into multiple tiers of your business applications. It resolves application problems quickly and reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/ gampad/clk?id=1444514301&iu=/ca-pub-7940484522588532 Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)
On 2016.04.09. 20:25, Ann Harrison wrote: On Fri, Apr 8, 2016 at 5:54 AM, Molnár Attilawrote: Optimizations - IS NOT NULL should use index. It is equivalent with >= min_value or <= max_value based on index direction Histograms and clustered indexes (if they're being considered) could help here to detect cases where IS NOT NULL returns a small subset of the records in a table. In general, searches that touch more than half the records in a table are more efficient when made in storage (natural) order rather than through an index. Remember that Firebird stored data and indexes separately, so setting up an indexed retrieval that will touch every page in a table is just overhead compared with straight-forwardly reading every page. I ment min_value and max_value not base on table data, but on column data type. - condition pre-evaluation and reduction. e.g.: WHERE 1 = 2 AND field = :param is always FALSE. Evaluation does not needed for all records, can decide at prepare time whether the result is an empty result set or an unfiltered result set. When InterBase was created, there was a lot of academic work on optimizing corner cases, with the result that academic databases tended to spend more time optimizing than retrieving. We made the deliberate choice not to spend optimizer time saving idiots from themselves. Thirty years later, maybe we'd choose differently. However, lots of programs depend on tricks like +0 and concatenating with an empty string to coerce unnatural but effective plans. I'd worry about the damage done to those cases. This optimization can only be done if constant conditions are in the SQL, so +0 like tricks would not be affected. - use index in "NATURAL" mode when column in a conditional appears in a multi column index, but not in the first place. You may reduce number of database page visits in this way : index page can hold more effective record data because it's narrower than the table data page record (also in worst case it could be worse than NATURAL because ot the mixed index and table data page read, but I think overall it could worth it, especially in big tables. measurements needed) I not sure what you mean by "NATURAL" index mode - "natural" usually means reading the data pages in storage order without any index. If you mean reading across the leaf level of the index to find matches in the second and subsequent keys in an index, you have no idea how hard that would be. Firebird index keys are mashed up values created so they compare bytewise in the desired order. When using an index, Firebird hasn't a clue where the boundaries fall between columns in multi-column index. It's just bytes. The format makes indexes dense and comparisons quick. Changing the key format to support partial matches on second and third columns seems like a bad idea, given that there's very little difference between having an index on each column and a multi-column index. Remember that Firebird uses multiple indexes on a single table. I ment the second case (reading across the leaf level). It's hard and costly (or even possible) to deconstruct the index format to get the original column values? The question is reading all leaf deconstruct and filter is faster than reading all records natural an filter. - SELECT DISTINCT FROM table is slow (natural scan on all records) and SELECT FROM table GROUP BY is also slow (worse! : index sc