Re: [Firebird-devel] Crash when caught exception is stored in StaticStatusVector

2016-04-11 Thread Dmitry Yemanov 
11.04.2016 22:49, Dimitry Sibiryakov wrote:

> Let me show that on example from TempSpace::setupFile():
>
>>  Firebird::StaticStatusVector status_vector;
> .
>>  try
>>  {
> .
>>  catch (const Firebird::system_error& ex)
>>  {
>>  ex.stuffException(status_vector);
>
> At this point status_vector contain pointers to dynamic strings allocated by
> system_exception object ex.
>
>>  continue;
>>  }
>
> At this point the exception object is released and all dynamic strings are 
> freed.
> status_vector now is pointing to freed memory.

In v2.5 destructor did not release the strings (because they were not 
dynamically allocated but used a circular buffer), so it worked.

>>  Firebird::Arg::Gds status(isc_out_of_temp_space);
>>  status.append(Firebird::Arg::StatusVector(status_vector.begin()));
>>  status.raise();
>
> Here newly created exception object is trying to copy strings from 
> status_vector, but
> they were already replaced with some garbage and the engine crash.

I believe some other kind of status holder should be used there. 
DynamicStatusVector maybe?

Do you know other places where this bug exists too?


Dmitry


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] Crash when caught exception is stored in StaticStatusVector

2016-04-11 Thread Dimitry Sibiryakov 
   Let me show that on example from TempSpace::setupFile():

>   Firebird::StaticStatusVector status_vector;
.
>   try
>   {
.
>   catch (const Firebird::system_error& ex)
>   {
>   ex.stuffException(status_vector);

   At this point status_vector contain pointers to dynamic strings allocated by 
system_exception object ex.

>   continue;
>   }

   At this point the exception object is released and all dynamic strings are 
freed. 
status_vector now is pointing to freed memory.

>   Firebird::Arg::Gds status(isc_out_of_temp_space);
>   status.append(Firebird::Arg::StatusVector(status_vector.begin()));
>   status.raise();

   Here newly created exception object is trying to copy strings from 
status_vector, but 
they were already replaced with some garbage and the engine crash.

-- 
   WBR, SD.

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5194) Invalid computed by definition generated by isql -x

2016-04-11 Thread Carlos H. Cantu (JIRA) 
Invalid computed by definition generated by isql -x
---

 Key: CORE-5194
 URL: http://tracker.firebirdsql.org/browse/CORE-5194
 Project: Firebird Core
  Issue Type: Bug
  Components: ISQL
Affects Versions: 3.0 RC2
Reporter: Carlos H. Cantu
Priority: Critical


This came from Alexandre B. Smith... isql -x from FB 3 RC2 is generated invalid 
sql script when computed by fields exists. The test works fine in FB 2.5:

create database "c:\sp\test.fdb";


CREATE TABLE T (A TIMESTAMP,
B COMPUTED BY (current_timestamp - a));


commit;


exit;


isql -user sysdba -password masterkey test.fdb -x > test.sql


The result is:
-
SET SQL DIALECT 3; 


CREATE DATABASE 'test.fdb' PAGE_SIZE 8192 DEFAULT CHARACTER SET NONE;




COMMIT WORK;


/* Table: T, Owner: SYSDBA */
CREATE TABLE T (A TIMESTAMP,
B NUMERIC(0, 9) COMPUTED BY (NULL));


/* Computed fields */


ALTER TABLE T 
ALTER B TYPE NUMERIC(0, 9) COMPUTED BY (current_timestamp - a);


-


Note the type of Column B defined as *NUMERIC(0,9)*


if you run it on isql you will get:
-
Use CONNECT or CREATE DATABASE to specify a database
SQL> CREATE DATABASE 'test2.fdb' PAGE_SIZE 8192 DEFAULT CHARACTER SET NONE;
SQL>
SQL>
SQL> COMMIT WORK;
SQL>
SQL> /* Table: T, Owner: SYSDBA */
SQL> CREATE TABLE T (A TIMESTAMP,
CON> B NUMERIC(0, 9) COMPUTED BY (NULL));
Statement failed, SQLSTATE = HY104
Dynamic SQL Error
-SQL error code = -842
-Precision must be from 1 to 18
SQL>
SQL> /* Computed fields */
SQL>
SQL> ALTER TABLE T
CON> ALTER B TYPE NUMERIC(0, 9) COMPUTED BY (current_timestamp - a);
Statement failed, SQLSTATE = HY104
Dynamic SQL Error
-SQL error code = -842
-Precision must be from 1 to 18
SQL>


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Ann Harrison 
On Mon, Apr 11, 2016 at 7:40 AM, Dimitry Sibiryakov 
wrote:

> 11.04.2016 13:28, Dmitry Yemanov wrote:
> > But it can be made possible. The question is whether it's worth it.
>
>While bug with orphan index nodes is living in the engine - index only
> scan is
> impossible completely.
>

Without transaction information in the index no purely indexed based scan
is possible.  Orphans don't make any difference.  It doesn't matter whether
the record isn't there or has had its key value  changed by a transaction
that's visible to the current transaction.

   Transformation numeric->double can lose data.


Err.  Not necessarily and probably there's a work around.  Conversions from
numeric to double are precise up to 56 bits.  For values greater than 56
bits, one could add the last byte of the value to the end of the mangled
double and get full precision.  With that, you could drop the special
indexes for INT64.



> Using integers as a key will disable
> altering of numeric columns. May be it worth considering.
>

Not necessary - Even with dealing with fractional values, decimal of fewer
than 57 bits will convert properly in both directions.  There may be some
slop if you try to compare the values exactly, but as long as the
conversion works a slight incompatibility doesn't matter.

   Transformation string->key by ICU does loose data, no way back.
>

I don't think that matters either.  If your collation is accent or case
insensitive, your lookup will also be case and accent insensitive.  You've
asked to lose that information, so it's loss is of no concern.

Cheers,

Ann
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Atri Sharma 
>
>
>
>  >
> > This optimization can only be done if constant conditions are in the
> > SQL, so +0 like tricks would not be affected.
>
> I already considered that, it seems easy enough to implement.
>
>
>
>
Worth having?



-- 
Regards,

Atri
*l'apprenant*
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Atri Sharma 
On Mon, Apr 11, 2016 at 6:15 PM, Dmitry Yemanov  wrote:

> 11.04.2016 14:40, Dimitry Sibiryakov wrote:
> >
> > While bug with orphan index nodes is living in the engine - index only
> scan is
> > impossible completely.
>
> Pure index-only scan - maybe. It's impossible without transaction
> numbers in indices too. But there may be other alternatives to consider.
>
> > Transformation numeric->double can loose data. Using integers as a key
> will disable
> > altering of numeric columns. May be it worth considering.
>
> Perhaps we could live with a requirement to rebuild an index for
> integer<->floating modifications. But I'd like to preserve "opaque"
> conversions between different NUMERICs.
>
>
+1


-- 
Regards,

Atri
*l'apprenant*
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Dmitry Yemanov 
11.04.2016 14:40, Dimitry Sibiryakov wrote:
>
> While bug with orphan index nodes is living in the engine - index only scan is
> impossible completely.

Pure index-only scan - maybe. It's impossible without transaction 
numbers in indices too. But there may be other alternatives to consider.

> Transformation numeric->double can loose data. Using integers as a key will 
> disable
> altering of numeric columns. May be it worth considering.

Perhaps we could live with a requirement to rebuild an index for 
integer<->floating modifications. But I'd like to preserve "opaque" 
conversions between different NUMERICs.

> Transformation string->key by ICU does loose data, no way back.

True for case- and accent-insensitive collations, but probably not 
always true for simple cases. Anyway, I care less about strings than 
about integers and dates/times.


Dmitry


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

[Firebird-devel] [FB-Tracker] Created: (CORE-5193) Operator precedence problem when use: NOT - in complex boolean expression

2016-04-11 Thread Pavel Zotov (JIRA) 
Operator precedence problem when use:  NOT  - in complex 
boolean expression


 Key: CORE-5193
 URL: http://tracker.firebirdsql.org/browse/CORE-5193
 Project: Firebird Core
  Issue Type: Bug
  Components: Engine
Affects Versions: 4.0 Initial, 3.0 RC2
Reporter: Pavel Zotov


1) select (not false = true) is not unknown from rdb$database; -- 

2) select not false = true is not unknown from rdb$database; -- 

Result of first query is obvious and expected. 
But result of second is equal to:

NOT (false = true is not unknown)

It seems that results should be evaluated as: (NOT false) = true is not unknown


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: 
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira



--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Dimitry Sibiryakov 
11.04.2016 13:28, Dmitry Yemanov wrote:
> But it can be made possible. The question is whether it's worth it.

   While bug with orphan index nodes is living in the engine - index only scan 
is 
impossible completely.
   Transformation numeric->double can loose data. Using integers as a key will 
disable 
altering of numeric columns. May be it worth considering.
   Transformation string->key by ICU does loose data, no way back.
   Date/time is fine.

-- 
   WBR, SD.

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Dmitry Yemanov 
11.04.2016 14:16, Dimitry Sibiryakov wrote:

> 11.04.2016 13:08, Molnár Attila wrote:
>> It's hard and costly (or even possible) to deconstruct the index format to 
>> get the
>> original column values?
>
> Yes, in common case it is impossible.

But it can be made possible. The question is whether it's worth it.


Dmitry


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Dmitry Yemanov 
11.04.2016 14:08, Molnár Attila wrote:
>
>> When InterBase was created, there was a lot of academic work on
>> optimizing corner cases, with the result that academic databases
>> tended to spend more time optimizing than retrieving.  We made the
>> deliberate choice not to spend optimizer time saving idiots from
>> themselves. Thirty years later, maybe we'd choose differently.
>> However, lots of programs depend on tricks like +0 and concatenating
>> with an empty string to coerce unnatural but effective plans.  I'd
>> worry about the damage done to those cases.
 >
> This optimization can only be done if constant conditions are in the
> SQL, so +0 like tricks would not be affected.

I already considered that, it seems easy enough to implement.


Dmitry


--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Dimitry Sibiryakov 
11.04.2016 13:08, Molnár Attila wrote:
> It's hard and costly (or even possible) to deconstruct the index format to 
> get the
> original column values?

   Yes, in common case it is impossible.

-- 
   WBR, SD.

--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial! http://pubads.g.doubleclick.net/
gampad/clk?id=1444514301=/ca-pub-7940484522588532
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Molnár Attila 

  
  


On 2016.04.09. 20:25, Ann Harrison
  wrote:


  

  On Fri, Apr 8, 2016 at 5:54 AM,
Molnár Attila 
wrote:

  
  
  
Optimizations
    - IS NOT NULL should use index. It is equivalent
with >= min_value or <= max_value based on index
direction
  



 Histograms and clustered indexes (if they're being
  considered) could help here to detect cases where IS NOT
  NULL returns a small subset of the records in a table.  In
  general, searches that touch more than half the records in
  a table are more efficient when made in storage (natural)
  order rather than through an index.  Remember that
  Firebird stored data and indexes separately, so setting up
  an indexed retrieval that will touch every page in a table
  is just overhead compared with straight-forwardly reading
  every page.
  

  

I ment min_value and max_value not base on table data, but on column
data type.

  

  



       - condition
pre-evaluation and reduction. e.g.: WHERE 1 = 2 AND
field = :param is always FALSE. Evaluation does not
needed for all records, can decide at prepare time
whether the result is an empty result set or an
unfiltered result set.
  



When InterBase was created, there was a lot of academic
  work on optimizing corner cases, with the result that
  academic databases tended to spend more time optimizing
  than retrieving.  We made the deliberate choice not to
  spend optimizer time saving idiots from themselves. 
  Thirty years later, maybe we'd choose differently. 
  However, lots of programs depend on tricks like +0 and
  concatenating with an empty string to coerce unnatural but
  effective plans.  I'd worry about the damage done to those
  cases.
  

  

This optimization can only be done if constant conditions are in the
SQL, so +0 like tricks would not be affected.

  

  
 


       - use index in
"NATURAL" mode when column in a conditional appears in a
multi column index, but not in the first place. You may
reduce number of database page visits in this way :
index page can hold more effective record data because
it's narrower than the table data page record (also in
worst case it could be worse than NATURAL because ot the
mixed index and table data page read, but I think
overall it could worth it, especially in big tables.
measurements needed)
  



I not sure what you mean by "NATURAL" index mode -
  "natural" usually means reading the data pages in storage
  order without any index. If you mean reading across the
  leaf level of the index to find matches in the second and
  subsequent keys in an index, you have no idea how hard
  that would be.  Firebird index keys are mashed up values
  created so they compare bytewise in the desired order. 
  When using an index, Firebird hasn't a clue where the
  boundaries fall between columns in multi-column index. 
  It's just bytes.   The format makes indexes dense and
  comparisons quick.   Changing the key format to support
  partial matches on second and third columns seems like a
  bad idea, given that there's very little difference
  between having an index on each column and a multi-column
  index.  Remember that Firebird uses multiple indexes on a
  single table. 

  

  

I ment the second case (reading across the leaf level). It's hard
and costly (or even possible) to deconstruct the index format to get
the original column values? The question is reading all leaf
deconstruct and filter is faster than reading all records natural an
filter.

  

  


 

       - SELECT
DISTINCT  FROM table is slow
(natural scan on all records) and SELECT  FROM table GROUP BY  is
also slow (worse! : index