Re: [Firebird-devel] isc_dpb_dbkey_scope

2022-05-26 Thread Molnár Attila
https://en.wikipedia.org/wiki/Multiversion_concurrency_control#History


-Eredeti üzenet-
Feladó: Pól Ua L. via Firebird-devel 
[mailto:firebird-devel@lists.sourceforge.net] 
Küldve: 2022. május 26., csütörtök 9:11
Címzett: For discussion among Firebird Developers 

Másolatot kap: Pól Ua L. 
Tárgy: Re: [Firebird-devel] isc_dpb_dbkey_scope



Hi Jim,

great to see that you're still keeping your finger in the Firebird pie!

> There is a very good reason that multi-version concurrency control won
> out big time (Wikipedia lists 80+ database systems with MVCC).
> Interbase was the second.

Just as a matter of historical interest, which was the first database to 
release with MVCC? Oracle?

Best regards,

Pól...


> Jim Starkey, AmorphousDB, LLC




Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Tablespaces proposal

2021-10-07 Thread Molnár Attila
"Tablespaces has meaning for large databases only that don't fit into single 
storage (terrabytes)."

That is not true. It has meaning whatever the programmers meant to use it. It 
might not be about read performance, but e.g. logical data serparation, backup 
speedup, etc...
Also you should not just thinking in a single huge database, but hundreds or 
thousands mid-to-large databases as well. That could also occupy huge amount of 
disk space. (cloud service)



-Eredeti üzenet-
Feladó: Dimitry Sibiryakov [mailto:s...@ibphoenix.com] 
Küldve: 2021. október 7., csütörtök 10:47
Címzett: For discussion among Firebird Developers 

Tárgy: Re: [Firebird-devel] Tablespaces proposal

Roman Simakov wrote 06.10.2021 21:49:
>> How are data moved?
>>
>> Under transaction control?
>>
>> In background (and interruptible) like db crypt?
> it's a dfw operation with EX database lock. I hope to relax this
> limitation later. So there are no concurrent changes.
> 1) copy all data pages
> 2) switch RDB$POINTER_PAGE and RDB$ROOT_PAGE transactionally
> 3) Rebuild RDB$PAGES
> 4) clear old data pages (as post-dfw operation)
> 
> It can be interrupted but not resumed. I'm afraid it starts from the 
> beginning.
> 
> I guess it's better to discuss in a context of PR.

   Tablespaces has meaning for large databases only that don't fit into single 
storage (terrabytes). For such volumes this algorithm is no-go. Better not to 
have move option in the initial implementation at all.

-- 
   WBR, SD.


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Dialect 3 inconsistent round/trunc - configurable calculation method needed

2021-09-01 Thread Molnár Attila
Ok, I understand it's not "wrong", works as designed/intended. Just put into 
documentation.

Also, how do you expect to people to leave dialect 1 when dialect 3 gives no 
benefit, in both dialect extra effort needed to get calculations right, plain 
operation and funcation usage is not enought to get correct results: 
dialect 1 requires rounding everywhere because the everything is a float
dialect 3 requires double precition cast and rounding in case of division 
because of low precision/scale during calculation

A new dialect or a new option should do the job.

-Eredeti üzenet-
Feladó: Dimitry Sibiryakov [mailto:s...@ibphoenix.com] 
Küldve: 2021. szeptember 1., szerda 11:27
Címzett: For discussion among Firebird Developers 

Tárgy: Re: [Firebird-devel] Dialect 3 inconsistent round/trunc - configurable 
calculation method needed

Molnár Attila wrote 01.09.2021 8:29:
> The problem: why the storage format has affect on calculation. It just feels 
> wrong.

   It is ordinary thing in all programming languages when data types decide 
calculation rules. In the Delphi product of integer division used to be 
floating 
point, but that's Pascal exclusive rules. SQL (including Firebird) is based 
rather on C where division of two integers produce integer.

> What is the point of dialect 3 when it does not spare work, it just replace 
> one boilerplate code writing to an other?

   If it doesn't work the way you expect it doesn't mean that it works wrong. 
Just accept that rules of arithmetic are different in different programming 
languages and perform calculations in the language which comply with your goals.

-- 
   WBR, SD.


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel

Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Dialect 3 inconsistent round/trunc - configurable calculation method needed

2021-09-01 Thread Molnár Attila
"I suggested you start a discussion about your wish that dialect 1 would not be 
dropped"
Oh, sorry for the misunederstanding.

After your comment on the GitHub issue I understand how the calcuation works.
The problem: why the storage format has affect on calculation. It just feels 
wrong.

I did not suggesed to throw away current method, I understand the concept of 
backward compability. I asked for a new configuration.

In dialect 1 you have to round always because of the float format storage.
In dialect 3 you have to cast to double always for not to loose precision.
So both dialect you have to write unnecessary boilerplace code. If I want 
divide, it just not enough to write a divide.

I want a mode, where calcuation do not require boilerplate round or cast:
 - numeric storage format is like dialect 3: exact value stored
 - calcuations are made in double precision float like dialect 1, OR made as in 
dialect 3, but with higher precision, and the rounding happens automatically 
when you assign/store a double value to a numeric.

What is the point of dialect 3 when it does not spare work, it just replace one 
boilerplate code writing to an other?

(The project is ~20 years old started with Delphi 5, which did not supported 
dialect 3, so dialect 1 was not a deliberate choice, but the only option.)



Attila


-Eredeti üzenet-
Feladó: Mark Rotteveel [mailto:m...@lawinegevaar.nl] 
Küldve: 2021. augusztus 31., kedd 19:02
Címzett: firebird-devel@lists.sourceforge.net
Tárgy: Re: [Firebird-devel] Dialect 3 inconsistent round/trunc - configurable 
calculation method needed

On 31-08-2021 16:19, Molnár Attila wrote:
> 127.13 / 3.4618 = 36,72366976717315

It isn't, taken literally in dialect 3, the result is 36,723669 ;).

> EXECUTE BLOCK
> RETURNS (
>    c0 NUMERIC(15,10),
>    c1 NUMERIC(15,10),
>    c2 NUMERIC(15,6),
>    c3 NUMERIC(15,5),
>    c4 NUMERIC(15,4))
> AS
> DECLARE VARIABLE a NUMERIC(15,2);
> DECLARE VARIABLE b NUMERIC(15,4);
> BEGIN
>    a = 127.13;
>    b = 3.4618;
>    c0 = a / CAST(b AS DOUBLE PRECISION);--36,7236697672, this is OK
>    c1 = a / b;--36,723669, this shoud be 36,723670
>    c2 = a / b;--36,723669, this shoud be 36,723670
>    c3 = a / b;--36,72367, this is OK
>    c4 = a / b;--36,7237, this is OK
>    SUSPEND;
> END
> 
> If return variable scale is bigger or equal than operation result scale, 
> then computed value is truncated, but when return variable scale is 
> smaller than operation result scale then computed value is rounded.
> I think in all case value should be rounded, this behaviour is 
> inconsistent, and also truncation is unexpected behaviour.
> 
> I opened a GitHub issue , but was closed : 
> https://github.com/FirebirdSQL/firebird/issues/6928 
> <https://github.com/FirebirdSQL/firebird/issues/6928>
> Mark suggested to start a conversation here.

I didn't suggest you start a discussion about this behaviour, I 
suggested you start a discussion about your wish that dialect 1 would 
not be dropped ("If you want to discuss retention of dialect 1, then 
please take that to firebird-devel.").

This has been the behaviour of NUMERIC/DECIMAL division in dialect 3 
since InterBase 6.0. Changing that behaviour now would break or 
invalidate all applications that have been built with dialect 3 
semantics of NUMERIC/DECIMAL division in mind.

It complies with the SQL standard. Though to be honest, that is a pretty 
low bar, as the specification doesn't specify much for NUMERIC/DECIMAL 
division other than that precision and rounding is implementation 
specific ("iv) The precision and scale of the result of division are 
implementation-defined." - SQL:2016-2, 6.29 )

> #1 : This behaviour should be documented, I found nothing regarding 
> this, and this result is unexpected. Calculation do not happens in 
> double precision, and calculation stops when result scale is reached.

The result is unexpected *to you* and does not match what you've been 
relying on in dialect 1. Yes, it is not explicitly documented other than 
the resulting precision, but neither is the behaviour of division in 
dialect 1 other than that it results in a DOUBLE PRECISION. You can 
think of NUMERIC/DECIMAL division in dialect 3 as a form of integer 
division like in most programming language. That also stops when the 
final digit has been calculated. For example, 14/3 = 4, not 5, 14.0/3 = 
4.6, not 4.7, 14.0/3.0 = 4.66, not 4.67, etc, in dialect 1, all those 
calculations produce 4.667. For dialect 3, you can think of 
14.0/3.0 as doing 14000/30 and then shifting the decimal by two 
positions (14000/30 = 466, shift decimal by 2 = 4.66).

You haven't brought forth any real arguments why 21+ years of dialect 3 
behaviour should be thrown out the window, other than that you don't 
like it because it isn't the same as in dialect 1.

I understa

[Firebird-devel] Dialect 3 inconsistent round/trunc - configurable calculation method needed

2021-08-31 Thread Molnár Attila
Hi *!

127.13 / 3.4618 = 36,72366976717315

EXECUTE BLOCK
RETURNS (
  c0 NUMERIC(15,10),
  c1 NUMERIC(15,10),
  c2 NUMERIC(15,6),
  c3 NUMERIC(15,5),
  c4 NUMERIC(15,4))
AS
DECLARE VARIABLE a NUMERIC(15,2);
DECLARE VARIABLE b NUMERIC(15,4);
BEGIN
  a = 127.13;
  b = 3.4618;
  c0 = a / CAST(b AS DOUBLE PRECISION);--36,7236697672, this is OK
  c1 = a / b;--36,723669, this shoud be 36,723670
  c2 = a / b;--36,723669, this shoud be 36,723670
  c3 = a / b;--36,72367, this is OK
  c4 = a / b;--36,7237, this is OK
  SUSPEND;
END

If return variable scale is bigger or equal than operation result scale, then 
computed value is truncated, but when return variable scale is smaller than 
operation result scale then computed value is rounded.
I think in all case value should be rounded, this behaviour is inconsistent, 
and also truncation is unexpected behaviour.

I opened a GitHub issue , but was closed : 
https://github.com/FirebirdSQL/firebird/issues/6928
Mark suggested to start a conversation here.

#1 : This behaviour should be documented, I found nothing regarding this, and 
this result is unexpected. Calculation do not happens in double precision, and 
calculation stops when result scale is reached.

#2 : Calculation method should be cofigurable
#2/A: possible configuration modes

-  default: current mode

-  double precision: operands converted to double, calculations happens 
in double, then result rounded and converted back to result type

-  operand max: current calcuation method, but calcuation stops at 
max(op1.precision, op2.precision) instead of res.scale, then result rounded to 
res.scale

-  : current calcuation method, but calcuation stops at 
 instead of res.scale, then result rounded res.scale

#2/B: possible configuration places


-  some kind of SQL command (connection level), e.g.: SET CALCULATION 
PRECISION DEFAULT | DOUBLE PRECISION | OP_MAX | . Then in a 
database on connect trigger can set this, when needed

-  new dialect (don't prefer, this just affects calculation not storage)

-  firebird.conf (don't prefer, this should be linked to the database 
file)

-  connection string (don't prefer, this should be linked to the 
database file)



[cid:image002.png@01D79E83.EB3C4920]<http://www.libra.hu/>
CÉGÜNK A LIBRA CSOPORT TAGJA

Molnár Attila
fejlesztő

LIBRA Szoftver Zrt.
1113 Budapest, Karolina út 65.
+36 (1) 255-3939
i...@libra.hu<mailto:i...@libra.hu> | www.libra.hu<http://www.libra.hu>







__ Information from ESET Mail Security, version of virus signature 
database 23884 (20210831) __

The message was checked by ESET Mail Security.
http://www.eset.com

Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] ODP: Modern C++: constexpr

2020-06-18 Thread Molnár Attila
In my point of view (not a native english speaker)

owner : ownership is a legal term basicly. It gives the owner rights over the 
owned property. 
master : master has actual power over the slave to command/control the slave. 



-Eredeti üzenet-
Feladó: Dimitry Sibiryakov [mailto:s...@ibphoenix.com] 
Küldve: 2020. június 18., csütörtök 11:37
Címzett: For discussion among Firebird Developers 

Tárgy: Re: [Firebird-devel] ODP: Modern C++: constexpr

18.06.2020 10:05, Mark Rotteveel wrote:
>>   BTW, when you start your search-and-replace work, don't forget to
>> find substitutions for "owner", "invalid", "dependency" and "creator".
>> First and third offend ex-slaves, second - disabled people, last -
>> religious ones.
> 
> I'm sorry, but that is just plain silly and trying to trivialize the problem.

   Sorry, it is out of my English again. What is the difference between 
"master" and "owner"?

-- 
   WBR, SD.


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel



__ Information from ESET Mail Security, version of virus signature 
database 21513 (20200618) __

The message was checked by ESET Mail Security.
http://www.eset.com




Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] ICU File Deployment Guidance

2020-04-29 Thread Molnár Attila
You can posrt the difference between new ICU versoins in firebirdsql.org to 
help developers/users/admins to decide wheter an update nedded at all.

-Eredeti üzenet-
Feladó: Paul Reeves [mailto:pree...@mail.ibphoenix.com] 
Küldve: 2020. április 29., szerda 13:38
Címzett: firebird-devel@lists.sourceforge.net
Tárgy: Re: [Firebird-devel] ICU File Deployment Guidance

On Wed, 29 Apr 2020 11:59:43 +0100
Tony Whyman  wrote:

> Hopefully, an installer package will eventually be made available to
> automate the process. However, the above manual procedure is all that is
> currently available.

I'm wondering how people think we could automate this process. Thoughts that
immediately come to mind are:

  - We have no control over the release cycle of tzdata. The update to
https://github.com/FirebirdSQL/firebird/tree/master/extern/icu/tzdata
seems to be manual at the moment.

  - Looking at the update history over at
https://github.com/unicode-org/icu-data/tree/master/tzdata/icunew there
can be up to 10 changes per year.

  - The changes will, in all probability, more than 99% of the time, have no
relevance for over 99% of our users. But can one take the risk of
ignoring them? 

  - Even if we can provide timely, automated updates how will a server
installation know to look for them? A task scheduler of some sort would
need to check. We don't have one built in to Firebird so it would mean
adding something platform specific.


Paul
-- 

Paul Reeves
http://www.ibphoenix.com
Supporting users of Firebird
 


Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


__ Information from ESET Mail Security, version of virus signature 
database 21244 (20200429) __

The message was checked by ESET Mail Security.
http://www.eset.com




__ Information from ESET Mail Security, version of virus signature 
database 21244 (20200429) __

The message was checked by ESET Mail Security.
http://www.eset.com




Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] interesting test fails with Firebird 3.0

2019-11-19 Thread Molnár Attila
Firebird UPDATE supports ORDER BY clause.

Feladó: marius adrian popa [mailto:map...@gmail.com]
Küldve: 2019. november 19., kedd 10:48
Címzett: For discussion among Firebird Developers 

Tárgy: [Firebird-devel] interesting test fails with Firebird 3.0

https://twitter.com/FranckPachot/status/1196415263031414786


update demo set n=n+1;

violation of PRIMARY or UNIQUE KEY constraint "INTEG_2" on table "DEMO" 
Problematic key value is ("N" = 2)
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Inserts and FKs

2019-09-06 Thread Molnár Attila
BTW I also agree that this shoud throw FK voilation exception on detail insert.


-Eredeti üzenet-
Feladó: Carlos H. Cantu [mailto:lis...@warmboot.com.br] 
Küldve: 2019. szeptember 6., péntek 14:13
Címzett: For discussion among Firebird Developers 

Tárgy: Re: [Firebird-devel] Inserts and FKs

MA> If you do master and detail in separate transaction then tx2 should 
MA> not be started before tx1, tx2 start MUST wait for tx1 SUCCESSFUL 
MA> commit, otherwise your business logis FLAWED.

Tx2 wasn't start before tx1. Read again my first email.

[]s
Carlos
http://www.firebirdnews.org
FireBase - http://www.FireBase.com.br



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


__ Information from ESET Mail Security, version of virus signature 
database 19976 (20190906) __

The message was checked by ESET Mail Security.
http://www.eset.com




__ Information from ESET Mail Security, version of virus signature  
database 19976 (20190906) __

The message was checked by ESET Mail Security.
http://www.eset.com




Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Inserts and FKs

2019-09-06 Thread Molnár Attila
Why do you insert master and detail record in separate transaction?
If you do master and detail in separate transaction then tx2 should not be 
started before tx1, tx2 start MUST wait for tx1 SUCCESSFUL commit, otherwise 
your business logis FLAWED.

This scenario is looks like "Dirty read" : 
https://en.wikipedia.org/wiki/Isolation_(database_systems)#Dirty_reads. You 
leak uncommied (for tx2) information.


-Eredeti üzenet-
Feladó: Carlos H. Cantu [mailto:lis...@warmboot.com.br] 
Küldve: 2019. szeptember 6., péntek 1:47
Címzett: firebird-devel@lists.sourceforge.net
Tárgy: [Firebird-devel] Inserts and FKs

Recently I had a real situation that drove me crazy for several days while 
trying to find what actually happened. After lots of thinking, checking the 
auditing logs and chatting with Vlad, the only scenario that could explain what 
happened is:

TableA (master)
TableB (detail)
There is FK between TableB and TableA

tx1 starts
tx1 inserts master record in TableA
tx2 starts (snapshot isolation)
tx1 commits
tx2 inserts a detail record in TableB
Trigger on TableB tries to update master record,
since the record is still not visible for this snapshot,
update has null effect
tx2 commits (no error)

The detail insert does not fail with broken FK because FK checking is out of 
transaction control, but the update ran by the trigger had no effect because 
the master record was not visible for the update.

>From the consistency point of view, in this example, if the trigger could not 
>see the record and modify it, the FK validation should fail too. IMHO, if the 
>transaction isolation doesn't allow the record to be seen (at trigger level) 
>it should not be seen at the FK validation level too.

I understand that there are other scenarios where the currently FK behavior is 
correct and makes sense, for example, in the case of avoiding deleting a master 
record with "commited but not visible childs", but for the reported example, 
the currently behavior looks incorrect, and for people with business logic 
implemented in triggers, it may/will lead to incorrect results.

Does anyone knows if this behavior is following the Standard?

Any comments?

[]s
Carlos
http://www.firebirdnews.org



Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


__ Information from ESET Mail Security, version of virus signature 
database 19974 (20190905) __

The message was checked by ESET Mail Security.
http://www.eset.com




__ Information from ESET Mail Security, version of virus signature  
database 19976 (20190906) __

The message was checked by ESET Mail Security.
http://www.eset.com




Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Database opened in another session

2017-12-07 Thread Molnár Attila

This might be a GC/sweep issue

http://tracker.firebirdsql.org/browse/CORE-4751
http://tracker.firebirdsql.org/browse/CORE-4745



On 2017.12.07 16:35, Vlad Khorsun via Firebird-devel wrote:

07.12.2017 17:16, Jiří Činčura wrote:

Nope. Still happening. Even with `-i`.


  Hmm... i was not attentive enough, error is about database not about 
listener.


  It means that one engine instance is already open database and 
another instance
going to open it too and both instances runs in different windows 
sessions.


  For example, one instance run in user session (for ex. embedded 
connection)

and second instance run as a service.

Regards,
Vlad

-- 


Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] [SPAM] Re: Start transaction from base transaction

2017-04-20 Thread Molnár Attila
Awesome! :)

On 2017.04.21. 1:30, Adriano dos Santos Fernandes wrote:
> Em 20/04/2017 17:53, Vlad Khorsun escreveu:
>> Also it is necessary
>> to teach engine to use that metadata (instead of current one) within 
>> attachment
>> working "in the past".
>>
> I'm doing a prototype implementation of this for active transactions,
> i.e., the things I mentioned in this thread start.
>
> I've it starting working, but implementation is very simple, weak and
> almost non-tested at the moment.
>
>
> Adriano
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> Firebird-Devel mailing list, web interface at 
> https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Start transaction from base transaction

2017-04-20 Thread Molnár Attila
+1 for this feature. I would be very happy for this. Also it would be 
awesmone if this consistent view were accessible later in time (this 
woudl mean garbage collection blocking).

On 2017.04.18. 20:56, Leyne, Sean wrote:
>
>> If you need simultaneous queries - make them possible,
>> what the point of transaction hacking?
> You want a single "view" of the database from multiple _connections_.
>
> There is nothing that provides this, today -- There is no way to ensure that 
> all connections have that same view.
>
>
> Sean
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> Firebird-Devel mailing list, web interface at 
> https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] [FB-Tracker] Created: (CORE-5507) Wrong value of the new field at the old records, created before that new field was added.

2017-03-24 Thread Molnár Attila
Hi!

I expect form the FB engine to return the default value at INSERT/UPDATE 
TIME, not the current default value. So default values should be kept in 
the rdb$format table, because versioning is needed.

On 2017.03.24. 8:18, Dmitry Yemanov wrote:
> 24.03.2017 09:33, Vlad Khorsun wrote:
>>> Firebird is known to upgrade the record format while reading. "Upgrade"
>>> here means using the latest (aka current) format. The current format is
>>> the one that can be seen in RDB$RELATION_FIELDS. So one may expect that
>>> the default value to be used is also the latest one, that's stored in
>>> RDB$ tables. IIRC, this is how FB 2.5 works.
>> Exactly. When i read the table and know that some field should have default
>> value, i expect to see this value at system catalog.
> It depends on the definition. In fact, adding a NOT NULL DEFAULT X
> column means two things: (1) DEFAULT X will be used implicitly for
> inserts or explicitly for the DEFAULT keyword and (2) X will be used to
> replace the missing values. For (1), everything works as expected. For
> (2), your expectation relies on the fact that X is substituted while
> reading. But this is just an implementation detail. The contract is
> "replacing missing values", whatever it could mean. Someone else does
> not know such internals and expects this "replacement" happening by some
> voodoo magic during ALTER ;-)
>
> Also, lets consider this:
>
> SQL> create table t (col1 int);
> SQL> insert into t values (1);
> SQL> commit;
> SQL> alter table t add col2 int default 123 not null;
> SQL> select * from t;
>
>   COL1 COL2
>  
>  1  123
>
> SQL> alter table t alter col2 drop default;
> SQL> select * from t;
>
>   COL1 COL2
>  
>  1  123
>
> There's no default at the end, but COL2 is still returning 123. This
> matches the "update" expectation, not the "convert missing to the
> current default" one.
>
> Yesterday I considered this issue being a bug, but now I'm not that
> sure. It may look obvious for some users but counter-intuitive for the
> others.
>
> Anyone outside the development team cares to comment? :-)
>
>
> Dmitry
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> Firebird-Devel mailing list, web interface at 
> https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Virtual metadata

2017-01-18 Thread Molnár Attila
Don't forget about SAVEPOINT handling! It can make things very messy in 
this case (BUT expected to work as part of the transaction logic from my 
point of view).
Lock should be applied in case of altering, parallel altering should be 
not allowed. (same as update a record, instant lock conflict error at 
command execution time, not commit time (even when different values are 
updated))

On 2017.01.18. 11:09, Dmitry Yemanov wrote:
> 18.01.2017 12:38, Alex Peshkoff wrote:
>> Currently with dfw we do have a lot of DDL errors raised at commit time
>> i.e. it's not a regression.
> True, but only because the actual work is performed during commit. If we
> claim that DDL changes are applied immediately, but error is thrown at
> commit, this looks weird. Especially if we find a way to allow mixed DDL
> and DML - imagine ALTER TABLE and subsequent UPDATE both executing OK
> but failing at commit because of the metadata conflict.
>
>   > But don't forget that under
>   > normal circumstances such conflicts will be very rare.
>
> I would seriously question the need to allow concurrent DDL against the
> same objects. This is simply not the way people work with the relational
> databases. I'd rather lock the metadata being changed at the DDL time
> and until commit.
>
>
> Dmitry
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> Firebird-Devel mailing list, web interface at 
> https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] FB2.5.6 - '' = ' ' is evaluated as true

2016-12-07 Thread Molnár Attila
Yes, this is correct. SQL specification defines this behaviour.


On 2016.12.07. 14:24, Slavomir Skopalik wrote:
> Hi,
>
> is it correct that empty string '' in comparison with one space string '
> ' is evaluated as true?
>
> SELECT * FROM rdb$database WHERE ''=' '
>
> FB 2.5.6, database dialect 1
>
> Slavek
>
>


--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] RFC: Timeouts

2016-08-18 Thread Molnár Attila
This is a GREAT idea! +1

And you might define the timeout in the CREATE/ALTER command (no need 
for config).

On 2016.08.18. 12:04, liviuslivius wrote:
> Kiling statement or transaction is not good as a general solution
> It must be customized for situations.
>
> I suppose better feature will be "timeout messaging" - something like
> TRIGGER ON STATEMENT_TIMEOUT
> TRIGGER ON TRANSACTION_TIMEOUT
>
> and inside it we have access to MON$ tables and we can cancel statement, 
> transaction if we need.


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] RFC: Timeouts

2016-08-18 Thread Molnár Attila
Hi!

I think timeout should depend on these independent factors :
- transaction parameters : RORC = false else true
- first fetch : not possible at the timeout moment = true else false
- average fetch time (start to measure after the first fetch) : very 
high (config) = true else fales

and optionally (config)
- plan : has NATURAL on some table (config) = true else false


If at least one of the factor is true then cancel the statement, else 
let the statement live.


On 2016.08.18. 9:37, liviuslivius wrote:
>
> W dniu 2016-08-18 09:26:22 użytkownik Vlad Khorsun  
> napisał:
>> 18.08.2016 10:08, liviuslivius пишет:
>>> Hi Vlad,
>>>
> I.e interactive Delphi application that fetch only really shown records 
> will get error
> when user press "Down" key,
  If user fetch one record per hour - yes, such application should be 
 better rewritten
>>> Is this query in different state that can be distinguished from "running" 
>>> queries? I see that yes.
>>> And should be possibility to exclude it from this feature or make 2 
>>> different settings for that.
>> Could you show good reasons to do it ? Real use case also welcome.
>>
>> Regards,
>> Vlad
>>
>> --
>> Firebird-Devel mailing list, web interface at 
>> https://lists.sourceforge.net/lists/listinfo/firebird-devel
>>
> Query timeout is good for queries that consume CPU resources and not finished 
> in limited time.
> But queries that feth e.g. 100 records for user and wait to fetch rest do not 
> consume much resources
> and user can fetch (rest or next portion) e.g. after 10 minutes.
> And that queries are not problem for DBA. Yes, such queries consume some 
> resources but not extensivly.
> But feature like timeout is for queries that can utilize all server resources.
>
> regards,
> Karol Bieniaszewski
>
>
>
> --
> Firebird-Devel mailing list, web interface at 
> https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-11 Thread Molnár Attila

  
  


On 2016.04.09. 20:25, Ann Harrison
  wrote:


  

  On Fri, Apr 8, 2016 at 5:54 AM,
Molnár Attila <amol...@mve.hu>
wrote:

  
  
  
Optimizations
    - IS NOT NULL should use index. It is equivalent
with >= min_value or <= max_value based on index
direction
  



 Histograms and clustered indexes (if they're being
  considered) could help here to detect cases where IS NOT
  NULL returns a small subset of the records in a table.  In
  general, searches that touch more than half the records in
  a table are more efficient when made in storage (natural)
  order rather than through an index.  Remember that
  Firebird stored data and indexes separately, so setting up
  an indexed retrieval that will touch every page in a table
  is just overhead compared with straight-forwardly reading
  every page.
  

  

I ment min_value and max_value not base on table data, but on column
data type.

  

  



       - condition
pre-evaluation and reduction. e.g.: WHERE 1 = 2 AND
field = :param is always FALSE. Evaluation does not
needed for all records, can decide at prepare time
whether the result is an empty result set or an
unfiltered result set.
  



When InterBase was created, there was a lot of academic
  work on optimizing corner cases, with the result that
  academic databases tended to spend more time optimizing
  than retrieving.  We made the deliberate choice not to
  spend optimizer time saving idiots from themselves. 
  Thirty years later, maybe we'd choose differently. 
  However, lots of programs depend on tricks like +0 and
  concatenating with an empty string to coerce unnatural but
  effective plans.  I'd worry about the damage done to those
  cases.
  

  

This optimization can only be done if constant conditions are in the
SQL, so +0 like tricks would not be affected.

  

  
 


       - use index in
"NATURAL" mode when column in a conditional appears in a
multi column index, but not in the first place. You may
reduce number of database page visits in this way :
index page can hold more effective record data because
it's narrower than the table data page record (also in
worst case it could be worse than NATURAL because ot the
mixed index and table data page read, but I think
overall it could worth it, especially in big tables.
measurements needed)
  



I not sure what you mean by "NATURAL" index mode -
  "natural" usually means reading the data pages in storage
  order without any index. If you mean reading across the
  leaf level of the index to find matches in the second and
  subsequent keys in an index, you have no idea how hard
  that would be.  Firebird index keys are mashed up values
  created so they compare bytewise in the desired order. 
  When using an index, Firebird hasn't a clue where the
  boundaries fall between columns in multi-column index. 
  It's just bytes.   The format makes indexes dense and
  comparisons quick.   Changing the key format to support
  partial matches on second and third columns seems like a
  bad idea, given that there's very little difference
  between having an index on each column and a multi-column
  index.  Remember that Firebird uses multiple indexes on a
  single table. 

  

  

I ment the second case (reading across the leaf level). It's hard
and costly (or even possible) to deconstruct the index format to get
the original column values? The question is reading all leaf
deconstruct and filter is faster than reading all records natural an
filter.

  

  


 

       - SELECT
DISTINCT  FROM table is slow
(natural scan on all records) and SELECT  FROM table GROUP BY  is

Re: [Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-08 Thread Molnár Attila
Hi Dmitry!

Hope never dies. I prefer optimizations over the other new features. ;)

Thanks for the reply.


On 2016.04.08. 12:30, Dmitry Yemanov wrote:
> 08.04.2016 12:54, Molnár Attila wrote:
>> Here is my list.
> List of v4 features is already composed. We may add some more
> improvements here and there, but no promises about them.
>
>
> Dmitry
>
>
> --
> Firebird-Devel mailing list, web interface at 
> https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Feature request & discussion for V4 (same as for V3)

2016-04-08 Thread Molnár Attila

  
  
Hi *!

Here is my list.

DOMAIN OF COLUMN .
    - should work like TYPE OF COLUMN, but CHECK and NOT NULL
constraints are checked when value assigned to this variable
    - gain : performance (allows early check, error raised before
the actual DML statement run)

SIZE OF riable name>,
  SCALE OF 
    - SIZE OF : returns max CHAR/VARCHAR length or NUMERIC
precision, SCALE OF : return scale of NUMERIC
    - gain : by defining a variable with DOMAIN or TYPE OF (or
DOMAIN OF) the declaration is dynamic. But the in the PSQL body
there is no dynamic access to this information, you still need hard
code these values. With this you can avoid hard coded constants.

ARRAY TYPE (only in PSQL)
    - static and dynamic, one and multi dimensional
    - gain : performance (no need for table write storing temporary
data), shorter and easy to understand PSQL code (UDF parameters with
array types?)

RECORD TYPE (only in PSQL)
    - gain : shorter and easy to understand PSQL code (UDF
parameters with record types?)

TRY-FINALLY
    - gain : code reduction (Now with AUTONOMUS TRANSACTION it make
sense to have try-finally structure. Currently I have the same code
in the successfull run path and also in the WHEN ANY path)

TRIGGER : NEW/OLD values accessible by column name, and column
  number (PSQL)
    - e.g.: NEW['id'], NEW[0]
    - gain : code reduction, dynamic code (don't have to alter the
trigger ICO the table structure altered)

Optimizations
    - CORE-2589
      - CORE-2589
    - CORE-4266 (please rename the ticket : "NULLS
FIRST/LAST should not cause index loss")
    - IS NOT NULL should use index. It is equivalent with >=
min_value or <= max_value based on index direction
    - condition pre-evaluation and reduction. e.g.: WHERE 1 = 2 AND
field = :param is always FALSE. Evaluation does not needed for all
records, can decide at prepare time whether the result is an empty
resultset or an unfiltered resultset.
    - use index in "NATURAL" mode when column in a conditional
appears in a multi column index, but not in the first place. You may
reduce number of database page visits in this way : index page can
hold more effective record data beacouse it's narrower than the
table data page record (also in worst case it could be worse than
NATURAL beacose ot the mixed index and table data page read, but I
think overall it could worth it, especially in big tables.
measurements needed)
    - SELECT DISTINCT  FROM table is slow
(natural scan on all records) and SELECT  FROM
table GROUP BY  is also slow (worse! : index
scan on all records). I think in this case it's not neccessary to
read all the records in the table, it should be enough to read #of
distinct  values from table. (currently you
have to keep a separate table with this information because you
can't access to this information fast)



    Thank You!
-- 
Molnár Attila
szoftverfejlesztő

Libra Szoftver Zrt.
1113 Budapest, Karolina út 65.
Tel.: +36 1 255 3939 
Fax: +36 1 209 1477
http://www.libraszoftver.hu



Tisztelt Ügyfelünk!

Tájékoztatjuk, hogy 2015. október 14-től megújult a Libra Groupware, a Libra Szoftver Zrt. bejelentő rendszere.
A rendszerhez, korábban már regisztrált ügyfeleink új jelszót a https://libragroupware.mve.hu/lgw/ link használatával kérhetnek.
Új regisztráció a regisztra...@mve.hu címre írt levélben igényelhető.
  


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Proposal of new feature: Event triggers

2015-02-26 Thread Molnár Attila

  
  
The goal is to send notification from one connection to other
connection(s) without roundrip to client using current event
mechanism.

On 2015.02.26. 15:56, James Starkey
  wrote:

Could you describe what you are actually trying to do
  rather than how a possible solution might work?  It's a lot easier
  to work from an actual problem than to reverse engineer from a
  murky proposal.
  

On Thursday, February 26, 2015, Carlos H. Cantu lis...@warmboot.com.br
wrote:
I would like
  to start a discussion to propose the implementation of
  "Event triggers" (for the lack of a better name for now).
  
  Those special triggers would be associated to an EVENT, not to
  a
  TABLE. When the associated event is fired in any of the active
  attachments, all the active attachments "interested" in that
  event
  would run the trigger.
  
  My personal example of use for such feature, would be to force
  the
  update of global context variables in all active attachments,
  when
  the EVENT is fired.
  
  Of course there are questions to be discussed, for example:
  the
  trigger would ran in what transaction context? Should it be
  forced to
  be ReadOnly (making things simpler, but much more limited)?
  
  This would be for FB 3.x or 4. If you guys thinks this is not
  the
  right moment to talk about it, I can just open a ticket and
  leave it
  in the tracker waiting for the right time.
  
  []s
  Carlos
  http://www.firebirdnews.org
  FireBase - http://www.FireBase.com.br
  
  
--
  Dive into the World of Parallel Programming The Go Parallel
  Website, sponsored
  by Intel and developed in partnership with Slashdot Media, is
  your hub for all
  things parallel software development, from weekly thought
  leadership blogs to
  news, videos, case studies, tutorials and more. Take a look
  and join the
  conversation now. http://goparallel.sourceforge.net/
  Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel

  
  
  
  -- 
  Jim Starkey
  
  
  
  --
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/
  
  
  
  Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel



  


--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the 
conversation now. http://goparallel.sourceforge.net/Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Firebird 3 memory manager

2014-09-22 Thread Molnár Attila
Hello all!

Have a look at ScaleMM.

It has great multithreaded performance, AND low fragmentation rate.
https://code.google.com/p/scalemm/

You should consider rewrite it to C++.

On 2014.09.20. 23:57, Jim Starkey wrote:
 Memory management is perhaps the single most performance part of a
 database system (at least once you figure out how never touch the
 disk).  A significant part of the effort on each of my umpteen database
 systems has been spent squeezing cycles out of the memory manager, AVL
 trees for hunk management, minimizing fragmentation, and eliminating
 contention with thread-specific sub-pools.

 That said, the NuoDB guys look a fresh look at the problem and evaluated
 various other open source memory managers.  One, jealloc, was only
 slightly slower than my best but did a significantly better job reducing
 fragmentation for a clear net gain.

 Nikolay's critique is sufficiently well taken that I saw no need to step
 into the fray.  Still, if there's a better mouse trap with an acceptable
 license (jealloc is BSD), why no go for it?  Without doubt, the Firebird
 memory allocator can be incrementally improved.  But unless memory
 management is your life's work, if an acceptable open source memory
 manager can be shown to be significantly better than what's in place,
 adapt it and go on to the next problem.




 On 9/19/2014 6:33 PM, Nikolay Samofatov wrote:
 Hello, All!

 I implemented intermediate versions GC algorithm and tried to run some 
 stress tests on Firebird 3 to
 check if I have broken something.
 The problem is that tests that I created were spending most of their time in 
 memory manager. This is
 not healthy.

 For example, in my tests I saw up to 30 iterations in this loop during 
 memory allocation:

for (hunk = smallHunks; hunk; hunk = hunk-nextHunk)
{
if (length = hunk-spaceRemaining)
{

}
}

 Dear Firebird engineers, why did you replace an algorithm which has 
 O(log(n)) performance with an
 algorithm that has O(n) performance in such a performance-critical part of 
 the engine?

 O(n) performance in certain scenarios was the reason why original memory 
 manager of Interbase was
 replaced in Firebird 1.5.

 I created a small query to demonstrate the effect of O(n) memory manager 
 performance.
 ===
 create table test7 (id integer);

 execute block as
 declare variable i integer = 1;
 begin
  while(i = 5000) do
  begin
insert into test7(id) values(:i);
i = i + 1;
  end
 end;

 commit;

 savepoint X;
 update test7 set id = id;
 delete from test7;
 ===

 Last statement uses 3.5 Gb of memory in small blocks. I forward-ported 
 memory manager from Firebird
 2.5 to Firebird 3 and compared performance of the last statement.
 With Firebird 3 memory manager the query runs for 634 s on my machine. With 
 Firebird 2.5 memory
 manger it completes in 427 seconds.
 I used very small number of rows, so you can run it on a desktop class 
 machine. It is easy to
 imagine a query that is a couple of orders of magnitude larger in ETL 
 applications, and see that
 such queries now become impossible to run (due to at least an order of 
 magnitude estimated slowdown).

 So the performance of the server is now O(N^2) for queries using large 
 amounts of memory, and the
 server hits the wall rather quickly.

From quick review of new memory manager code I can see the following 
 problems:
 1) O(n) performance in small hunk allocation
 2) O(n) performance in large hunk deallocation
 3) Worst case allocation cost in large hunk allocation algorithm is bounded, 
 but much worse than for
 older memory manager
 4) Lots of memory waste in various scenarios, which is a security issue with 
 mild risk.
 5) Not all debug facilities have been preserved.

 Problem 1 and 2 can be relatively inexpensively fixed with existing code 
 base. Are there volunteers?
 Problems 3 and 4 require going back to global best-fit allocation strategy, 
 and thus re-design.

 Alternatively, I may fix Firebird 2.5 memory manager to have better 
 performance in simple cases
 without compromising worst-case performance.
 This is rather easy. B-Tree can be replaced with specially designed array of 
 pointers (see FastMM
 code, for example).
 Dmitry, do you want me to do this?

 My understanding is that Firebird project probably does not care about large 
 installs and security,
 so it will continue using new memory manager as it is marginally faster in 
 simple cases. For Red
 Database version based on Firebird 3 we shall certainly back out this memory 
 manager as it has
 unpredictable performance and introduces new security risks.
 Good thing is that changing or replacing memory manager is very simple task 
 for existing code base.


 Best Regards,
 Nikolay Samofatov


 --
 Slashdot TV.  Video for Nerds.  Stuff 

[Firebird-devel] New Interface

2014-08-11 Thread Molnár Attila

  
  
Hi!

Some note from an outsider view.

#1 : both client  plugin API MUST support C-tyle
API (lowest common dominator)
    - standard data types, zero terminated strings,
pointers/handles, records, stdcall/cdecl functions, NO
objects/interfaces
    - most of the API developers will and can only
use this. (if will be not supported most of the user base will be
lost)
   
#2 : old client API should not be changed at all
    - why the new API if old one is also developed?
    - backward compatibility

#3 : client  plugin OO API 
    - only C++ developers can use is (small part of the whole user
base pie)
    - not just the different Object memory layout is the problem,
but the different memory managers also. Object can be shared via API
just in case of same compiler with same memory optimization
settings AND shared memory manager.
    - why maintain a second one API? Can gain siginficant
performance over C style API? What are the cons?

#4 : new API should be designed from scratch, in mind of :
    - eliminate/expand current limits
    - better multithread scalability
    - make place for future FB developements
    - make place for supporting missing SQL standard features


-- 
Molnár Attila
szoftverfejlesztő
Tel : 372-
E-mail: amol...@mve.hu
 
LIBRA Szoftver zrt.
1113 Bp. Karolina út 65.
Tel: 372-
Fax: 209-1477
Web: www.mve.hu
E-mail: i...@mve.hu
Olvasson ügyfeleinkkel elért közös sikereinkről:
http://www.mve.hu/hu/referenciaink
  


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] New Interface

2014-08-11 Thread Molnár Attila

  
  
Lets see the next scenatios 

#1 : API call creates object and returned to host
    - in this case object has to be readonly (at least buffer like
fields)1
    - host has to support reference counting2 

#2 : API call parameterised by a host created object
    - see #1


1 : can't write because in that case data contained by the object
could be allocated by different memory managers
2 : Must have all mayor language/compilers support ref.count and has
to be the same



With this restrictions what gain do you have with OO/Interface API
over C style API?


On 2014.08.11. 10:02, Dimitry
  Sibiryakov wrote:


 It is not true as well. Memory manager is not a problem as long as object are freed in 
the same module where were created. Reference counting solves this well.


  


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] New Interface

2014-08-11 Thread Molnár Attila

  
  
BTW API and host has different (can't have) class information (class
pointer, VMT, etc..) for the same object, so by definition they not
communicate with the SAME class. The classes could have same name,
and can act the same, but the equality operation would give you
FALSE result. Also inheritance handling definiatily would be problem
(different VMT, and different memory mapping at the two sides,
etc...).

So basicly pure Object based API is off the table.
Interface based might be good, and we ended up with COM.


On 2014.08.11. 10:49, Molnár Attila
  wrote:


  
  Lets see the next scenatios 
  
  #1 : API call creates object and returned to host
      - in this case object has to be readonly (at least buffer like
  fields)1
      - host has to support reference counting2 
  
  #2 : API call parameterised by a host created object
      - see #1
  
  
  1 : can't write because in that case data contained by the object
  could be allocated by different memory managers
  2 : Must have all mayor language/compilers support ref.count and
  has to be the same
  
  
  
  With this restrictions what gain do you have with OO/Interface API
  over C style API?
  
  
  On 2014.08.11. 10:02, Dimitry
Sibiryakov wrote:
  
  
   It is not true as well. Memory manager is not a problem as long as object are freed in 
the same module where were created. Reference counting solves this well.
  
  


  


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] New Interface

2014-08-11 Thread Molnár Attila
So this Interface is not designed and made for clinet component 
developers and communicatin with server but just for Firebird engine 
inside use?
In that case just do what you want. ;)

On 2014.08.11. 11:05, Dimitry Sibiryakov wrote:
   Remember, that new API wasn't designed as a public API. It was supposed to 
 be a new
 interface between Y-valve and providers, nothing more. ISC API in this place 
 would add too
 much overhead or require significant changes in engine code.


--
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] DDL Triggers, how to retrieve type?

2014-05-23 Thread Molnár Attila
E.g. our application still in dialect 1. It would be a huge job to 
switch dialect 3.

On 2014.05.23. 8:41, Martijn Tonies (Upscene Productions) wrote:
 - Any other suggestion?
 Drop dialect 1 support.
Allow dialect 1 to have access to BIGINT fields.
 I don't have to work for this to happen, so I don't really have a say...

 but the question that arises is why?

 What are the reasons to continue to use dialect 1?



 With regards,

 Martijn Tonies
 Upscene Productions
 http://www.upscene.com

 Download Database Workbench for Oracle, MS SQL Server, Sybase SQL
 Anywhere, MySQL, InterBase, NexusDB and Firebird!



 --
 Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
 Instantly run your Selenium tests across 300+ browser/OS combos.
 Get unparalleled scalability from the best Selenium testing platform available
 Simple to use. Nothing to install. Get started now for free.
 http://p.sf.net/sfu/SauceLabs
 Firebird-Devel mailing list, web interface at 
 https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free.
http://p.sf.net/sfu/SauceLabs
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-12 Thread Molnár Attila
e.g.

- Select value from two dynamic source, add them and write back.
- a generic or a very dynamic code (EXECUTE STATEMENT) currenly have to 
store values in VARCHAR variables, because in other case (variable 
defined for every data type) the code would be VERY ugly. And because of 
this VARCHAR conversations must be used heavly. With VARIANT the code 
become cleaner and could avoid conversations.

new context variable access method in PSQL with variant support would be 
great also



On 2014.05.10. 16:43, Adriano dos Santos Fernandes wrote:
 On 10-05-2014 04:03, Molnár Attila wrote:
 *VARIANT data type in PSQL*
 This would be very easy to implement (variables only). What would be the
 use cases?


 Adriano

 --
 Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
 #149; 3 signs your SCM is hindering your productivity
 #149; Requirements for releasing software faster
 #149; Expert tips and advice for migrating your SCM now
 http://p.sf.net/sfu/perforce
 Firebird-Devel mailing list, web interface at 
 https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free.
http://p.sf.net/sfu/SauceLabs
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Molnár Attila

On 2014.05.10. 9:52, Dimitry Sibiryakov wrote:
 10.05.2014 9:03, Molnár Attila wrote:
 *VARIANT data type in PSQL*
   - gain : a little memory and/or CPU overhead but much cleaner code. 
 Also
 rdb$get/set_context value colud be variant.
 Welcome to the hell of unpredictable type conversion.

 *Optimization II.
 *- temporal indexing of materialization : e.g. when ORDER/GROUP BY has 
 no index then
 currently the whole resultset is materialized, and the sorting moves the 
 whole row each
 time. Instead of this it should create a temporal index on the order/group 
 columns then
 fetching on the temporal index. In this way much less writes needed. This 
 shold be applied
 after a treshold : common sense sais after index size/row size rate is 
 smaller than 0.5.
 Uncommon sense says that random IO can be much more than two times slower.

OK, this just means smaller treshold rate. But with this some case you 
can gain HUGE performance boost (especially when the resultset can't fit 
into memory and have to write to disk)



--
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
#149; 3 signs your SCM is hindering your productivity
#149; Requirements for releasing software faster
#149; Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-10 Thread Molnár Attila
Derived table usage for this problem is GREAT idea! Why the ... I didn't 
tought that!

Thank you!

On 2014.05.10. 10:26, Dimitry Sibiryakov wrote:
 10.05.2014 10:12, Molnár Attila wrote:
 OK, this just means smaller treshold rate. But with this some case you
 can gain HUGE performance boost (especially when the resultset can't fit
 into memory and have to write to disk)
 Big result set is a sign of wrong application design. And in any case you 
 can use
 derived tables to gain the same effect right now without hacking the engine.



--
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
#149; 3 signs your SCM is hindering your productivity
#149; Requirements for releasing software faster
#149; Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Molnár Attila
Hello Sean!

On 2014.05.01. 21:24, Leyne, Sean wrote:
 TRIGGER : NEW/OLD values accessible by column name, and column number
 (PSQL)
  - e.g.: NEW['id'], NEW[0]
  - gain : code reduction, dynamic code (don't have to alter the trigger 
 ICO
 the table structure altered)
 1- What advantage does NEW['id'] have over the current syntax new.id???
I could iterate trough columns/values, currently I can't.

 2- column numbers reference is extremely dangerous, the column position can 
 be changed using simple DDL.  So, all Trigger references would be invalidated!
I do not want hardcode index numbers it's about iteration. But you're 
right, support only access by name is quite enough.


 Optimization I.
  - VOLATILE (default) / DETERMINISTIC flag for UDF and FUNCTION (maybe
 lifecycle for deterministic : statement, transaction, connection)
  - gain : preformance (can cache DETERMINISTIC UDF/FUNCTION results)
 Actually, I think that with the exception of a random number generator, the 
 default for UDF should be DETERMINISTIC (Over 99% of UDFs are not volatile).
User shold define deterministic flag. Only slow UDF/FUNCTION shold be 
flagged this or else the cache size and lookup time would increase too big.


  - SELECT DISTINCT index fields FROM table is slow (natural scan on all
 records) and SELECT index fields FROM table GROUP BY index fields is
 also slow (worse! : index scan on all records).
  - currently you have to keep a separate table with this information 
 because
 you can't access to this information fast
  - solution 1 : use index in this case;
 Using an index may not help:

 1- an index is stored without regard to physical disk location, so using it 
 will create a huge amount of random disk IO.  Whereas a NATURAL scan follows 
 the table.

 2- In an version based database like Firebird each row will need to be read 
 to confirm the current value of the target field.


 Index usage optimization II.
  IS NOT NULL should use index. It is equivalent with = min_value or =
 max_value based on index direction
 I don't think this is possible.

 With an MVCC it is possible for all rows to have both a NULL and NOT NULL 
 values stored in the field index, so reading each rows is required.  But as 
 noted in #1 above, reading by index can lead to significant disk 
 IO/degradation
I don't understand this. An equivalent statement can use index and 
produce the same result and this why can't? What I'm missing?

 Index usage optimization III.
  - condition pre-evaluation and reduction. e.g.: WHERE 1 = 2 AND field =
 :param is always FALSE. Evaluation does not needed for all records, can
 decide at prepare time whether the result is an empty resultset or an
 unfiltered resultset.
 I agree that the engine needs to add support for 'execution time' 
 optimization.  To allow for conditions to be bypassed (as your example) as 
 well as eliminate unneeded JOIN in SELECT and Views.


 PSQL exception handling : new variables aside GDSCODE and SQLCODE
  - ERRORMESSAGE : form fbclient
  - GDSCODE_ORI, SQLCODE_ORI, ERRORMESSAGE_ORI : execute statement
 errors on external database hides the original error code, you should access
 them with these
  - gain : better logging and error tracking and user feedback
 User feedback?
Based on error code we have our extended error messages and suggestions 
to deal with it, but in this case we can't properly do this.
 rename objects
  it's a nightmare to rename soething with many-many dependencies


 Save blob as file to disk at server side
  - gain : easy UDF deploy and update (and many other thing)
  - it's also a seurity risk, so some options needed in firebird.conf
 This would violate data integrity requirements.  If I save data to a Blob, I 
 expect the data to be available -- saving externally would allow for external 
 data to be deleted without the database/me knowing.

 I believe that there are UDFs available for this already, no?
A very old and unsuppoted udf. In my case I do not expect to read back 
from file to DB just one way.


 select from execute block
  - gain : avoid stored procedure creation. With select from select it's 
 easy to
 generate code, but can't do that with execute block. (select from procedure
 is allowed, but execute block can't replace the procedure in this case)
 EXECUTE BLOCK already supports returning data, what are you looking for?

Once I was needed the following logic (via code generation result). I 
bypassed it using stored proc.

SELECT
FROM
 (EXECUTE BLOCK
 AS
 BEGIN
   SELECT
   FROM (
   EXECUTE BLOCK
   AS
   BEGIN
   END)
 )


 --
 Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
 Instantly run your Selenium tests across 300+ browser/OS combos.  Get
 unparalleled scalability from the best Selenium testing platform available.
 Simple 

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Molnár Attila
Hello Dmitry!

On 2014.05.02. 8:34, Dmitry Yemanov wrote:
 30.04.2014 13:50, Molnár Attila wrote:
 *SIZE OF CHAR/VARCHAR domain or variable name, SCALE OF NUMERIC
 domain or variable name*
   - SIZE OF : returns max CHAR/VARCHAR length or NUMERIC precision,
 SCALE OF : return scale of NUMERIC
   - gain : by defining a variable with DOMAIN or TYPE OF (or DOMAIN
 OF) the declaration is dynamic. But the in the PSQL body there is no
 dynamic access to this information, you still need hard code these
 values. With this you can avoid hard coded constants.
 Why wouldn't you want to create domains and use TYPE OF / DOMAIN OF?
I'm createing domains, and using TYPE OF. But as I write it's not 
enough. This is just for variable declaration but I need a pair in PSQL 
body. Maybe this example would help to understand.

EXECUTE BLOCK
AS
DECLARE VARIABLE tmp TYPE OF COLUMN rdb$database.rdb$character_set_name;
DECLARE VARIABLE i INTEGER;
BEGIN
   i = 1;
   WHILE (i = 31)--I want avoid hard coded 31 with SIZE OF tmp OR 
SIZE OF COLUMN rdb$database.rdb$character_set_name
   DO BEGIN
 --some logic here
 i = i + 1;
   END
END


 *increase or remove completly the context limit* (not the context
 variable but the table/view/sp stream, current limit is 255)
   - gain : performance (bigger EXECUTE BLOCK with more DML, e.g.: a
 lot of INSERT in a single SQL command)
 Multi-value (multi-row) INSERT would serve the same goal without
 affecting the context limit. Batch DML API would also allow remove the
 need for EXECUTE BLOCK to execute batch INSERTs.

 Why else would you need a bigger EXECUTE BLOCK?
I won't if these two are available. I wasn't aware of them.


 *TRIGGER : NEW/OLD values accessible by column name, and column number
 (PSQL)*
   - e.g.: NEW['id'], NEW[0]
   - gain : code reduction, dynamic code (don't have to alter the
 trigger ICO the table structure altered)
 Dynamic code is not going to work reliably, unless you just iterate
 fields in the loop. In this case, I see it just as a feature misuse,
 some other solution may be required instead.
It's about iteration. Define misuse.

 *Optimization**I.*
   - VOLATILE (default) / DETERMINISTIC flag for UDF and FUNCTION
 (maybe lifecycle for deterministic : statement, transaction, connection)
   - gain : preformance (can cache DETERMINISTIC UDF/FUNCTION results)
 Partically done in FB3, to be improved later.

 *Index usage optimization III.*
   - condition pre-evaluation and reduction. e.g.: WHERE 1 = 2 AND
 field = :param is always FALSE. Evaluation does not needed for all
 records, can decide at prepare time whether the result is an empty
 resultset or an unfiltered resultset.
 http://tracker.firebirdsql.org/browse/CORE-1287

 *PSQL exception handling : new variables aside GDSCODE and SQLCODE*
   - ERRORMESSAGE : form fbclient
 Also exists in the tracker.

   - GDSCODE_ORI, SQLCODE_ORI, ERRORMESSAGE_ORI : execute statement
 errors on external database hides the original error code, you should
 access them with these
 Maybe an ability to iterate all status vection arguments would be a
 better (and more generic) solution?
Status vectos is at API level if I'n not mistaken. What about PSQL level?

 *performance info*
   - ms is not so accurate and it seems also affecte by the thread
 time slice size. Need a more accurate measure : maybe gettickcount?
 What performance info do you mean here? CPU time reported by ISQL?
anything is a good candidate which has better scaleing than the curreny one.

 *Foreign key column or referenced column can be a const value*
   - e.g. : ALTER TABLE table ADD CONSTRAINT FKNAME FOREIGN KEY
 (field1, field2) REFERENCES referenced_table ('CONST', field3);
   - gain : data integrity without useless data storage
 And if you change master.field1 to become 'NONCONST' in some record your
 database gets logically corrupted due to constraint violation?

 Also, the prefix compression nearly completely removes your 'CONST' from
 the FK index anyway, so there's no useless data storage.

 *select from execute block*
   - gain : avoid stored procedure creation. With select from select
 it's easy to generate code, but can't do that with execute block.
 (select from procedure is allowed, but execute block can't replace the
 procedure in this case)
 The only syntax I could live with (still not being happy at all!) is
 using CTE, something like:

 WITH EXECUTE BLOCK ...
 AS ...
 SELECT ...

 IMHO, it's terribly ugly, but other options are even uglier.
I could live with that kind of ugly. This is not priority for me.


 Dmitry


 --
 Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
 Instantly run your Selenium tests across 300+ browser/OS combos.  Get
 unparalleled scalability from the best Selenium testing platform available.
 Simple to use. Nothing to install. Get started now for free.
 http://p.sf.net

Re: [Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-05-05 Thread Molnár Attila

On 2014.05.05. 9:27, Dmitry Yemanov wrote:
 05.05.2014 11:05, Molnár Attila wrote:
 I'm createing domains, and using TYPE OF. But as I write it's not
 enough. This is just for variable declaration but I need a pair in PSQL
 body. Maybe this example would help to understand.

 EXECUTE BLOCK
 AS
 DECLARE VARIABLE tmp TYPE OF COLUMN rdb$database.rdb$character_set_name;
 DECLARE VARIABLE i INTEGER;
 BEGIN
  i = 1;
  WHILE (i = 31)--I want avoid hard coded 31 with SIZE OF tmp OR
 SIZE OF COLUMN rdb$database.rdb$character_set_name
  DO BEGIN
--some logic here
i = i + 1;
  END
 END
 Now I see your point, thanks.

 *increase or remove completly the context limit* (not the context
 variable but the table/view/sp stream, current limit is 255)
 - gain : performance (bigger EXECUTE BLOCK with more DML, e.g.: a
 lot of INSERT in a single SQL command)
 Multi-value (multi-row) INSERT would serve the same goal without
 affecting the context limit. Batch DML API would also allow remove the
 need for EXECUTE BLOCK to execute batch INSERTs.

 Why else would you need a bigger EXECUTE BLOCK?
   
 I won't if these two are available. I wasn't aware of them.
 They are not available, but they're the proper solution to the problem
 and should be considered instead.

 *TRIGGER : NEW/OLD values accessible by column name, and column number
 (PSQL)*
 - e.g.: NEW['id'], NEW[0]
 - gain : code reduction, dynamic code (don't have to alter the
 trigger ICO the table structure altered)
 Dynamic code is not going to work reliably, unless you just iterate
 fields in the loop. In this case, I see it just as a feature misuse,
 some other solution may be required instead.
 It's about iteration. Define misuse.
 Iterator and ordinal number are two different things, even if they can
 be implemented the same way. I don't want ordinal numbers for iterating
 *only*, as people will start using them for other purposes finally
 shooting themselves in the foot.
Access by name only is good for me.
(BTW I just realised that in this case it would great to have a VARIANT 
data type)

 - GDSCODE_ORI, SQLCODE_ORI, ERRORMESSAGE_ORI : execute statement
 errors on external database hides the original error code, you should
 access them with these
   
 Maybe an ability to iterate all status vection arguments would be a
 better (and more generic) solution?
   
 Status vectos is at API level if I'n not mistaken. What about PSQL level?
 This is exactly my point - PSQL access to the status vector. E.g.
 besides GDSCODE, make available also GDSCODE[n] or whatever iteration
 solution we choose.

 *performance info*
 - ms is not so accurate and it seems also affecte by the thread
 time slice size. Need a more accurate measure : maybe gettickcount?
 What performance info do you mean here? CPU time reported by ISQL?
 anything is a good candidate which has better scaleing than the curreny one.
 My question was what bad one / current one do you mean? AFAIR, the
 only one we have now is what ISQL reports (via perf API). Is this what
 you're talking about?
Very fast stored procedures run time can't be measuerd standalone, have 
to build an EXECUTE BLOCK and repeate many times then divide the run 
time to get average run time. But in this case there are also side 
effects (like HW cache, branch prediction, server side statement cache, 
data cache, etc...) and overhead which influences the measurement.

Currenty on my XP it reports 0ms or 16ms runtime, but nothing in 
between. (this number suggest me the thread time slice size percision of 
the current one)

 Dmitry


 --
 Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
 #149; 3 signs your SCM is hindering your productivity
 #149; Requirements for releasing software faster
 #149; Expert tips and advice for migrating your SCM now
 http://p.sf.net/sfu/perforce
 Firebird-Devel mailing list, web interface at 
 https://lists.sourceforge.net/lists/listinfo/firebird-devel


--
Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
#149; 3 signs your SCM is hindering your productivity
#149; Requirements for releasing software faster
#149; Expert tips and advice for migrating your SCM now
http://p.sf.net/sfu/perforce
Firebird-Devel mailing list, web interface at 
https://lists.sourceforge.net/lists/listinfo/firebird-devel


[Firebird-devel] Feature request discussion (Reply to Planning the post v3 development)

2014-04-30 Thread Molnár Attila

  
  
Hello Dimitry!

Here is my wish list.

DOMAIN OF COLUMN table or view.column
    - should work like TYPE OF COLUMN, but CHECK and NOT NULL
constraints are checked when value assigned to this variable
    - gain : performance (allows early check, error raised before
the actual DML statement run)

SIZE OF CHAR/VARCHAR domain or variable name,
  SCALE OF NUMERIC domain or variable name
    - SIZE OF : returns max CHAR/VARCHAR length or NUMERIC
precision, SCALE OF : return scale of NUMERIC
    - gain : by defining a variable with DOMAIN or TYPE OF (or
DOMAIN OF) the declaration is dynamic. But the in the PSQL body
there is no dynamic access to this information, you still need hard
code these values. With this you can avoid hard coded constants.

ARRAY TYPE (only in PSQL)
    - static and dynamic, one and multi dimensional
    - gain : performance (no need for table write storing temporary
data), shorter and easy to understand PSQL code (UDF parameters with
array types?)

RECORD TYPE (only in PSQL)
    - gain : shorter and easy to understand PSQL code (UDF
parameters with record types?)

increase or remove completly the context limit (not the
context variable but the table/view/sp stream, current limit is 255)
    - gain : performance (bigger EXECUTE BLOCK with more DML, e.g.:
a lot of INSERT in a single SQL command)

TRY-FINALLY
    - gain : code reduction (Now with AUTONOMUS TRANSACTION it make
sense to have try-finally structure. Currently I have the same code
in the successfull run path and also in the WHEN ANY path)

TRIGGER : NEW/OLD values accessible by column name, and column
  number (PSQL)
    - e.g.: NEW['id'], NEW[0]
    - gain : code reduction, dynamic code (don't have to alter the
trigger ICO the table structure altered)

Optimization I.
    - VOLATILE (default) / DETERMINISTIC flag for UDF and FUNCTION
(maybe lifecycle for deterministic : statement, transaction,
connection)
    - gain : preformance (can cache DETERMINISTIC UDF/FUNCTION
results)

Optimization II.
    - SELECT DISTINCT index fields FROM table is slow
(natural scan on all records) and SELECT index fields FROM
table GROUP BY index fields is also slow (worse! : index
scan on all records). 
    - currently you have to keep a separate table with this
information because you can't access to this information fast
    - solution 1 : use index in this case; solution 2 : a system
table with this information; soultion 3 : select from index (not so
SQL standard)
    - gain : redundancy reduction and/or speed

Index usage optimization I.
    CORE-2589

Index usage optimization II.
    IS NOT NULL should use index. It is equivalent with =
min_value or = max_value based on index direction

Index usage optimization III.
    - condition pre-evaluation and reduction. e.g.: WHERE 1 = 2 AND
field = :param is always FALSE. Evaluation does not needed for all
records, can decide at prepare time whether the result is an empty
resultset or an unfiltered resultset.

More details for exception 335544321
    - Arithmetic exception, numeric overflow, or string truncation :
table  fieldname (procedure  variable name) in exception
message

PSQL exception handling : new variables aside GDSCODE and SQLCODE
    - ERRORMESSAGE : form fbclient
    - GDSCODE_ORI, SQLCODE_ORI, ERRORMESSAGE_ORI : execute statement
errors on external database hides the original error code, you
should access them with these
    - gain : better logging and error tracking and user feedback

longer object names 
    CORE-749

rename objects
    it's a nightmare to rename soething with many-many dependencies

performance info
    - ms is not so accurate and it seems also affecte by the thread
time slice size. Need a more accurate measure : maybe
gettickcount?

Foreign key column or referenced column can be a const value
    - e.g. : ALTER TABLE table ADD CONSTRAINT FKNAME FOREIGN
KEY (field1, field2) REFERENCES referenced_table ('CONST',
field3);
    - gain : data integrity without useless data storage

Save blob as file to disk at server side
    - gain : easy UDF deploy and update (and many other thing)
    - it's also a seurity risk, so some options needed in
firebird.conf
    
select from execute block
    - gain : avoid stored procedure creation. With select from
select it's easy to generate code, but can't do that with execute
block. (select from procedure is allowed, but execute block can't
replace the procedure in this case)



Thank You!

-- 
Molnár Attila
szoftverfejlesztő
Tel : 372-
E