Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-17 Thread Doug Chamberlin
It seems Oracle has a related issue with their SCN values: http://www.infoworld.com/d/security/revealed-fundamental-oracle-flaw-184163?source=IFWNLE_nlt_daily_2012-01-17 -- Keep Your Developer Skills Current with

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-16 Thread u.maso...@libero.it
Hi all, i've read many comments on this argument, but seems that ideas to solve this problem (if it is a true real problem in near future) are not many. I remember that in 8086 era, when addressable memory became cheaper and bigger than dimension reserved in opcode space (24 bits IIRC),

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-16 Thread W O
Excellent Ann, you had explained it very clearly (as usual). Greetings. Walter. On Mon, Jan 16, 2012 at 12:51 PM, Ann Harrison aharri...@nuodb.com wrote: On Mon, Jan 16, 2012 at 8:24 AM, u.maso...@libero.it u.maso...@libero.it wrote: I remember that in 8086 era, when addressable memory

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-15 Thread Alex Peshkoff
On 01/13/12 23:57, Yi Lu wrote: Hi All, When we are trying to change structure “header page” in file ods.h (For example hdr_next_transaction from SLONG to SINT64) We have the following error C:\Program Files\Firebird\Firebird_2_5\bingbak -c -r -user sysdba -password masterkey -service

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-13 Thread Yi Lu
Hi All, When we are trying to change structure “header page” in file ods.h (For example hdr_next_transaction from SLONG to SINT64) We have the following error C:\Program Files\Firebird\Firebird_2_5\bingbak -c -r -user sysdba -password masterkey -service localhost:service_mgr C:\Program

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-04 Thread Vlad Khorsun
How do I debug if firebird server running with code changes that we made? Are you asking how to debug applications ? Using debugger ;) I am running fbserver.exe with -a -p 3050 -database localhost:..\myDatabase.fdb. Firebird tray icon is showing me that nothing has been attached and

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-03 Thread Yi Lu
We have read that negative transaction IDs are used to indicate an invalid transaction. Thus, we have decided to continue with the signed approach to prevent logic problems. -- View this message in context:

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject

2012-01-03 Thread Leyne, Sean
Jesus Single server without downtime is a myth anyway. The problem is not downtime is how much downtime. Backup and restore is so much downtime. If that is the case, how much downtime is acceptable? There are a couple of possible solutions which would reduce the downtime; - a new

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-03 Thread Dimitry Sibiryakov
03.01.2012 18:51, Leyne, Sean wrote: - a data port utility which would allow for data to be ported from a live database to a new database while live is active but would need a finalization step where the live database is shutdown to apply the final data changes and add FK constraints.

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject

2012-01-03 Thread Ann Harrison
Sean, The problem is not downtime is how much downtime. Backup and restore is so much downtime. There are a couple of possible solutions which would reduce the downtime; - a new backup/restore tool which would use multiple readers/writers to minimize execution time, Here we're talking

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-03 Thread Ann Harrison
Dimitry, - a data port utility which would allow for data to be ported from a live database to a new database while live is active but would need a finalization step where the live database is shutdown to apply the final data changes and add FK constraints.   And exactly this utility is

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject - Email found in subject

2012-01-03 Thread Leyne, Sean
Ann, - a new backup/restore tool which would use multiple readers/writers to minimize execution time, Here we're talking about a logical backup that can be used to restart transaction numbers. Record numbers are based loosely on record storage location. Since a logical backup/restore

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-03 Thread Ann Harrison
Woody, Maybe I'm a little dense, (probably :), but doesn't FB already know what the oldest interesting transaction id is? Why couldn't transaction numbers be allowed to wrap back around up to that point? As long as transactions are committed at some point, the oldest transaction would move

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-03 Thread Kjell Rilbe
Den 2012-01-03 20:14 skrev Woody såhär: Maybe I'm a little dense, (probably :), but doesn't FB already know what the oldest interesting transaction id is? Why couldn't transaction numbers be allowed to wrap back around up to that point? As long as transactions are committed at some point, the

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject

2012-01-03 Thread Thomas Steinmaurer
The problem is not downtime is how much downtime. Backup and restore is so much downtime. There are a couple of possible solutions which would reduce the downtime; - a new backup/restore tool which would use multiple readers/writers to minimize execution time, Here we're talking about a

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject

2012-01-03 Thread Dimitry Sibiryakov
03.01.2012 20:37, Leyne, Sean wrote: As for the FKs, I see the tool as being a some which needs to have the maximum performance. If you are going to move whole database at once - yes. Fortunately, in most cases it is not necessary. -- SY, SD.

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-03 Thread Kjell Rilbe
Den 2012-01-01 21:31 skrev Kjell Rilbe såhär: 3. Stay at 32 bit id:s but somehow compact all id:s older than OIT(?) into OIT-1. Allow id:s to wrap around. Pros: - No extra disk space. - Probably rather simple code changes for id usage. Cons: - May be difficult to find an effective way to

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-03 Thread Yi Lu
How do I debug if firebird server running with code changes that we made? I am running fbserver.exe with -a -p 3050 -database localhost:..\myDatabase.fdb. Firebird tray icon is showing me that nothing has been attached and nothing is happening outside of the windows message loop. Also, when

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Dimitry Sibiryakov
01.01.2012 23:43, Kjell Rilbe wrote: Den 2012-01-01 23:25 skrev Dimitry Sibiryakov såhär: 01.01.2012 21:31, Kjell Rilbe wrote: So far, these are the suggestions I've seen Hey, how about my suggestion leave everything as is and perform backup-restore cycle on demand?.. Pros: - No

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Dimitry Sibiryakov
02.01.2012 14:46, Jesus Garcia wrote: he problem is not downtime is how much downtime. Backup and restore is so much downtime. Not more downtime that simple restore after complete server destruction by flood or plane crash. -- SY, SD.

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Mark Rotteveel
On 2-1-2012 15:50, Dimitry Sibiryakov wrote: 02.01.2012 14:46, Jesus Garcia wrote: he problem is not downtime is how much downtime. Backup and restore is so much downtime. Not more downtime that simple restore after complete server destruction by flood or plane crash. Could you stop

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Dimitry Sibiryakov
02.01.2012 16:10, Mark Rotteveel wrote: Could you stop with the absurd comparisons? One is normal maintenance and the other is (extreme) disaster recovery which are in no way comparable. But downtime is downtime. Customers don't care about its reason, do they?.. -- SY, SD.

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Lester Caine
Mark Rotteveel wrote: 02.01.2012 14:46, Jesus Garcia wrote: he problem is not downtime is how much downtime. Backup and restore is so much downtime. Not more downtime that simple restore after complete server destruction by flood or plane crash. Could you stop with the

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Mark Rotteveel
On 2-1-2012 16:43, Dimitry Sibiryakov wrote: 02.01.2012 16:10, Mark Rotteveel wrote: Could you stop with the absurd comparisons? One is normal maintenance and the other is (extreme) disaster recovery which are in no way comparable. But downtime is downtime. Customers don't care about its

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Lester Caine
Dimitry Sibiryakov wrote: Could you stop with the absurd comparisons? One is normal maintenance and the other is (extreme) disaster recovery which are in no way comparable. But downtime is downtime. Customers don't care about its reason, do they?.. In my own cases, ANY downtime

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Yi Lu
Approach 1 seems to be least risky. Disk space should not be a big issue with today's hardware. -- View this message in context: http://firebird.1100200.n4.nabble.com/Firebird-Transaction-ID-limit-solution-tp4230210p4254287.html Sent from the firebird-devel mailing list archive at Nabble.com.

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Ann Harrison
On Mon, Jan 2, 2012 at 2:32 PM, Yi Lu y...@ramsoft.biz wrote: Approach 1 seems to be least risky. Disk space should not be a big issue with today's hardware. The problem is not disk space, but locality of reference. With small records, adding four bytes could decrease the number of rows per

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Lester Caine
Dimitry Sibiryakov wrote: In my own cases, ANY downtime during office hours is unacceptable, so any failure that brings the whole system down would result in penalties! Organised downtime is possible on many sites, but some sites are running 24/7, So we maintain data in a manor that

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Yi Lu
We are trying to take the approach #1, involving ODS change and data type changes for transaction id to use signed 64 bit integer. We just start looking at the code from today and following changes should be applied on code changes so far, 1. hdr_oldest_transaction, hdr)oldest_active,

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-02 Thread Philippe Makowski
Yi Lu [2012-01-03 02:02] : Also, we would be willing to contribute to the Firebird foundation in order to help us to develop this feature as we can't wait until Firebird 3.0. Please contact me directly, to see what are your need and willing so we can find the best solution -- Philippe

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Dmitry Yemanov
01.01.2012 14:34, Kjell Rilbe wrote: Are there unused flag bits available in the current record format, that could be used for this purpose? Or how are such flag bits encoded? We have 7 or 8 bits (out of 16) currently available. Dmitry

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Dmitry Yemanov
01.01.2012 16:48, Jesus Garcia wrote: Is not better use 64 bits integer?. In the near future computers speed, hdd speeds, etc. will increase, and transaction id of 64 bits overhead will affect less and less to performance. Disk speed was always the issue, it doesn't increase that fast as

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Dimitry Sibiryakov
01.01.2012 14:06, Dmitry Yemanov wrote: Disk speed was always the issue, it doesn't increase that fast as CPU/memory speed. So the on-disk size always matters quite a lot. CPU speed also has stopped growing. -- SY, SD.

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Jesús García
CPU speed also has stopped growing. You are right, speed has stopped but performance has increased. Jesus -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Adriano dos Santos Fernandes
On 01-01-2012 15:11, Jesús García wrote: I think the question is if it is neccesary and good for Firebird, and if the pros of having transactions id of 64bits are better that the cons of not having it. From my POV using 32 bits id is an important restriction in the engine, and i would

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread james
Nobody needs more than 4 billions active transactions concurrently. But that's not what you'll get, because you can complete transactions and get a hol, as it were. Can a process grab a transaction id and hold onto it 'for ever' without messing up the system in other ways that preventing the

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Kjell Rilbe
So far, these are the suggestions I've seen (I'm adding my own suggestion at the bottom): 1. Implement 64 bit id:s and use them everywhere always. Pros: - Straightforward once all code has been changed to use 64 bit id:s. - No overhead for various special flags and stuff. Cons: - Will increase

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Dimitry Sibiryakov
01.01.2012 21:31, Kjell Rilbe wrote: So far, these are the suggestions I've seen Hey, how about my suggestion leave everything as is and perform backup-restore cycle on demand?.. Pros: - No overhead. - No additional disk space. - No code change. Cons: - DBA with brain is required. --

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Kjell Rilbe
Den 2012-01-01 23:25 skrev Dimitry Sibiryakov såhär: 01.01.2012 21:31, Kjell Rilbe wrote: So far, these are the suggestions I've seen Hey, how about my suggestion leave everything as is and perform backup-restore cycle on demand?.. Pros: - No overhead. - No additional disk space. -

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Lester Caine
Dimitry Sibiryakov wrote: 01.01.2012 21:31, Kjell Rilbe wrote: So far, these are the suggestions I've seen Hey, how about my suggestion leave everything as is and perform backup-restore cycle on demand?.. Pros: - No overhead. - No additional disk space. - No code change. Cons: -

Re: [Firebird-devel] Firebird Transaction ID limit solution

2012-01-01 Thread Kjell Rilbe
Den 2012-01-01 23:47 skrev Lester Caine såhär: Dimitry Sibiryakov wrote: 01.01.2012 21:31, Kjell Rilbe wrote: So far, these are the suggestions I've seen Hey, how about my suggestion leave everything as is and perform backup-restore cycle on demand?.. Pros: - No overhead. - No

Re: [Firebird-devel] Firebird Transaction ID limit solution

2011-12-31 Thread Dmitry Yemanov
31.12.2011 17:57, Alexander Peshkov wrote: This will make each version of the record (not the record- but EACH version of it) 4 bytes longer. Not strictly necessary. We could use a variable-length encoding for txn ids longer than 32 bits and mark such records with a new flag. It would add

Re: [Firebird-devel] Firebird Transaction ID limit solution

2011-12-31 Thread Alexander Peshkov
В Сб., 31/12/2011 в 18:08 +0400, Dmitry Yemanov пишет: 31.12.2011 17:57, Alexander Peshkov wrote: This will make each version of the record (not the record- but EACH version of it) 4 bytes longer. Not strictly necessary. We could use a variable-length encoding for txn ids longer than

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject

2011-12-29 Thread Alex Peshkoff
On 12/28/11 22:51, Leyne, Sean wrote: It is feasible to roll over the transaction ID without putting the database offline? i.e. when ID is close to limit, reset it to 0 from the code? In theory there is a simple code change which would provide you another 6 months breathing room. But there

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject

2011-12-29 Thread Dimitry Sibiryakov
29.12.2011 20:41, Jesus Garcia wrote: Would not be better, instead of that, If transaction id is equal To 0, no transaction, else transaction. There is transaction number zero. As now there is a problem with transactionid and heavy loaded systems, that could solve in a little the

Re: [Firebird-devel] Firebird Transaction ID limit solution

2011-12-28 Thread Yi Lu
It is feasible to roll over the transaction ID without putting the database offline? i.e. when ID is close to limit, reset it to 0 from the code? -- View this message in context: http://firebird.1100200.n4.nabble.com/Firebird-Transaction-ID-limit-solution-tp4230210p4240359.html Sent from the

Re: [Firebird-devel] Firebird Transaction ID limit solution

2011-12-27 Thread Dimitry Sibiryakov
27.12.2011 19:17, Leyne, Sean wrote: That type of solution is not what I would define as a cluster. As you wish. But the rest of world consider this kind of system to be called a shared-nothing cluster. -- SY, SD.

Re: [Firebird-devel] Firebird Transaction ID limit solution - Email found in subject

2011-12-27 Thread Leyne, Sean
Dimitry, 27.12.2011 19:17, Leyne, Sean wrote: That type of solution is not what I would define as a cluster. As you wish. But the rest of world consider this kind of system to be called a shared-nothing cluster. You are correct! a shared-nothing cluster is a type of cluster, one

Re: [Firebird-devel] Firebird Transaction ID limit solution

2011-12-27 Thread Dimitry Sibiryakov
27.12.2011 22:17, Leyne, Sean wrote: I should have said: That type of solution is not what immediately comes to mind for me, since I see a shared disk solution (using redundant SAN storage) to be much easier to implement for FB. Unfortunately, shared-storage cluster doesn't solve

Re: [Firebird-devel] Firebird Transaction ID limit solution

2011-12-24 Thread Dmitry Kuzmenko
Hello, Yi! Saturday, December 24, 2011, 4:37:08 AM, you wrote: YL In the past week, we have a few databases from several customers run out of YL the 2^31-1 transaction ID limit. After some research I found this problem to YL have been around since five years ago. Some suggestions were given such