Re: [Firebird-devel] (possible) Database header
I’ll investigate that later. Thanks. -Tee- From: James Starkey Sent: Wednesday, August 24, 2022 4:16 PM To: For discussion among Firebird Developers Subject: Re: [Firebird-devel] (possible) Database header Of course there should. The ODS was designed with extensibility in mind, so the initial part of the header page was sacrosanct (i.e. immutable). This was an essential part of the architecture for forward and backward compatibility. The Y-valve, by design, would pass a connection string to all available engines until one was able to open the file and determine from the database header whether it could manage that database. Since the original architecture was robust and there was no rational reason to change it, I have very little doubt that you just open the a database file, read enough of the header page to get the ODS, and proceed from there. On Tue, Aug 23, 2022 at 12:14 AM Tommi Prami mailto:tommi.pr...@suonentieto.fi>> wrote: Hello, I was thinking that if there is some kind of header (simple enough) in Firebird Databases and/or backup files, maybe some one could point me into the right direction. I would preferably need (mainly for installer) small tool that would check the ODS version of the DB before I try to connect into it. And depending on that, do some preprocessing if needed or make some other choices based upon that. Currently I open the database and detect ODS version that way, but it makes life bit too complicated. -Tee- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Jim Starkey Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] (possible) Database header
same to use our own utility, gstat needs client libraries as well as our own, so we do not gain anything. Thanks for info. -Tee- -Original Message- From: Tommi Prami Sent: Wednesday, August 24, 2022 10:21 AM To: For discussion among Firebird Developers Subject: Re: [Firebird-devel] (possible) Database header Dang... Try to use GStat instead. -Tee- -Original Message- From: Dimitry Sibiryakov Sent: Tuesday, August 23, 2022 1:11 PM To: For discussion among Firebird Developers Subject: Re: [Firebird-devel] (possible) Database header Tommi Prami wrote 23.08.2022 5:40: > I was thinking that if there is some kind of header (simple enough) in > Firebird Databases and/or backup files, maybe some one could point me > into the right direction. No. Of course Firebird database has header page described in ods.h but it has no signature and even "fixed" part is unstable between versions. The utility must read whole fist kilobyte and try to map that data to several expected header versions checking whether several fields has sane values. -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] (possible) Database header
Dang... Try to use GStat instead. -Tee- -Original Message- From: Dimitry Sibiryakov Sent: Tuesday, August 23, 2022 1:11 PM To: For discussion among Firebird Developers Subject: Re: [Firebird-devel] (possible) Database header Tommi Prami wrote 23.08.2022 5:40: > I was thinking that if there is some kind of header (simple enough) in > Firebird Databases and/or backup files, maybe some one could point me > into the right direction. No. Of course Firebird database has header page described in ods.h but it has no signature and even "fixed" part is unstable between versions. The utility must read whole fist kilobyte and try to map that data to several expected header versions checking whether several fields has sane values. -- WBR, SD. Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] (possible) Database header
Hello, I was thinking that if there is some kind of header (simple enough) in Firebird Databases and/or backup files, maybe some one could point me into the right direction. I would preferably need (mainly for installer) small tool that would check the ODS version of the DB before I try to connect into it. And depending on that, do some preprocessing if needed or make some other choices based upon that. Currently I open the database and detect ODS version that way, but it makes life bit too complicated. -Tee- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5709) new Client 2.5.8 to server 2.5.7 server caused problem
new Client 2.5.8 to server 2.5.7 server caused problem -- Key: CORE-5709 URL: http://tracker.firebirdsql.org/browse/CORE-5709 Project: Firebird Core Issue Type: Bug Components: API / Client Library Affects Versions: 2.5.8 Environment: Windows 10 Reporter: Tommi Prami Priority: Minor We accidentally updated 2.5.8 client to environment where is older 2.5.7 server. With field defined as CREATE DOMAIN CMN_BOOLEAN AS SMALLINT CHECK (VALUE IS NULL OR VALUE IN (0, 1)); Now it crashes in our Delphi client application (Using IbDAC from DevArt) that field can't be accessed AsBoolean. (Just fixed from reading it AsInteger to AsBoolean to work with new client and server and with pure 2.5.7 environment) This might be IbDAC issue, or Firebird issue. Just filed this ticket to Firebird team can check is there some difference reading same field with 2.5.7 or 2.5.8 client when server is 2.5.7. I have feeling that using mismatched fbclient.dll have not been much of a problem in past versions, (If it is same major version at least) -Tee- -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5622) Extend SEQUENCE SEQUENSE syntact with CURRENT
Extend SEQUENCE SEQUENSE syntact with CURRENT -- Key: CORE-5622 URL: http://tracker.firebirdsql.org/browse/CORE-5622 Project: Firebird Core Issue Type: New Feature Environment: Any Reporter: Tommi Prami Priority: Minor Would be nice to do this SELECT CURRENT VALUE FOR CUSTOMER_SEQUENCE FROM RDB$DATABASE Would be logical (Not sure would it be standard) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [FB-Tracker] Created: (CORE-5621) Extend ALTER SEQUENCE syntax with SELECT
Extend ALTER SEQUENCE syntax with SELECT Key: CORE-5621 URL: http://tracker.firebirdsql.org/browse/CORE-5621 Project: Firebird Core Issue Type: New Feature Affects Versions: 3.0.2 Environment: any Reporter: Tommi Prami Priority: Minor ALTER SEQUENCE RESTART WITH (SELECT MAX(ID) FROM CUSTOMER); Sometimes the select is not that start forward, but anyways would do good to be able to fetch new value with select, Would make easier to reset the Sequences in some scenarios like when you have to enter data to db by hand etc (Like in development phase) or when there is an bug in software. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://tracker.firebirdsql.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Foreign key error even there should not be one (maybe, Weird problem in one database)
I've said I can provide the DB and needed data if it is kept secret :) -Tee- On Tue, Sep 20, 2016 at 3:55 PM, Vlad Khorsun wrote: > 19.09.2016 14:06, Tommi Prami wrote: > > Weird part was that I actually pumped the data to the empty database and > it behaved the same. > > > > This is more than less puzzling. > > > > And the Backup and restore did not help,. which also, I think, should > recreate the indexes and so on... > >Without reproducible example it is impossible even to confirm problem > existance. > > Regards, > Vlad > > > > -- > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel > -- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Foreign key error even there should not be one (maybe, Weird problem in one database)
Weird part was that I actually pumped the data to the empty database and it behaved the same. This is more than less puzzling. And the Backup and restore did not help,. which also, I think, should recreate the indexes and so on... -Tee- On Thu, Sep 15, 2016 at 10:58 AM, Thomas Steinmaurer wrote: > > Whole story here : > > > > https://plus.google.com/105626236962310493174/posts/3UZB71NjUhA > > > > Copy of message > > - - - - - - - - - - - - - - - - > > Is there a tool (or way) to debug database step by step. > > > > Now we have weird situation where delete from one database fails, on > > foreign key error, which sounds like it would be trivial thing, but seems > > so far it is not. > > > > I was just thinking that if it would be our application code I would be > > able to debug through all of the related code and see if there is the > > culprit or not. > > > > We suspect it has somethign to do with specific data on the rows. because > > some of them delete perfectly fine. This happens in this one database > only. > > I've done backup restore for it, I ran gfix stuff on it. I even pumped > the > > data in empty DB. > > > > it should be problem of database structure (trickers and such), data or > the > > Firebird. and most likely it problem of first two. > > > > I was just thinking that is there an tool I could see step by step all > the > > operations in the DB before the foreigh key error is raised. So I could > > step by step see the order the trickers and so fires and what they do. > > > > Or if someone at Firebird project could check if it is indeed the > Firebird > > bug. I think we could send the DB, if it he/she keeps it safe and keeps > it > > secret :) > > If you re-create the FK constraint - which re-creates an index behind the > scene - solves the problem, then some sort of data/index corruption might > be an explanation. > > > Thomas > > > -- > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel > -- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Foreign key error even there should not be one (maybe, Weird problem in one database)
It is cascade thingy (I am not the DB guy...) So it normally works OK...
If we drop the foreign key and create it again it starts to work. Data from
Both tables gets removed correctly.
Appears the there is small difference in the newly created foreign key.
For this foreign key following stuff in system tables are :
RDB$UNIQUE_FLAG = 0 RDB$INDEX_INACTIVE = 0 RDB$INDEX_TYPE = 0
And we create it again those are NULL
Do not know if that is an actual cause but difference we see.
-Tee-
On Thu, Sep 15, 2016 at 10:44 AM, Dimitry Sibiryakov
wrote:
> 15.09.2016 9:39, Tommi Prami wrote:
> > Foreign key references are present for the record
> > Problematic key value is ("TOSITEID" = 945)
> >
> > Data is there, it can be selected but the delete fails on this DB
>
>Delete fails exactly because data is there. You must delete referencing
> records first
> and only then - record in master table.
>
>
> --
>WBR, SD.
>
>
> --
> Firebird-Devel mailing list, web interface at
> https://lists.sourceforge.net/lists/listinfo/firebird-devel
>
--
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Foreign key error even there should not be one (maybe, Weird problem in one database)
We use
2.5.6.27020 (32bit)
[#1: Delete - PLAN (TOSITE INDEX (PK_TOSITE))]
violation of FOREIGN KEY constraint "FK_VIENTI_TOSITE" on table "VIENTI"
Foreign key references are present for the record
Problematic key value is ("TOSITEID" = 945)
Data is there, it can be selected but the delete fails on this DB
(Hope this helps)
On Thu, Sep 15, 2016 at 10:26 AM, Alex Peshkoff wrote:
> On 15.09.2016 10:14, Tommi Prami wrote:
> > Whole story here :
> >
> > https://plus.google.com/105626236962310493174/posts/3UZB71NjUhA
> >
> > Copy of message
> > - - - - - - - - - - - - - - - -
> > Is there a tool (or way) to debug database step by step.
> >
> > Now we have weird situation where delete from one database fails, on
> > foreign key error, which sounds like it would be trivial thing, but seems
> > so far it is not.
> >
> > I was just thinking that if it would be our application code I would be
> > able to debug through all of the related code and see if there is the
> > culprit or not.
> >
> > We suspect it has somethign to do with specific data on the rows. because
> > some of them delete perfectly fine. This happens in this one database
> only.
> > I've done backup restore for it, I ran gfix stuff on it. I even pumped
> the
> > data in empty DB.
> >
> > it should be problem of database structure (trickers and such), data or
> the
> > Firebird. and most likely it problem of first two.
> >
> > I was just thinking that is there an tool I could see step by step all
> the
> > operations in the DB before the foreigh key error is raised. So I could
> > step by step see the order the trickers and so fires and what they do.
> >
> > Or if someone at Firebird project could check if it is indeed the
> Firebird
> > bug. I think we could send the DB, if it he/she keeps it safe and keeps
> it
> > secret :)
> >
> > - - - -
> >
> > This is slightly scary. We dropped the FK and created it back again and
> it
> > works like a charm. And friend checked that foreign key from the System
> > tables and it seems to be exactly same.
> >
> > So could this be the bug or issue in firebird after all?
>
> What FB version do you use?
> Modern firebird versions (like FB3 for example) produce good explanation
> what FK failed and why.
> For example (in employee.fdb):
>
> SQL> delete from country where country = 'Japan';
> Statement failed, SQLSTATE = 23000
> violation of FOREIGN KEY constraint "INTEG_11" on table "JOB"
> -Foreign key references are present for the record
> -Problematic key value is ("COUNTRY" = 'Japan')
>
>
>
> --
> Firebird-Devel mailing list, web interface at
> https://lists.sourceforge.net/lists/listinfo/firebird-devel
>
--
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Foreign key error even there should not be one (maybe, Weird problem in one database)
Whole story here : https://plus.google.com/105626236962310493174/posts/3UZB71NjUhA Copy of message - - - - - - - - - - - - - - - - Is there a tool (or way) to debug database step by step. Now we have weird situation where delete from one database fails, on foreign key error, which sounds like it would be trivial thing, but seems so far it is not. I was just thinking that if it would be our application code I would be able to debug through all of the related code and see if there is the culprit or not. We suspect it has somethign to do with specific data on the rows. because some of them delete perfectly fine. This happens in this one database only. I've done backup restore for it, I ran gfix stuff on it. I even pumped the data in empty DB. it should be problem of database structure (trickers and such), data or the Firebird. and most likely it problem of first two. I was just thinking that is there an tool I could see step by step all the operations in the DB before the foreigh key error is raised. So I could step by step see the order the trickers and so fires and what they do. Or if someone at Firebird project could check if it is indeed the Firebird bug. I think we could send the DB, if it he/she keeps it safe and keeps it secret :) - - - - This is slightly scary. We dropped the FK and created it back again and it works like a charm. And friend checked that foreign key from the System tables and it seems to be exactly same. So could this be the bug or issue in firebird after all? -- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Lock manager's hash calculations
I stumbled upon quite evesome Hash-algorithm xxHash. There are lightning fast implementations. https://github.com/Cyan4973/xxHash Noted that usually the Hash algorithm is not the bottleneck, but still pretty interesting... -Tee- On Tue, Dec 29, 2015 at 6:05 PM, Dimitry Sibiryakov wrote: > 29.12.2015 11:43, Alex Peshkoff wrote: > >> That I do not expect to get something better than this bell curve: >> > > I was unable to create a good load on database with my weak notebook, > but oltp-emul with 5 windows gave me these numbers for lock manager using > CRC32: > > Hash slots: 8191, Hash lengths (min/avg/max):0/ 0/ 6 >> Hash lengths distribution: >> 0 : 3591 (43%) >> 1 : 2992 (36%) >> 2 : 1194 (14%) >> 3 : 324 (3%) >> 4 : 71 (0%) >> 5 : 14 (0%) >> 6 :5 (0%) >> > > It looks like the curve falls faster. > > -- > WBR, SD. > > > -- > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > -- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Crypto Algoritm Performance
The Hardware AES uses differend MAgic constant (Seed?) that many popular software implementations (If I recall, reading from somewhere). So the initial state of non HW accelerated implementation should match the HW one, that's all. PS: Can't remember the details just pumped into this one, that I once stumbled upon some article talking about that On Sat, Sep 5, 2015 at 9:50 PM, Leyne, Sean wrote: > Jim and Boris, > > > Something you may want to investigate is replacing the "pure C" > > implementation of ChaCha20 with the rotate step replaced with either a > > compiler intrinsic (Microsoft) or a bit of assembler (gcc). SHA1 has > > the same issue. I haven't a clue as to why popular crypto algoritms > > use a rotate, virtually all microprocessors have rotate instructions, > > but C lacks a rotate operator and the standard libraries neglect to > support it. > > Forgive my naïve point of view, but given that AES instruction set has > been built into AMD and Intel CPUs since 2011, why do you feel that it is > necessary to push for ChaCha20***? > > To my reading, Boris' numbers have shown that AES performance is more than > adequate (53.2 AVG seconds to process 256MB = 4+MB/s). > > Further, considering that the use can is the encryption of data blocks > which would be much smaller than even 1MB, will be performance difference > really be noticeable? > > > Sean > > *** Separately, with Intel HyperThreaded CPUs and considering that AES in > "on-chip" wouldn't that allow the core processing the encryption to shift > to focus on the other thread instruction while the first thread wait for > the on chip AES processor operates? In other words, isn't it possible that > ChaCha20 is only faster when CPUs are being "single minded" and that real > world performance on a server dealing with several tasks might favor CPUs > with native AES instructions? > > > > Here are numbers: > > -- > > --- AES, BOTAN based code, with AES-NI instruction set all enc > > > > 531.153.2 > > > > -- > > > > AES, INTEL based code, with AES-NI instruction set all enc > > > > 544.876.6 > > > > > > -- > > AES, code based on Bouncy Castle (Java) , without AES-NI instruction set > > allenc > > > > 2071.8 1620.6 > > > > > > -- > > ChaCha20, code based on Bouncy Castle (Java) > > > > 1712.7 1234.8 > > > > -- > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel > -- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Insecure hashing and encryption in Firebird 3
Just my 0.02€ http://ianix.com/pub/chacha-deployment.html ChaCha20 is in TSL and OpenSSH etc. and mainly because it is secure and it is fast. -Tee- On Thu, Aug 20, 2015 at 2:26 PM, James Starkey wrote: > > > On Thursday, August 20, 2015, Michal Kubecek wrote: > >> On Wed, Aug 19, 2015 at 02:55:33PM -0400, Jim Starkey wrote: >> > >> > The best alternative to RC4 is AES-128. It is "more" (but not >> > measurably) secure but also a couple of hundred times as expensive to >> > compute. If you don't believe me, run your own numbers. >> >> I did... >> >> >> So it looks like "couple of hundred" means 6-8 when calculating AES in >> software and 1.25-1.35 when using in-CPU implementation (in this case, >> AMD FX-8150 from 2012). >> >> Anyway, I agree with most of the rest. >> >> >> Interesting. The tests I ran were "in situ" in early NuoDB, which > encrypts all message traffic, including quite significant replication > traffic. The RC4 was one I coded myself. The AES implimentation was from > the open source Tom's Crypt Library. The overhead compared with > unencrypted traffic was about 86% for AES-128 and about 5% for RC4. > Perhaps much of the difference was in the qualities of the implementation. > > RC4 operates on a 256 byte state table, swapping two entries per operation > XORing one with the next byte of the clear text. AES-128 operates on a 4 x > 4 matrix of bytes, performing 10 rounds of four matrix operations per > round. That said, much of the efficiency of AES is dependent on the key > schedule algorithm computed at initialization time. > > I have no argument against AES, used properly, or RC4, again used > properly. AES/ECB, aka simple AES, in some applications is more vulnerable > than RC4 as used in WEP. But AES-128 and RC4 with pseudo random keys that > are never reused are each unbreakable. If the press had latched onto the > ineffectiveness of AES/ECB for encrypting an image, AES would be in the > same dog house as RC4. AES/CBC XORs each input block with the encrypted > output of the previous block, solving the problem. > > If you were starting over in a world when most servers had the AES > instruction set, it would be a no brainer to use AES/CBC. But that isn't > an argument switching with an installed base, though it does recommend an > architecture where the choice of algorithm is a connection parameter. > > Anyway, you've convinced me to think about an overhaul of the encryption > architecture of AmorphousDB. Thanks for pointing out my > shameless exaggeration. > > > -- > Jim Starkey > > > -- > > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel > > -- Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Firebird 3 API demo usage with Pascal
If you use {$A+} all fields should appear to aligned to 8 byte boundaries
(The default).
I think would be easier to make it controllable, to use packed record and
add some "Foo Bar" padding variables if needed.
If the original structure is continuous then packed record with correct
field sizes should be used. it there are gaps in original still i would use
packed record and add padding into the gaps.
Anyhow very nice to see this kind of effort going on!
-Tee-
On Tue, Mar 17, 2015 at 11:40 AM, Vlad Khorsun
wrote:
> 17.03.2015 10:06, liviusliv...@poczta.onet.pl wrote:
> >> but I see what hapened
> >> Whe should decide about buffer alignment
> >> I can use "packed record" instead of "record" then data are as is
> >> But what is your buffer alignment when you fill data in that record?
> >>
> >
> >>> Search for type_alignments:
> >>> https://github.com/FirebirdSQL/core/blob/master/src/jrd/align.h#L83
> >>> Adriano
> >
> > This does not help
> > Something is wrong when i add types like
> > - Int64
> > - SmallInt
> >
> > i tried all alignments Byte, Word, DWord, QWord
>
>After re-reading docs
>
>
> http://docwiki.embarcadero.com/RADStudio/XE6/en/Internal_Data_Formats#Record_Types
>
>i'd say you should not use nor special(explicit) alignment nor packed
> records. Just ensure you have {$A+} when record type is declared and
> you use record members with data types of correct size each.
>
>Sorry, i have no Delphi right now to check by myself
>
> Regards,
> Vlad
>
>
> --
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now. http://goparallel.sourceforge.net/
> Firebird-Devel mailing list, web interface at
> https://lists.sourceforge.net/lists/listinfo/firebird-devel
>
--
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
