Re: [Firebird-devel] RFC: Tablespaces

2016-03-04 Thread Lester Caine
On 03/03/16 20:13, Jim Starkey wrote: >> Where i can see a useful segregation is archival data which will never >> be modified. Would be very usefull if all that could be backed up on a >> Slow cycle, and only the real dynamic data kept in the primary table >> space? But one can do most of that by

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Jim Starkey
se painful email clients! -Original Message- From: Ann Harrison To: For discussion among Firebird Developers Sent: Thu, 03 Mar 2016 18:37 Subject: Re: [Firebird-devel] RFC: Tablespaces On Thu, Mar 3, 2016 at 1:02 PM, Jim Starkey <mailto:j...@jimstarkey.net&

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Alexandre Benson Smith
Em 3/3/2016 04:04, Dmitry Yemanov escreveu: > 03.03.2016 09:48, Dmitry Yemanov wrote: >>> Blobs could be moved into separate tablespace. It could make backup of >>> "data" tablespace faster and smaller. We can even think about "offline" >>> tablespace. >> Really good idea. Maybe more important for

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Adriano dos Santos Fernandes
Em 03/03/2016 13:43, Vlad Khorsun escreveu: > >I would add > > 4. Allow to specify FW and Read-Only settings on per tablespace level > 5. Ability to bring individual tablespace off-line (and back online, of > course) > ... It's not a tablespace feature, but a schema feature integrated wi

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread lester
? Sent from my android device so quoting is crap ... need to kill these painful email clients! -Original Message- From: Ann Harrison To: For discussion among Firebird Developers Sent: Thu, 03 Mar 2016 18:37 Subject: Re: [Firebird-devel] RFC: Tablespaces On Thu, Mar 3, 2016 at 1:02

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Ann Harrison
On Thu, Mar 3, 2016 at 1:02 PM, Jim Starkey wrote: > > >> Non-goals: > >> > ... > >> 2. Store blobs in other than the table's data space. > > Why not allow blobs to be separated from regular data ? > > OK, reasonable question. Obviously they could, but it would require > either storing sma

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Jim Starkey
On 3/3/2016 11:43 AM, Vlad Khorsun wrote: > 03.03.2016 17:57, Jim Starkey wrote: >> The place to start is with a clear, well understood, and agreed set of >> requirements. Here is a suggested starting point. >> >> First, some definitions. A "data space" is the collection of pointer and >> data

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Ann Harrison
On Wed, Mar 2, 2016 at 4:51 PM, Vlad Khorsun wrote: > >Blobs could be moved into separate tablespace. It could make backup of > "data" tablespace faster and smaller. We can even think about "offline" > tablespace. > > Interesting idea. I'm not quite sure how it would work ... new blobs should

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Ann Harrison
On Wed, Mar 2, 2016 at 4:23 PM, Dmitry Yemanov wrote: > > When we speak about tablespaces, it usually means that the database > consists of multiple files and different database object are stored in > different files. Each such file is named within a database and called a > tablespace. > It's p

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Vlad Khorsun
03.03.2016 17:57, Jim Starkey wrote: > The place to start is with a clear, well understood, and agreed set of > requirements. Here is a suggested starting point. > > First, some definitions. A "data space" is the collection of pointer and > data pages that hold the data from a single table. An

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Jim Starkey
The place to start is with a clear, well understood, and agreed set of requirements. Here is a suggested starting point. First, some definitions. A "data space" is the collection of pointer and data pages that hold the data from a single table. An "index" (in this context), is the set of in

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread liviuslivius
Vlad proposition is better then writting somethink on every transaction and should work without ovehead look at this (if i understand Vlad's proposition correctly) two fields MAIN_DB_GUID, MAIN_OPENED and the same for all table space files (as S_DB_GUID, S_OPENED) 1. First connect to db 2. Read

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Dalton Calford
We use Oracle and table spaces. We specify which drives/drive channels hold which data/tables/indexes to optimize various OLAP/OLTP needs. We also use firebird. We use multiple files as large multi-raid drive arrays have problems in that the larger arrays may have multiple drives fail faster t

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Vlad Khorsun
03.03.2016 15:36, Dimitry Sibiryakov wrote: > 03.03.2016 13:38, Vlad Khorsun wrote: >>> May be transaction counters in each header written on each commit that >>> change the tablespace can work better? >> This will create crazy overhead > > Yes. But aren't tablespaces considered for

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Dimitry Sibiryakov
03.03.2016 13:38, Vlad Khorsun wrote: >> May be transaction counters in each header written on each commit that >> change the >> >tablespace can work better? > This will create crazy overhead Yes. But aren't tablespaces considered for enterprise usage where IO for every tablespace file i

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Vlad Khorsun
03.03.2016 13:42, Dimitry Sibiryakov wrote: > 03.03.2016 12:29, Vlad Khorsun wrote: >>> How will be enforced consistency between tablespaces? How to detect >>> and handle cases if tablespace file is lost or is older/newer that main file? >> I'm afraid it will require some mark (GUI

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Dimitry Sibiryakov
03.03.2016 12:29, Vlad Khorsun wrote: >> How will be enforced consistency between tablespaces? How to detect and >> handle cases if >> >tablespace file is lost or is older/newer that main file? > I'm afraid it will require some mark (GUID ?) stored in both tablespace > (at header page ?)

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Vlad Khorsun
02.03.2016 23:58, Dimitry Sibiryakov wrote: > 02.03.2016 22:51, Vlad Khorsun wrote: >> Blobs could be moved into separate tablespace. It could make backup of >> "data" tablespace faster and smaller. We can even think about "offline" >> tablespace. > >> Also, PIP and Header page. Should eve

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Vlad Khorsun
03.03.2016 9:04, Dmitry Yemanov wrote: > 03.03.2016 09:48, Dmitry Yemanov wrote: >> >>> Blobs could be moved into separate tablespace. It could make backup of >>> "data" tablespace faster and smaller. We can even think about "offline" >>> tablespace. >> >> Really good idea. Maybe more important for

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Dimitry Sibiryakov
03.03.2016 11:10, Atri Sharma wrote: > > > I feel that tablespaces allow managing multiple independent identities > inside a database > > conveniently, and making stuff like backups pretty fast and granular. > > How are you going to solve problem with inconsistency between > table

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Atri Sharma
> > > I feel that tablespaces allow managing multiple independent identities > inside a database > > conveniently, and making stuff like backups pretty fast and granular. > >How are you going to solve problem with inconsistency between > tablespaces? > > > Which specific inconsistency can you

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Dimitry Sibiryakov
03.03.2016 10:33, Atri Sharma wrote: > Personally, I don't care much about tablespaces per se. But I foresee > that table partitioning could be stacked over the tablespace > implementation and I like this idea. > > > > I feel that tablespaces allow managing multiple independent identiti

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Atri Sharma
> > > And another question: does RDB$PAGES need to include the pagespace ID or > we never allow tablespaces for TIP, generator pages and system tables? > Pagespace ID for PP/IRP can be retrieved from RDB$ tables based on > relation_id, so I don't see a problem to store just the "local" page > numbe

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread Atri Sharma
On Thu, Mar 3, 2016 at 11:01 AM, Dmitry Yemanov wrote: > 03.03.2016 00:23, Dmitry Yemanov wrote: > > > > Someone may think about per-tablespace physical backups and other > > possible usage cases. > > Personally, I don't care much about tablespaces per se. But I foresee > that table partitioning

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread liviuslivius
I see that something strange happend with my email. It is broken :( something happened in place "ecod version." -< i do not write this and also at the end i see duplicate lines, cuts and other strange things. Very strange for me... I paste below orginal message --

Re: [Firebird-devel] RFC: Tablespaces

2016-03-03 Thread liviuslivius
Hi, please do not touch shadows. Shadows are used by many organisations. I saw it in almost any of them where i do some job for it. And someone in near past worked on extending its usage e.g. CORE-4955 I agree with you that already existing feature "multi file database" is really "Database spac

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Dmitry Yemanov
03.03.2016 09:48, Dmitry Yemanov wrote: > >> Blobs could be moved into separate tablespace. It could make backup of >> "data" tablespace faster and smaller. We can even think about "offline" >> tablespace. > > Really good idea. Maybe more important for an average FB user than the > other usage case

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Atri Sharma
Can we start fleshing subtasks which need to be started to progress this please? The discussion is going really well! On 3 Mar 2016 12:20 pm, "Dmitry Yemanov" wrote: > 03.03.2016 00:51, Vlad Khorsun wrote: > > > > Blobs could be moved into separate tablespace. It could make backup of > > "data" t

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Dmitry Yemanov
03.03.2016 00:51, Vlad Khorsun wrote: > > Blobs could be moved into separate tablespace. It could make backup of > "data" tablespace faster and smaller. We can even think about "offline" > tablespace. Really good idea. Maybe more important for an average FB user than the other usage cases. > The

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Dmitry Yemanov
03.03.2016 02:12, Leyne, Sean wrote: > > While I can see possible use cases, I am concerned that implementing > tablespaces would only make database management more complicated. True, but only for those using it. > Based on your outline/proposal, however, aren't Tablespaces just enabling > mult

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Dmitry Yemanov
03.03.2016 00:23, Dmitry Yemanov wrote: > > Someone may think about per-tablespace physical backups and other > possible usage cases. Personally, I don't care much about tablespaces per se. But I foresee that table partitioning could be stacked over the tablespace implementation and I like this

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Adriano dos Santos Fernandes
Em 02/03/2016 20:12, Leyne, Sean escreveu: > > While I can see possible use cases, I am concerned that implementing > tablespaces would only make database management more complicated. > Only for who wants adavanced settings. > The filename/path would become integral to the database and thus c

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Leyne, Sean
Dmitry, > When we speak about tablespaces, it usually means that the database > consists of multiple files and different database object are stored in > different > files. Each such file is named within a database and called a tablespace. And > each tablespace has its own page set and page number

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Dimitry Sibiryakov
02.03.2016 22:51, Vlad Khorsun wrote: > Blobs could be moved into separate tablespace. It could make backup of > "data" tablespace faster and smaller. We can even think about "offline" > tablespace. >Also, PIP and Header page. Should every tablespace contains its own PIP ? > Currently, eve

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Vlad Khorsun
02.03.2016 23:23, Dmitry Yemanov wrote: > > Historically, Firebird databases consist of a sequential set of pages of > the fixed size (4-16KB currently). This page set is distributed across > one (usually) or multiple files (*) The page number initially was SLONG, > now it's ULONG. So the theoretic

Re: [Firebird-devel] RFC: Tablespaces

2016-03-02 Thread James Starkey
Multi-file databases were implemente to handle databases larger than the disk or file systen size. Disks have gotten a great deal larger, but so have databases, so obviously something has to give to support your 7 TB database.I sense an interesting design opportunity. On Wednesday, March 2,

[Firebird-devel] RFC: Tablespaces

2016-03-02 Thread Dmitry Yemanov
Historically, Firebird databases consist of a sequential set of pages of the fixed size (4-16KB currently). This page set is distributed across one (usually) or multiple files (*) The page number initially was SLONG, now it's ULONG. So the theoretically possible maximum database size is curren