RE: [firebird-support] Firebird using all available memory

['Thomas Kragh-Pedersen (TKG)' t...@kmd.dk [firebird-support]]

Hey Fabiano
Perhaps DefaultDbCachePages is overwritten by database header information. 
Gstat -h will tell you that. Look for “Page buffers”


From: firebird-support@yahoogroups.com 
Sent: 18. maj 2018 16:12
To: firebird-support@yahoogroups.com
Subject: [firebird-support] Firebird using all available memory


Hi,

I have a Debian 9 Linux server with 8GB of memory, a database with 30GB, page 
size 4KB, and Firebird 3.0.3 running in superserver mode and about 150-200 
concurrent connections.

In my firebird.conf I have changed the following properties:

DefaultDbCachePages = 524288 # 2GB
TempCacheLimit = 1024M # 1GB
LockMemSize = 2M

As superserver uses a shared cache, I presumed the total used memory should 
stay between 3GB and 4GB, but it keeps growing during the day until it reaches 
the 8GB limit and Firebird shuts down with this message:

debian  Fri May 18 10:45:35 2018
Shutting down the server with 150 active connection(s) to 2 
database(s), 0 active service(s)

I am checking the active transactions during the day, and altought there are 
some old (2h to 3h) active transactions, I don't that an old active transaction 
should cause this, but I am reporting anyway, maybe it is relevant.

Do someone have some advice to what should I do to prevent this from happening?

Maybe my understanding about how Firebird uses memory is incorrect, so any 
advice about how I should calculate the required amount of memory needed is 
welcome.

PS: I have migrated from FB 2.5 recently (previously I did use superclassic 
mode), and these problems started after the migration.

Regards,

Fabiano


Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information. Hvis 
du ved en fejltagelse modtager e-mailen, beder vi dig venligst informere 
afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi dig slette 
e-mailen i dit system uden at videresende eller kopiere den. Selvom e-mailen og 
ethvert vedhæftet bilag efter vores overbevisning er fri for virus og andre 
fejl, som kan påvirke computeren eller it-systemet, hvori den modtages og 
læses, åbnes den på modtagerens eget ansvar. Vi påtager os ikke noget ansvar 
for tab og skade, som er opstået i forbindelse med at modtage og bruge e-mailen.

Please note that this message may contain confidential information. If you have 
received this message by mistake, please inform the sender of the mistake by 
sending a reply, then delete the message from your system without making, 
distributing or retaining any copies of it. Although we believe that the 
message and any attachments are free from viruses and other errors that might 
affect the computer or it-system where it is received and read, the recipient 
opens the message at his or her own risk. We assume no responsibility for any 
loss or damage arising from the receipt or use of this message.

[firebird-support] Re: Upgrade from superclassic to superserver

[blackfalconsoftw...@outlook.com [firebird-support]]

Thomas...
 

 You may want to consider purchasing Carlos' e-book on Firebird Migration at 
the following link...
 

 https://www.firebirdnews.org/migration-guide-to-firebird-3/ 
https://www.firebirdnews.org/migration-guide-to-firebird-3/
 

 It is very well written and has a lot of information regarding the different 
types of Firebird servers and how to decide between them...

 

Re: [firebird-support] Upgrade from superclassic to superserver

['livius' liviusliv...@poczta.onet.pl [firebird-support]]

Hi,

few things:
1. instead FB3.0.3 use recent snapshot. It contain many fixes.

2. look at settings about buffers in your DB itself
if it utilize config then increase DefaultDbCachePages. 
Currently you set it to only to 8GB – but db size is much bigger. Is this 
server to serve only this database or it is also for other things?
If it is only for this database then increase value of DefaultDbCachePages.
If you do only backup on this server without test restore then you can utilize 
much more RAM.
But i say from Windows perspective not Centos...

3. TempBlockSize you set to only 2MB which is rather small. 
I know that you say about 3000 concurrent web users but with this value you 
will go with sort to the disc quite offten.
You must look if your queries are optimized 
and they do not use disk sorting.

You must balance this - use bigger DefaultDbCachePages or bigger sorting cache.

regards,
Karol Bieniaszewski

From: kragh.tho...@yahoo.com [firebird-support] 
Sent: Tuesday, May 22, 2018 10:58 PM
To: firebird-support@yahoogroups.com 
Subject: [firebird-support] Upgrade from superclassic to superserver

  

Hey

I am in the process of upgrading our Firebird database from 2.5.7 to 3.0.3. 
While making the upgrade, I want to switch from SuperClassic to SuperServer in 
order to gain better scalability. However configuring SuperServer i new to me. 

Background information:

The database server is a 16 core virtual machine with 128Gb of RAM, and ssd 
SAN. The database is about 100Gb and growing by 20Gb/year. Daily load is about 
3000 concurrent web users. The server is running CentOs 7



Here is the configurations that i have come up with so far. Pagesize is 16K

Firebird config:

DefaultDbCachePages = 524288(8Gb) 


On the old server this was 1024. On the new server this value is the one that I 
am most concerned about, i hope that someone can guide me. I have seen values 
of up to 2M(32Gb) in other guides, and I tempted to do the same. 



FileSystemCacheThreshold = 1048576


The old server used defa ult for 2.5.x



TempCacheLimit = 4294967296


Same as the old server



TempBlockSize = 2097152


Same as the old server



LockMemSize = 9437184

Same as the old server




LockHashSlots = 30011

Same as the old server







Regarding Centos:


Number of open files has been configured to 64K



Reading this Guide: 
https://firebirdsql.org/file/community/tour-2017/firebird.tuning.2017.english.pdf
 vm.pagecache should be limited to 40-50% However it looks like vm.pagecache is 
not availabel in CentOs, should  vfs_cache_pressure be used instead? 




I hope that someone can point me in the right direction or give some feedback.

[firebird-support] Upgrade from superclassic to superserver

[kragh.tho...@yahoo.com [firebird-support]]

Hey
 I am in the process of upgrading our Firebird database from 2.5.7 to 3.0.3. 
While making the upgrade, I want to switch from SuperClassic to SuperServer in 
order to gain better scalability. However configuring SuperServer i new to me. 
 Background information:
 The database server is a 16 core virtual machine with 128Gb of RAM, and ssd 
SAN. The database is about 100Gb and growing by 20Gb/year. Daily load is about 
3000 concurrent web users. The server is running CentOs 7
 

 Here is the configurations that i have come up with so far. Pagesize is 16K
 Firebird config:
 DefaultDbCachePages = 524288(8Gb) 

 On the old server this was 1024. On the new server this value is the one that 
I am most concerned about, i hope that someone can guide me. I have seen values 
of up to 2M(32Gb) in other guides, and I tempted to do the same. 
 

 FileSystemCacheThreshold = 1048576

 The old server used default for 2.5.x
 

 TempCacheLimit = 4294967296

 Same as the old server
 

 TempBlockSize = 2097152

 Same as the old server
 

 LockMemSize = 9437184
 Same as the old server

 

 LockHashSlots = 30011
 Same as the old server

 

 

 Regarding Centos:

 Number of open files has been configured to 64K
 

 Reading this Guide: 
https://firebirdsql.org/file/community/tour-2017/firebird.tuning.2017.english.pdf
 vm.pagecache should be limited to 40-50% However it looks like vm.pagecache is 
not availabel in CentOs, should  vfs_cache_pressure be used instead? 

 

 I hope that someone can point me in the right direction or give some feedback.
  

Re: [firebird-support] Choosing best server / processor for Firebird 3.03 64 bits database on Windows server 2016 64 bits

['livius' liviusliv...@poczta.onet.pl [firebird-support]]

Hi,

as always RAM is crucial for DB, but with such size of DB you can only benefit 
“a little”
I do not know the reall usage scenario – what data is used and how many data 
are “analyzed” by normal user.
Does database contain historical data not comonly accessed or user “scan” whole 
db every day?
You must check how the usage looks like – how many phisical reads you got every 
day.
If you see very big number with big RAM then your only way to speed things up 
is storage.

I have worked with 680 GB FB with 256 GB RAM and for me RAM was crucial, cache 
do the job.
But in my scenario there was “historical data” accessed only occasionally (once 
per day).

regards,
Karol Bieniaszewski

From: fabia...@itbizolutions.com.au [firebird-support] 
Sent: Sunday, May 20, 2018 7:08 PM
To: firebird-support@yahoogroups.com 
Subject: Re: [firebird-support] Choosing best server / processor for Firebird 
3.03 64 bits database on Windows server 2016 64 bits

  
Thank you Mark!!

So the key here would be RAM and SSD disks correct?

Cheers,
Fabian

Re: [firebird-support] PASCAL UDR - stored procedure BLOB param and return

['livius' liviusliv...@poczta.onet.pl [firebird-support]]

Anyone can point me in some direction?

regards,
Karol Bieniaszewski

From: 'livius' liviusliv...@poczta.onet.pl [firebird-support] 
Sent: Monday, May 21, 2018 9:35 PM
To: firebird-support@yahoogroups.com 
Subject: [firebird-support] PASCAL UDR - stored procedure BLOB param and return

  

Hi,

i can work in udr with near to all types of params but
how can i work with blob as input parameter  and output parameter 
for stored procedure?

should this be as BLOB or something else? 

InMessage = record
nr: Integer;
nrNull: WordBool;
dane: Blob;
daneNull: WordBool;
end;

InMessagePtr = ^InMessage;

OutMessage = record
result: Integer;
resultNull: WordBool;
dane: Blob;
daneNull: WordBool;
end;

OutMessagePtr = ^OutMessage;

how to consume this?

please advice

regards,
Karol Bieniaszewski