Re: [firebird-support] Page buffers & cache threshold

[Kjell Rilbe kjell.ri...@marknadsinformation.se [firebird-support]]

Den 2020-03-08 kl. 23:16, skrev liviuslivius liviusliv...@poczta.onet.pl 
[firebird-support]:
> Hi
>
> I do not know which rel notes have you read but this one is quite good 
> about
>
> https://firebirdsql.org/rlsnotesh/rnfb25-fbconf-fscache.html

Yes, that's the one. A few things are unclear to me.

1. What does " allow the page cache to be duplicated to the filesystem 
cache" mean? Does it mean that the same pages are kept in both FB's page 
buffer and in the file system cache? What's the alternative? Is it that 
FB prevents Windows from keeping DB pages in the file system cache? What 
are the pros and cons of either alternative?

2. OK, so the threshold compared to page buffer size determines if file 
system cache is used or not. But how do different threshold values 
affect things apart from that? I.e. if page buffer size is X, and 
threshold is changed from Y to Z where both Y and Z > X, what happens? 
Similarly, if both Y and Z < X, what happens? Nothing? It's just a 
threshold with yes/no effect? FileSystemCacheSize controls the other 
aspects?

Regards,
Kjell



[Non-text portions of this message have been removed]

Re: [firebird-support] Page buffers & cache threshold

[liviuslivius liviusliv...@poczta.onet.pl [firebird-support]]

HiI do not know which rel notes have you read but this one is quite good 
abouthttps://firebirdsql.org/rlsnotesh/rnfb25-fbconf-fscache.htmlRegards,Karol 
Bieniaszewski
null

Re: [firebird-support] Page buffers & cache threshold

[Kjell Rilbe kjell.ri...@marknadsinformation.se [firebird-support]]

Den 2020-03-08 kl. 10:45, skrev liviuslivius liviusliv...@poczta.onet.pl 
[firebird-support]:
> About 3
> "DefaultDbCachePages" in firebird.conf is used only at server start. 
> Then restart is needed. But you have 2 more options and you should 
> check it maybe you override default. Buffers you can set also in 
> databases.conf and in db file itself by gfix buffers.
>
> And it is used in priority.
> 1. buffers set in db by gfix if 0 then go to point 2.
> 2. Setting in databases.conf if not present go to point 3.
> 3. Setting from firebird.conf

Ah, that explains the limited firebird.exe process commit size at least. 
Apparently I have 131072 set in the database, so it overrides the 
setting n databases.conf. I wasn't aware of that priority order. Will 
change DB setting to 0 to be able to control it from databases.conf.

Thanks!

Other questions remain...

Regards,
Kjell



[Non-text portions of this message have been removed]

RE: [firebird-support] Page buffers & cache threshold

[liviuslivius liviusliv...@poczta.onet.pl [firebird-support]]

About 3"DefaultDbCachePages" in firebird.conf is used only at server start. 
Then restart is needed. But you have 2 more options and you should check it 
maybe you override default. Buffers you can set also in databases.conf and in 
db file itself by gfix buffers.And it is used in priority.1. buffers set in db 
by gfix if 0 then go to point 2.2. Setting in databases.conf if not present go 
to point 3.3. Setting from firebird.confRegards,Karol Bieniaszewski
 Oryginalna wiadomość Od: "Kjell Rilbe 
kjell.ri...@marknadsinformation.se [firebird-support]" 
 Data: 08.03.2020  10:10  (GMT+01:00) Do: 
firebird-support@yahoogroups.com Temat: [firebird-support] Page buffers & cache 
threshold 
 



  



  
  
  Hi,

Trying to figure out the best settings for my database and app usage. 
Firebird 3.0.4 SuperServer on virtualized Windows Server 2016 with 40 
Gbyte RAM and fast SSD disks (separate disks for OS and DB). Database 
almost 200 Gbyte. Page size 16k.

We can assume that no other heavy work is running on the server. The app 
itself uses a few Gbyte RAM.

I have set in databases.conf:
DefaultDbCachePages = 1310720
FileSystemCacheThreshold = 640K

And in firebird.conf:

FileSystemCacheSize = 50%
TempCacheLimit = 4G
MaxUnflushedWrites = 100
MaxUnflushedWriteTime = 60

All other settings (except some security ones) are set to default.

1. Would you recommend any changes to these settings, and why?

2. I see that the firebird.exe process has commit size about 2.2 Gbyte, 
which seems small considering the DefaultDbCachePages = 1310720 setting. 
Shouldn't this result in a page cache of 20 Gbyte?

3. I did change the DefaultDbCachePages setting without restarting the 
superserver process, but I did make sure all connections to the database 
in questions were closed. Does this setting take effect only when the 
server process is restarted?

4. It's not clear to me how the FileSystemCacheThreshold setting 
actually works. I have read the release notes for FB 2.5 as well as the 
comments inside firebird.conf and a couple of support list messages. But 
I still find it a bit muddy... I can see that the database file is 
cached, using RAMMap.exe. It reports "Standby" as "28733844 K" for that 
file.

It seems that the file system cache works and keeps large parts of the 
database file in the cache, but I do realize that the batch will for the 
most part need to read "new" data. The important thing ought to be to 
keep the indexes in cache. Right?

Reason/background: The use case is a batch operation that will do mostly 
(only?) reads and will read large parts of the database. The batch does 
its job in "chunks", and I can see that after a while the time to 
process each chunk goes up from about half a minute to about a full 
minute, or even worse. Restarting the app makes it go back to about half 
a minute per chunk, but just stopping the batch operation (without 
restarting the app) doesn't seem to help. I will investigate this more, 
of course. It's possible that the app has some garbage piling up that 
might slow things down.

Mvh,
Kjell

Kjell Rilbe
Telefon: 0733-44 24 64

Marknadsinformation i Sverige AB
Sundbybergs Torg 1 H 7 tr
172 67 Sundbyberg, Sweden
www.marknadsinformation.se <http://www.marknadsinformation.se/>
08-514 905 90

Företagskontakt.se <http://www.f%C3%B6retagskontakt.se/> 
Personkontakt.se <http://www.personkontakt.se/> DI Gasell 2018DI Gasell 
2019

[Non-text portions of this message have been removed]




 

[firebird-support] Page buffers & cache threshold

[Kjell Rilbe kjell.ri...@marknadsinformation.se [firebird-support]]

Hi,

Trying to figure out the best settings for my database and app usage. 
Firebird 3.0.4 SuperServer on virtualized Windows Server 2016 with 40 
Gbyte RAM and fast SSD disks (separate disks for OS and DB). Database 
almost 200 Gbyte. Page size 16k.

We can assume that no other heavy work is running on the server. The app 
itself uses a few Gbyte RAM.

I have set in databases.conf:
DefaultDbCachePages = 1310720
FileSystemCacheThreshold = 640K

And in firebird.conf:

FileSystemCacheSize = 50%
TempCacheLimit = 4G
MaxUnflushedWrites = 100
MaxUnflushedWriteTime = 60

All other settings (except some security ones) are set to default.

1. Would you recommend any changes to these settings, and why?

2. I see that the firebird.exe process has commit size about 2.2 Gbyte, 
which seems small considering the DefaultDbCachePages = 1310720 setting. 
Shouldn't this result in a page cache of 20 Gbyte?

3. I did change the DefaultDbCachePages setting without restarting the 
superserver process, but I did make sure all connections to the database 
in questions were closed. Does this setting take effect only when the 
server process is restarted?

4. It's not clear to me how the FileSystemCacheThreshold setting 
actually works. I have read the release notes for FB 2.5 as well as the 
comments inside firebird.conf and a couple of support list messages. But 
I still find it a bit muddy... I can see that the database file is 
cached, using RAMMap.exe. It reports "Standby" as "28733844 K" for that 
file.

It seems that the file system cache works and keeps large parts of the 
database file in the cache, but I do realize that the batch will for the 
most part need to read "new" data. The important thing ought to be to 
keep the indexes in cache. Right?

Reason/background: The use case is a batch operation that will do mostly 
(only?) reads and will read large parts of the database. The batch does 
its job in "chunks", and I can see that after a while the time to 
process each chunk goes up from about half a minute to about a full 
minute, or even worse. Restarting the app makes it go back to about half 
a minute per chunk, but just stopping the batch operation (without 
restarting the app) doesn't seem to help. I will investigate this more, 
of course. It's possible that the app has some garbage piling up that 
might slow things down.

Mvh,
Kjell



Kjell Rilbe
Telefon: 0733-44 24 64

Marknadsinformation i Sverige AB
Sundbybergs Torg 1 H 7 tr
172 67 Sundbyberg, Sweden
www.marknadsinformation.se 
08-514 905 90

Företagskontakt.se  
Personkontakt.se  DI Gasell 2018DI Gasell 
2019


[Non-text portions of this message have been removed]