Re: [HACKERS] patch: preload dictionary new version

2010-07-20 Thread Pavel Stehule
Hello 2010/7/20 Itagaki Takahiro : > 2010/7/14 Pavel Stehule : >> this patch is significantly reduced original patch. It doesn't propose >> a simple allocator - just eliminate a high memory usage for ispell >> dictionary. > > I don't think introducing new methods is a good idea. If you want a > si

Re: [HACKERS] patch: preload dictionary new version

2010-07-19 Thread Itagaki Takahiro
2010/7/14 Pavel Stehule : > this patch is significantly reduced original patch. It doesn't propose > a simple allocator - just eliminate a high memory usage for ispell > dictionary. I don't think introducing new methods is a good idea. If you want a simple allocator, MemoryContextMethods layer see

Re: [HACKERS] patch: preload dictionary new version

2010-07-14 Thread Pavel Stehule
Hello this patch is significantly reduced original patch. It doesn't propose a simple allocator - just eliminate a high memory usage for ispell dictionary. without this patch the ispell dictionary takes 55MB for tsearch2 context and 27MB in temp context. With this patch it takes "only" 25MB tsear

Re: [HACKERS] patch: preload dictionary new version

2010-07-12 Thread Pavel Stehule
2010/7/12 Tom Lane : > Itagaki Takahiro writes: >> 2010/7/8 Tom Lane : >>> For example, the dictionary-load code could automatically execute >>> the precompile step if it observed that the precompiled copy of the >>> dictionary was missing or had an older file timestamp than the source. I am not

Re: [HACKERS] patch: preload dictionary new version

2010-07-11 Thread Tom Lane
Itagaki Takahiro writes: > 2010/7/8 Tom Lane : >> For example, the dictionary-load code could automatically execute >> the precompile step if it observed that the precompiled copy of the >> dictionary was missing or had an older file timestamp than the source. > There might be a problem in automa

Re: [HACKERS] patch: preload dictionary new version

2010-07-11 Thread Itagaki Takahiro
2010/7/8 Tom Lane : > For example, the dictionary-load code could automatically execute > the precompile step if it observed that the precompiled copy of the > dictionary was missing or had an older file timestamp than the source. There might be a problem in automatic precompiler -- Where should w

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Pavel Stehule
2010/7/8 Tom Lane : > Pavel Stehule writes: >> 2010/7/8 Robert Haas : >>> A precompiler can give you all the same memory management benefits. > >> I use mmap(). And with  mmap the precompiler are not necessary. >> Dictionary is loaded only one time - in original ispell format. I >> think, it is mu

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Tom Lane
Pavel Stehule writes: > 2010/7/8 Robert Haas : >> A precompiler can give you all the same memory management benefits. > I use mmap(). And with mmap the precompiler are not necessary. > Dictionary is loaded only one time - in original ispell format. I > think, it is much more simple for administr

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Pavel Stehule
2010/7/8 Robert Haas : > On Thu, Jul 8, 2010 at 7:03 AM, Pavel Stehule wrote: >> 2010/7/8 Robert Haas : >>> On Wed, Jul 7, 2010 at 10:50 PM, Takahiro Itagaki >>> wrote: This patch allocates memory with non-file-based mmap() to preload text search dictionary files at the server sta

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Robert Haas
On Thu, Jul 8, 2010 at 7:03 AM, Pavel Stehule wrote: > 2010/7/8 Robert Haas : >> On Wed, Jul 7, 2010 at 10:50 PM, Takahiro Itagaki >> wrote: >>> This patch allocates memory with non-file-based mmap() to preload text >>> search >>> dictionary files at the server start. Note that dist files are no

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Pavel Stehule
2010/7/8 Robert Haas : > On Wed, Jul 7, 2010 at 10:50 PM, Takahiro Itagaki > wrote: >> This patch allocates memory with non-file-based mmap() to preload text search >> dictionary files at the server start. Note that dist files are not mmap'ed >> directly in the patch; mmap() is used for reallocata

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Robert Haas
On Wed, Jul 7, 2010 at 10:50 PM, Takahiro Itagaki wrote: > This patch allocates memory with non-file-based mmap() to preload text search > dictionary files at the server start. Note that dist files are not mmap'ed > directly in the patch; mmap() is used for reallocatable shared memory. I thought

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Pavel Stehule
Hello I found a page http://www.genesys-e.org/jwalter//mix4win.htm where is section >>Emulation of mmap/munmap<<. Can be a solution? Regards Pavel Stehule 2010/7/8 Pavel Stehule : > Hello > > 2010/7/8 Takahiro Itagaki : >> >> Pavel Stehule wrote: >> >>> this version has enhanced AllocSet alloc

Re: [HACKERS] patch: preload dictionary new version

2010-07-08 Thread Pavel Stehule
Hello 2010/7/8 Takahiro Itagaki : > > Pavel Stehule wrote: > >> this version has enhanced AllocSet allocator - it can use a  mmap API. > > I review your patch and will report some comments. However, I don't have > test cases for the patch because there is no large dictionaries in the > default po

Re: [HACKERS] patch: preload dictionary new version

2010-07-07 Thread Takahiro Itagaki
Pavel Stehule wrote: > this version has enhanced AllocSet allocator - it can use a mmap API. I review your patch and will report some comments. However, I don't have test cases for the patch because there is no large dictionaries in the default postgres installation. I'd like to ask you to sup