On Thu, Jan 9, 2014 at 12:46 PM, Claudio Freire klaussfre...@gmail.com wrote:
On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote:
It would be nice to have better operating system support for this.
For example, IIUC, 64-bit Linux has 128TB of address space available
for
On Thu, Jan 9, 2014 at 2:09 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Thu, Jan 9, 2014 at 12:21 AM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com
On Fri, Jan 10, 2014 at 3:23 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Jan 9, 2014 at 12:46 PM, Claudio Freire klaussfre...@gmail.com
wrote:
On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote:
It would be nice to have better operating system support for this.
On Fri, Jan 10, 2014 at 1:35 PM, Claudio Freire klaussfre...@gmail.com wrote:
You can map a segment at fork time, and unmap it after forking. That
doesn't really use RAM, since it's supposed to be lazily allocated (it
can be forced to be so, I believe, with PROT_NONE and MAP_NORESERVE,
but I
Robert Haas robertmh...@gmail.com writes:
I've often thought that 64-bit machines are so capable that there's no
reason to go any higher. But lately I've started to wonder. There
are already machines out there with 2^40 bytes of physical memory,
and the number just keeps creeping up. When
On Wed, Jan 8, 2014 at 2:39 PM, knizhnik knizh...@garret.ru wrote:
I wonder what is the intended use case of dynamic shared memory?
Is is primarly oriented on PostgreSQL extensions or it will be used also in
PosatgreSQL core?
My main motivation is that I want to use it to support parallel
On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote:
It would be nice to have better operating system support for this.
For example, IIUC, 64-bit Linux has 128TB of address space available
for user processes. When you clone(), it can either share the entire
address space
On Thu, Jan 9, 2014 at 12:21 AM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com wrote:
Well, right now we just reopen the same object from all of the
On 01/09/2014 09:22 PM, Robert Haas wrote:
On Wed, Jan 8, 2014 at 2:39 PM, knizhnik knizh...@garret.ru wrote:
I wonder what is the intended use case of dynamic shared memory?
Is is primarly oriented on PostgreSQL extensions or it will be used also in
PosatgreSQL core?
My main motivation is
On 01/09/2014 09:46 PM, Claudio Freire wrote:
On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote:
It would be nice to have better operating system support for this.
For example, IIUC, 64-bit Linux has 128TB of address space available
for user processes. When you clone(),
On Thu, Jan 9, 2014 at 4:24 PM, knizhnik knizh...@garret.ru wrote:
On 01/09/2014 09:46 PM, Claudio Freire wrote:
On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote:
It would be nice to have better operating system support for this.
For example, IIUC, 64-bit Linux has
On 01/09/2014 11:09 PM, Amit Kapila wrote:
On Thu, Jan 9, 2014 at 12:21 AM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com wrote:
Well, right now we just
On Fri, Jan 10, 2014 at 1:00 AM, knizhnik knizh...@garret.ru wrote:
On 01/09/2014 11:09 PM, Amit Kapila wrote:
Using DuplicateHandle(), we can make segment stick for Postmaster
lifetime. I have used below test (used dsm_demo module) to verify:
As far as I understand DuplicateHandle() should
On 01/09/2014 11:30 PM, Claudio Freire wrote:
On Thu, Jan 9, 2014 at 4:24 PM, knizhnik knizh...@garret.ru wrote:
On 01/09/2014 09:46 PM, Claudio Freire wrote:
On Thu, Jan 9, 2014 at 2:22 PM, Robert Haas robertmh...@gmail.com wrote:
It would be nice to have better operating system support for
On Thu, Jan 9, 2014 at 4:39 PM, knizhnik knizh...@garret.ru wrote:
At fork time I only wrote about reserving the address space. After
reserving it, all you have to do is implement an allocator that works
in shared memory (protected by a lwlock of course).
In essence, a hypothetical
On Thu, Jan 9, 2014 at 4:48 PM, Claudio Freire klaussfre...@gmail.com wrote:
On Thu, Jan 9, 2014 at 4:39 PM, knizhnik knizh...@garret.ru wrote:
At fork time I only wrote about reserving the address space. After
reserving it, all you have to do is implement an allocator that works
in shared
On 01/09/2014 11:48 PM, Claudio Freire wrote:
On Thu, Jan 9, 2014 at 4:39 PM, knizhnik knizh...@garret.ru wrote:
At fork time I only wrote about reserving the address space. After
reserving it, all you have to do is implement an allocator that works
in shared memory (protected by a lwlock of
On 1/9/14, 1:18 PM, knizhnik wrote:
So it is clear why do we need shared memory for parallel query execution. But
why it has to be dynamic? Why it can not be preallocated at start time as most
of other resources used by PostgreSQL?
That would limit us to doing something like allocating a
On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Jan 6, 2014 at 4:04 PM, james ja...@mansionfamily.plus.com wrote:
The point remains that you need to duplicate it into every process that
On 01/08/2014 10:51 PM, Robert Haas wrote:
On Tue, Jan 7, 2014 at 10:20 PM, Amit Kapila amit.kapil...@gmail.com wrote:
On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Jan 6, 2014 at 4:04 PM, james ja...@mansionfamily.plus.com wrote:
The point remains that you
On Tue, Jan 7, 2014 at 2:46 AM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Jan 6, 2014 at 4:04 PM, james ja...@mansionfamily.plus.com wrote:
The point remains that you need to duplicate it into every process that
might
want to use it subsequently, so it makes sense to DuplicateHandle
On 06/01/2014 03:14, Robert Haas wrote:
That's up to the application. After calling dsm_create(), you call
dsm_segment_handle() to get the 32-bit integer handle for that
segment. Then you have to get that to the other process(es) somehow.
If you're trying to share a handle with a background
On 06/01/2014 04:20, Amit Kapila wrote:
Duplicate handle should work, but we need to communicate the handle
to other process using IPC.
Only if the other process needs to use it. The IPC is not to transfer
the handle to
the other process, just to tell it which slot in its handle table
On Mon, Jan 6, 2014 at 4:04 PM, james ja...@mansionfamily.plus.com wrote:
The point remains that you need to duplicate it into every process that
might
want to use it subsequently, so it makes sense to DuplicateHandle into the
parent, and then to advertise that handle value publicly so that
On Sat, Jan 4, 2014 at 3:27 PM, knizhnik knizh...@garret.ru wrote:
1. I want IMCS to work with PostgreSQL versions not supporting DSM (dynamic
shared memory), like 9.2, 9.3.1,...
Yeah. If it's loaded at postmaster start time, then it can work with
any version. On 9.4+, you could possibly make
On 05/01/2014 16:50, Robert Haas wrote:
But on Windows, segments are*automatically*
destroyed*by the operating system* when the last process unmaps them,
so it's not quite so clear to me how we can allow it there. The main
shared memory segment is no problem because the postmaster always has
On Sun, Jan 5, 2014 at 12:34 PM, james ja...@mansionfamily.plus.com wrote:
On 05/01/2014 16:50, Robert Haas wrote:
But on Windows, segments are *automatically*
destroyed *by the operating system* when the last process unmaps them,
so it's not quite so clear to me how we can allow it there.
From my point of view it is not a big problem that it is not possible
to place LWLock in DSM.
I can allocate LWLocks in standard way - using RequestAddinLWLocks and
use them for synchronization.
Concerning support of huge pages - actually I do not think that it
should involve something more
On 05/01/2014 18:02, Robert Haas wrote:
On Sun, Jan 5, 2014 at 12:34 PM, jamesja...@mansionfamily.plus.com wrote:
On 05/01/2014 16:50, Robert Haas wrote:
But on Windows, segments are*automatically*
destroyed*by the operating system* when the last process unmaps them,
so it's not quite so
On Sun, Jan 5, 2014 at 1:28 PM, knizhnik knizh...@garret.ru wrote:
From my point of view it is not a big problem that it is not possible to
place LWLock in DSM.
I can allocate LWLocks in standard way - using RequestAddinLWLocks and use
them for synchronization.
Sure, well, that works fine if
On Sun, Jan 5, 2014 at 1:44 PM, james ja...@mansionfamily.plus.com wrote:
I'm intrigued - how are the handles shared between children that are peers
in the current scheme? Some handle transfer must already be in place.
That's up to the application. After calling dsm_create(), you call
On Sun, Jan 5, 2014 at 11:04 PM, james ja...@mansionfamily.plus.com wrote:
On 05/01/2014 16:50, Robert Haas wrote:
But on Windows, segments are *automatically*
destroyed *by the operating system* when the last process unmaps them,
so it's not quite so clear to me how we can allow it there.
I'm sorry I misunderstood about the extension you wrote.
Is there some way not to use shared memory for it?
Cheers,
David.
On Sat, Jan 04, 2014 at 11:46:25AM +0400, knizhnik wrote:
Hi David,
Sorry, but I do not completely understand your suggestions:
1. IMCS really contains single patch
On 01/04/2014 12:05 PM, David Fetter wrote:
I'm sorry I misunderstood about the extension you wrote.
Is there some way not to use shared memory for it?
No, IMCS (In-Memory Columnar Store) is storing data in shared memory.
Certainly I could allocate shared memory myself, but due to portability
knizhnik knizh...@garret.ru writes:
On 01/04/2014 12:05 PM, David Fetter wrote:
Is there some way not to use shared memory for it?
No, IMCS (In-Memory Columnar Store) is storing data in shared memory.
It would probably be better if it made use of the dynamic shared memory
features that exist
On 01/04/2014 11:11 PM, Tom Lane wrote:
knizhnik knizh...@garret.ru writes:
On 01/04/2014 12:05 PM, David Fetter wrote:
Is there some way not to use shared memory for it?
No, IMCS (In-Memory Columnar Store) is storing data in shared memory.
It would probably be better if it made use of the
On Thu, Jan 02, 2014 at 08:48:24PM +0400, knizhnik wrote:
I want to announce implementation of In-Memory Columnar Store
extension for PostgreSQL.
Vertical representation of data is stored in PostgreSQL shared memory.
Thanks for the hard work!
I noticed a couple of things about this that
Hi David,
Sorry, but I do not completely understand your suggestions:
1. IMCS really contains single patch file sysv_shmem.patch.
Applying this patch is not mandatory for using IMCS: it just solves the
problem with support of 256Gb of shared memory.
Right now PostgreSQL is not able to use
38 matches
Mail list logo
