Re: [HACKERS] xeon processors
Christopher Browne [EMAIL PROTECTED] writes: Centuries ago, Nostradamus foresaw when [EMAIL PROTECTED] (Jaime Casanova) would write: Can anyone tell me if postgresql has problems with xeon processors? If so, there is any fix or project of fix it? Well, there's a known issue that IA-32 systems with more than 4GB of memory use an extension known as PAE to bank-switch between the banks of memory. AIUI, it's not really bank switching in the old 8-bit sense. Rather, there is a big linear 36-bit physical address space, and the processor's page tables have been extended so they can point to any page in this space. Any one process still sees a 4GB (32-bit) address space since that's how big the registers are. Any time you switch banks, there's a fair little bit of work to be done. That includes multitasking systems that need to context switch a few thousand times per second. I don't think this is any more overhead than a normal context switch--cache misses, TLB flush etc. The fix for this problem is to rewrite all of your applications so that they become conscious of which bits of memory they're using so they can tune their own behaviour. This, of course, requires discarding useful notions such as virtual memory that are _assumed_ by most modern operating systems. This is only if you need to address more than 32-bits worth of address in a single process, which is not always the case on server-class systems (on scientific/numerical workloads, it's often a big win). In which case you are certainly right: The fix is to buy hardware that hasn't been hacked up so badly. 64-bit systems get cheaper all the time... :) -Doug ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] xeon processors
Christopher Browne wrote: The fix for this problem is to rewrite all of your applications so that they become conscious of which bits of memory they're using so they can tune their own behaviour. This, of course, requires discarding useful notions such as virtual memory that are _assumed_ by most modern operating systems. This is misleading: PAE means that a 32-bit cpu can have more that 4 GB physical memory. Each process can map at most 4 (in reality: ~2) GB memory. Many databases manage their own, huge buffer pool and read/write the database tables with O_DIRECT. These apps must support buffer pools 2 GB, which requires some work. Linux and Solaris contain a special syscall that helps Oracle to manage it's buffer pool for such setups (remap_page_rage()). OTHO postgres has a small user space buffer pool, the majority of the file buffers are handled by OS. Thus no changes are required inside postgres for PAE, all it needs is an OS that support PAE for the buffer pool. Regarding hyperthreading: I'm aware of two changes: - busy loops must contain PAUSE instructions. Postgres does that. - virtual aliases should be avoided: If two processes access memory at the same virtual address, then this can cause cache collisions and then misses. I think this is handled by the C library by randomizing the return addresses of malloc() and Intel mitigated the issue by improving the cache. -- Manfred ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] xeon processors
Centuries ago, Nostradamus foresaw when [EMAIL PROTECTED] (Jaime Casanova) would write: Can anyone tell me if postgresql has problems with xeon processors? If so, there is any fix or project of fix it? Well, there's a known issue that IA-32 systems with more than 4GB of memory use an extension known as PAE to bank-switch between the banks of memory. Any time you switch banks, there's a fair little bit of work to be done. That includes multitasking systems that need to context switch a few thousand times per second. The fix for this problem is to rewrite all of your applications so that they become conscious of which bits of memory they're using so they can tune their own behaviour. This, of course, requires discarding useful notions such as virtual memory that are _assumed_ by most modern operating systems. The fix is to buy hardware that hasn't been hacked up so badly. -- select 'cbbrowne' || '@' || 'ntlug.org'; http://www3.sympatico.ca/cbbrowne/lsf.html There is something in the lecture course which may not have been visible so far, which is reality ... -- Arthur Norman ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] xeon processors
On Sat, Jun 26, 2004 at 07:31:59PM -0600, Scott Marlowe wrote: On Fri, 2004-06-25 at 14:13, Jaime Casanova wrote: Hi all, Can anyone tell me if postgresql has problems with xeon processors? If so, there is any fix or project of fix it? To PostgreSQL, there's no difference between a dual CPU machine with no hyperthreading, and a single CPU machine with hyperthreading. At some point there was trouble with spinlocks on some of the newer Xeons (maybe those with hyperthreading?). I think there was a good deal of discussion and resulting development because of that. According to my archives it seems to be around december 2003. There's even a CVS entry, listed by cvs2cl as 2003-12-27 17:58 tgl * src/: backend/storage/lmgr/s_lock.c, include/storage/s_lock.h: Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom. Not sure if this made the 7.4.3 release ... -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) You knock on that door or the sun will be shining on places inside you that the sun doesn't usually shine (en Death: The High Cost of Living) ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] xeon processors
On Fri, 2004-06-25 at 14:13, Jaime Casanova wrote: Hi all, Can anyone tell me if postgresql has problems with xeon processors? If so, there is any fix or project of fix it? To PostgreSQL, there's no difference between a dual CPU machine with no hyperthreading, and a single CPU machine with hyperthreading. HOWEVER, there have been some issues with certain Operating Systems running slower with hyperthreading enabled than without it. Late model Linux kernels (2.6) seem to have gotten enough logic into the scheduler to get good performance on a hyperthreaded system. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[HACKERS] xeon processors
Hi all, Can anyone tell me if postgresql has problems with xeon processors? If so, there is any fix or project of fix it? Thanx in advance, Jaime Casanova Do You Yahoo!? Todo lo que quieres saber de Estados Unidos, América Latina y el resto del Mundo. Visíta Yahoo! Noticias.
Re: [HACKERS] xeon processors
Hello, I seem to recall that HyperThreading and PostgreSQL != good stuff... There was a whole bunch of stuff recently on this... google the archives. Sincerely, Joshua D. Drake Jaime Casanova wrote: Hi all, Can anyone tell me if postgresql has problems with xeon processors? If so, there is any fix or project of fix it? Thanx in advance, Jaime Casanova *Do You Yahoo!?* http://espanol.yahoo.com/mail_tagline/*http://espanol.news.yahoo.com Todo lo que quieres saber de Estados Unidos, América Latina y el resto del Mundo. Visíta Yahoo! Noticias http://espanol.yahoo.com/mail_tagline/*http://espanol.news.yahoo.com. -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL begin:vcard fn:Joshua D. Drake n:Drake;Joshua D. org:Command Prompt, Inc. adr:;;PO Box 215;Cascade Locks;Oregon;97014;USA email;internet:[EMAIL PROTECTED] title:Consultant tel;work:503-667-4564 tel;fax:503-210-0034 note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl. x-mozilla-html:FALSE url:http://www.commandprompt.com/ version:2.1 end:vcard ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] xeon processors
thanx"Joshua D. Drake" [EMAIL PROTECTED] wrote: Hello,I seem to recall that HyperThreading and PostgreSQL != good stuff...There was a whole bunch of stuff recently on this... google the archives.Sincerely,Joshua D. DrakeJaime Casanova wrote: Hi all, Can anyone tell me if postgresql has problems with xeon processors? If so, there is any fix or project of fix it? Thanx in advance, Jaime Casanova *Do You Yahoo!?* Todo lo que quieres saber de Estados Unidos, América Latina y el resto del Mundo. Visíta Yahoo! Noticias .-- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBCPostgresql support, programming shared hosting and dedicated hosting.+1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.comMammoth PostgreSQL Replicator. Integrated Replication for PostgreSQLbegin:vcardfn:Joshua D. Draken:Drake;Joshua D.org:Command Prompt, Inc.adr:;;PO Box 215;Cascade Locks;Oregon;97014;USAemail;internet:[EMAIL PROTECTED]title:Consultanttel;work:503-667-4564tel;fax:503-210-0034note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl.x-mozilla-html:FALSEurl:http://www.commandprompt.com/version:2.1end:vcardDo You Yahoo!? Todo lo que quieres saber de Estados Unidos, América Latina y el resto del Mundo. Visíta Yahoo! Noticias.
