Re: JIT compiling with LLVM v9.0

Hi, On Fri, Jan 26, 2018 at 6:40 PM, Andres Freund wrote: >> I would like to see if we can get a combination of JIT and LTO to work >> together to specialize generic code at runtime. > > Well, LTO can't quite work. It relies on being able to mark code in > modules linked

Re: JIT compiling with LLVM v9.0

On Wed, Jan 24, 2018 at 11:02 PM, Andres Freund wrote: > Not entirely sure what you mean. You mean why I don't inline > slot_getsomeattrs() etc and instead generate code manually? The reason > is that the generated code is a *lot* smarter due to knowing the > specific

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 6:40 PM, Thomas Munro wrote: > Thanks for looking into this. Yeah. I think you're right that it > could add a bit of overhead in some cases (ie if you receive a lot of > signals that AREN'T caused by fork failure, then you'll enter >

Re: Removing WITH clause support in CREATE FUNCTION, for isCachable and isStrict

On Fri, Jan 26, 2018 at 12:30:12PM -0500, Tom Lane wrote: > And done. I failed to resist the temptation to rename > compute_attributes_sql_style, since the "sql_style" bit no longer > conveys anything. I'd always found compute_attributes_with_style > to be confusingly named --- seemed like it

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 7:30 PM, Peter Geoghegan wrote: > On Thu, Jan 25, 2018 at 10:00 PM, Amit Kapila wrote: >> However, now I see that you and Thomas are trying to find a different >> way to overcome this problem differently, so not sure if I should go

Re: Setting BLCKSZ 4kB

Hi, On 2018-01-27 00:28:07 +0100, Tomas Vondra wrote: > But does that make the internal page size relevant to the atomicity > question? For example, let's say we write 4kB on a drive with 2kB > internal pages, and the power goes out after writing the first 2kB of > data (so losing the second 2kB

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

I've pushed this mostly as-is. I fixed the missed places in reloptions code as we discussed. I also took out the parser changes related to allowing unquoted PARALLEL in old-style CREATE AGGREGATE, because that is not a goal I consider worthy of adding extra grammar complexity. We don't document

Re: Setting BLCKSZ 4kB

On 01/27/2018 12:06 AM, Andres Freund wrote: > Hi, > > On 2018-01-26 23:53:33 +0100, Tomas Vondra wrote: >> But more importantly, I don't see why the size of the internal page >> would matter here at all? SSDs have non-volatile write cache (DRAM with >> battery), protecting all the internal

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

> On 26 Jan 2018, at 23:58, Tom Lane wrote: > > Daniel Gustafsson writes: >>> On 26 Jan 2018, at 22:32, Tom Lane wrote: >>> I notice that there are two reloptions-related >>> "pg_strncasecmp" calls that did not get converted to

Re: Setting BLCKSZ 4kB

Hi, On 2018-01-26 23:53:33 +0100, Tomas Vondra wrote: > But more importantly, I don't see why the size of the internal page > would matter here at all? SSDs have non-volatile write cache (DRAM with > battery), protecting all the internal writes to pages. If your SSD does > not do that correctly,

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

Hello Doug, Patch applies, compiles, tests ok. > [...] Replaced USE_PPOLL with HAVE_PPOLL as having both seems redundant. I'm okay with that. I'm wondering whether there should be a way to force using one or the other when both are available. Not sure. Added option to force use of

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

Daniel Gustafsson writes: >> On 26 Jan 2018, at 22:32, Tom Lane wrote: >> I notice that there are two reloptions-related >> "pg_strncasecmp" calls that did not get converted to "strncmp": >> reloptions.c:804 > The way I read transformRelOptions(), oldOptions

Re: Setting BLCKSZ 4kB

On 01/26/2018 02:56 PM, Bruce Momjian wrote: > On Wed, Jan 17, 2018 at 02:10:10PM +0100, Fabien COELHO wrote: >> >> Hello, >> >>> What are the cons of setting BLCKSZ as 4kB? When saw the results published >>> on [...]. >> >> There were other posts and publications which points to the same

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

> On 26 Jan 2018, at 22:32, Tom Lane wrote: > > Michael Paquier writes: >> On Wed, Jan 24, 2018 at 09:47:50AM +0100, Daniel Gustafsson wrote: >>> Attached is a rebased v7 patch which has your amendments (minus >>> propname) which passes make check

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

Michael Paquier writes: > On Wed, Jan 24, 2018 at 09:47:50AM +0100, Daniel Gustafsson wrote: >> Attached is a rebased v7 patch which has your amendments (minus >> propname) which passes make check without errors. > Confirmed. I am switching the status as ready for

Re: [HACKERS] why not parallel seq scan for slow functions

On Tue, Jan 2, 2018 at 6:38 AM, Amit Kapila wrote: > [ new patch ] I think that grouping_planner() could benefit from a slightly more extensive rearrangement. With your patch applied, the order of operations is: 1. compute the scan/join target 2. apply the scan/join

Re: [HACKERS] Partition-wise aggregation/grouping

On Thu, Jan 18, 2018 at 8:55 AM, Jeevan Chalke wrote: > Attached patch with other review points fixed. Committed 0001 and 0002 together, with some cosmetic changes, including fixing pgindent damage. Please pgindent your patches before submitting. -- Robert Haas

Re: JIT compiling with LLVM v9.0

Hi, On 2018-01-26 13:06:27 +0300, Konstantin Knizhnik wrote: > One more question: do you have any idea how to profile JITed code? Yes ;). It depends a bit on what exactly you want to do. Is it sufficient to get time associated with the parent caller, or do you need instruction-level access. >

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 11:17 AM, Robert Haas wrote: > Hmm, I like the idea of making it a #define instead of having it > depend on parallel_leader_participation. Let's do that. If the > consensus is later that it was the wrong decision, it'll be easy to > change it back.

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 2:04 PM, Peter Geoghegan wrote: > On Fri, Jan 26, 2018 at 10:33 AM, Robert Haas wrote: >> I'm busy with other things, so no rush. > > Got it. > > There is one question that I should probably get clarity on ahead of > the next revision,

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 10:33 AM, Robert Haas wrote: > I'm busy with other things, so no rush. Got it. There is one question that I should probably get clarity on ahead of the next revision, which is: Should I rip out the code that disallows a "degenerate parallel CREATE

Re: \describe*

> > It would be about as hard to memorize \describe-schemas as it is to > memorize \dn: > You'd have to remember that it is "-" and not "_", that it is "describe", > not "desc" > and that it is "schemas", not "schema". > You wouldn't memorize them. You'd discover them with tab completion. Type

Re: unique indexes on partitioned tables

On 1/22/18 17:55, Alvaro Herrera wrote: > Alvaro Herrera wrote: >> Version 4 of this patch, rebased on today's master. + if (key->partattrs[i] == 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), +

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Jan 26, 2018 at 11:56 AM, Stephen Frost wrote: > >> I think you've chosen a terrible design and ought to throw the whole > >> thing away and start over. > > > > I'll all for throwing away the existing test once

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 1:17 PM, Peter Geoghegan wrote: > On Fri, Jan 26, 2018 at 10:01 AM, Robert Haas wrote: >> On Fri, Jan 26, 2018 at 1:30 AM, Peter Geoghegan wrote: >>> I had imagined that WaitForParallelWorkersToAttach() would give me an

Re: [Sender Address Forgery]Re: pg_(total_)relation_size and partitioned tables

On Fri, Jan 26, 2018 at 7:45 AM, Michael Paquier wrote: > There could be value in having a version dedicated to inheritance trees > as well, true enough. As well as value in having something that shows > both. Still let's not forget that partition sets are structured

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 10:01 AM, Robert Haas wrote: > On Fri, Jan 26, 2018 at 1:30 AM, Peter Geoghegan wrote: >> I had imagined that WaitForParallelWorkersToAttach() would give me an >> error in the style of WaitForParallelWorkersToFinish(), without >>

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Jan 26, 2018 at 1:30 AM, Peter Geoghegan wrote: > I had imagined that WaitForParallelWorkersToAttach() would give me an > error in the style of WaitForParallelWorkersToFinish(), without > actually waiting for the parallel workers to finish. +1. If we're going to go that

Re: [HACKERS] generated columns

On Thu, Jan 25, 2018 at 10:26 PM, Peter Eisentraut wrote: >> Does the SQL spec mention the matter? How do other systems >> handle such cases? > > In Oracle you get the same overflow error. That seems awful. If a user says "SELECT * FROM tab" and it fails, how

Re: Removing WITH clause support in CREATE FUNCTION, for isCachable and isStrict

On Fri, Jan 26, 2018 at 12:30 PM, Tom Lane wrote: > Michael Paquier writes: >> On Thu, Jan 25, 2018 at 06:30:04PM -0500, Tom Lane wrote: >>> In short, I'm on board with removing the WITH clause. I've not >>> reviewed the patch in detail, but will

Re: Removing WITH clause support in CREATE FUNCTION, for isCachable and isStrict

Michael Paquier writes: > On Thu, Jan 25, 2018 at 06:30:04PM -0500, Tom Lane wrote: >> In short, I'm on board with removing the WITH clause. I've not >> reviewed the patch in detail, but will do so and push it if there's >> not objections pretty soon. > Glad to hear

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

On Fri, Jan 26, 2018 at 11:56 AM, Stephen Frost wrote: >> I think you've chosen a terrible design and ought to throw the whole >> thing away and start over. > > I'll all for throwing away the existing test once we've got something > that covers at least what it does (ideally

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Robert Haas writes: > I figured you would, but it's still my opinion. I guess my basic > objection here is to the idea that we somehow know that the 6000+ line > test case file actually contains only correct tests. That vastly > exceeds the ability of any normal human

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Jan 26, 2018 at 11:09 AM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> On Thu, Jan 25, 2018 at 4:52 PM, Alvaro Herrera > >> wrote: > >> > My proposal is

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

On Fri, Jan 26, 2018 at 11:09 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Jan 25, 2018 at 4:52 PM, Alvaro Herrera >> wrote: >> > My proposal is that instead of looking at three hundred lines, you'd >> > look for

Re: Boolean partitions syntax

Robert Haas writes: > On Fri, Jan 26, 2018 at 11:07 AM, Stephen Frost wrote: >> I've already had two people mention that it'd be neat to have PG support >> it, so I don't believe it'd go unused. As for if we should force people >> to use quotes, my

Re: Boolean partitions syntax

On Fri, Jan 26, 2018 at 11:07 AM, Stephen Frost wrote: > I've already had two people mention that it'd be neat to have PG support > it, so I don't believe it'd go unused. As for if we should force people > to use quotes, my vote would be no because we don't require that for >

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Jan 25, 2018 at 4:52 PM, Alvaro Herrera > wrote: > > My proposal is that instead of looking at three hundred lines, you'd > > look for 50 lines of `pg_restore -l` output -- is element XYZ in there > > or

Re: Boolean partitions syntax

Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Jan 25, 2018 at 8:44 PM, Amit Langote > wrote: > > Attached updated patch. > > I wonder if this patch is just parser bloat without any real benefit. > It can't be very common to want to partition on a

Re: [HACKERS] Custom compression methods

Hello Ildus, I continue reviewing your patch. Here are some thoughts. 1. When I set column storage to EXTERNAL then I cannot set compression. Seems reasonable: create table test(id serial, msg text); alter table test alter column msg set storage external; alter table test alter column msg set

Re: Documentation of pgcrypto AES key sizes

On Thu, Jan 25, 2018 at 8:19 PM, Michael Paquier wrote: > On Fri, Jan 26, 2018 at 12:33:41PM +1300, Thomas Munro wrote: >> I noticed that the documentation for encrypt()/decrypt() says "aes — >> AES (Rijndael-128)", but in fact 192 and 256 bit keys are also >>

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

On Thu, Jan 25, 2018 at 4:52 PM, Alvaro Herrera wrote: > My proposal is that instead of looking at three hundred lines, you'd > look for 50 lines of `pg_restore -l` output -- is element XYZ in there > or not. Quite a bit simpler for the guy adding a new test. This tests

Re: Boolean partitions syntax

On Thu, Jan 25, 2018 at 8:44 PM, Amit Langote wrote: > Attached updated patch. I wonder if this patch is just parser bloat without any real benefit. It can't be very common to want to partition on a Boolean column, and if you do, all this patch does is let you drop

Re: Redefining inet_net_ntop

Emre Hasegeli writes: >> port.h declares inet_net_ntop and we always compile our own from >> port/inet_net_ntop.c . > There is another copy of it under backend/utils/adt/inet_cidr_ntop.c. > The code looks different but does 90% the same thing. Their naming > and usage is

Re: Updating timezone data to 2018c

Michael Paquier writes: > I have just bumped into tzdata (https://www.iana.org/time-zones), to > notice that 2018c has been released. Surely, there will be a refresh for > the next release? Yeah, it's on my to-do list for next week. > At the same time I have played

Re: Invalid result from hash_page_items function

On Thu, Jan 25, 2018 at 4:50 PM, Masahiko Sawada wrote: > This appears at PostgreSQL 10 and current HEAD. The cause of this > seems that hash_page_items allocates the memory space for the page > before switching memory context. AFAICS there is no similar problem in >

Re: \describe*

On 01/26/2018 03:49 PM, David Fetter wrote:> They are indeed terse and cryptic, and what's worse, they're not available to clients other than psql, so I propose that we do what at least MySQL, Oracle, and DB2 do and implement DESCRIBE as its own command. Especially handy would be a variant

Re: JIT compiling with LLVM v9.0

On Thu, Jan 25, 2018 at 11:20:28AM -0800, Andres Freund wrote: > On 2018-01-25 18:40:53 +0300, Konstantin Knizhnik wrote: > > Another question is whether it is sensible to redundantly do > > expensive work (llvm compilation) in all backends. > > Right now we kinda have to, but I really want to

Re: \describe*

On Thu, Jan 25, 2018 at 08:11:00PM -0500, Corey Huinker wrote: > Some of the discussions about making psql more user friendly (more > tab completions help, exit, etc) got me thinking about other ways > that psql could be more friendly, and the one that comes to mind is > our terse but cryptic \d*

Re: Should we introduce a "really strict" volatility about return values?

On Thu, Jan 25, 2018 at 6:51 PM, Andres Freund wrote: > I don't plan to work on this anytime soon, but I though it's interesting > enough to put out there and see what others think. I mean, if it buys enough performance, it's probably worth doing. Sort of annoying to have to

Updating timezone data to 2018c

Hi all, I have just bumped into tzdata (https://www.iana.org/time-zones), to notice that 2018c has been released. Surely, there will be a refresh for the next release? At the same time I have played with the instructions in src/timezone/README to generate the attached. That's always an

Re: PATCH: Exclude unlogged tables from base backups

On Wed, Jan 24, 2018 at 1:25 PM, David Steele wrote: > I think you mean DEBUG1? It's already at DEBUG2. > > I considered using DEBUG1 but decided against it. The other exclusions > will produce a limited amount of output because there are only a few of > them. In the case

Re: [HACKERS] WIP: Aggregation push-down

On Fri, Jan 26, 2018 at 8:04 AM, Antonin Houska wrote: > So one problem is that the grouping expression can be inappropriate for > partial aggregation even if there's no type change during the > translation. What I consider typical for this case is that the equality > operator

Re: Setting BLCKSZ 4kB

On Wed, Jan 17, 2018 at 02:10:10PM +0100, Fabien COELHO wrote: > > Hello, > > >What are the cons of setting BLCKSZ as 4kB? When saw the results published > >on [...]. > > There were other posts and publications which points to the same direction > consistently. > > This matches my deep belief

Configuring messages language on Windows

Hello hackers, As it mentioned in pg_locale.c, the variable LC_MESSAGES is ignored in Windows(pg_locale.c:162). In other systems, this variable is used to select a messages language. But in Windows, the language is selected based on system locale and couldn't be changed via configuration.

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

On Thu, Jan 25, 2018 at 8:54 PM, Tsunakawa, Takayuki wrote: > Yes, that's pg_test_fsync output. Isn't pg_test_fsync the tool to determine > the value for wal_sync_method? Is this manual misleading? Hmm. I hadn't thought about it as misleading, but now that you

Re: [HACKERS] WIP: Aggregation push-down

Robert Haas wrote: > On Fri, Dec 22, 2017 at 10:43 AM, Antonin Houska wrote: > > Michael Paquier wrote: > >> On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska wrote: > >> > I'm not about to add any other

Re: AS OF queries

On Fri, Jan 26, 2018 at 10:56:06AM +0300, Konstantin Knizhnik wrote: > >>Yeh, I suspected that just disabling autovacuum was not enough. > >>I heard (but do no know too much) about microvacuum and hot updates. > >>This is why I was a little bit surprised when me test didn't show lost of >

Re: [Sender Address Forgery]Re: pg_(total_)relation_size and partitioned tables

On Fri, Jan 26, 2018 at 07:00:43PM +0900, Amit Langote wrote: > I wonder what pg_partition_tree_tables() should return when passed a table > that doesn't have partitions under it? Return a 1-member set containing > itself? Yes. A table alone is itself part of a partition set, so the result

Re: list partition constraint shape

(2018/01/26 10:15), Amit Langote wrote: On 2018/01/25 21:17, Etsuro Fujita wrote: Some minor comments: + /* +* Construct an ArrayExpr for the non-null partition +* values +*/ + arrexpr =

Re: [HACKERS] [PATCH] Lockable views

On Thu, 25 Jan 2018 20:51:41 +1300 Thomas Munro wrote: > On Sun, Dec 31, 2017 at 11:57 PM, Yugo Nagata wrote: > > On Fri, 29 Dec 2017 23:39:39 +0900 (JST) > > Tatsuo Ishii wrote: > >> Your addition to the doc: > >> +

Re: STATISTICS retained in CREATE TABLE ... LIKE (INCLUDING ALL)?

Moin, On Fri, January 26, 2018 2:30 am, David Rowley wrote: > On 21 January 2018 at 19:21, David Rowley > wrote: >> On 20 January 2018 at 18:50, Tom Lane wrote: >>> Stephen Froehlich writes: Are custom statistics

Re: \describe*

Corey Huinker wrote: > Some of the discussions about making psql more user friendly (more tab > completions help, exit, etc) got me thinking about other ways that psql could > be more friendly, and the one that comes to mind is our terse but cryptic \d* > commands. > > I think it would be

Re: Redefining inet_net_ntop

> port.h declares inet_net_ntop and we always compile our own from > port/inet_net_ntop.c . There is another copy of it under backend/utils/adt/inet_cidr_ntop.c. The code looks different but does 90% the same thing. Their naming and usage is confusing. I recently needed to format IP addresses

Re: JIT compiling with LLVM v9.0

On 26.01.2018 11:23, Andres Freund wrote: Hi, Thanks for testing things out! Thank you for this work. One more question: do you have any idea how to profile JITed code? There is no LLVMOrcRegisterPerf in LLVM 5, so jit_profiling_support option does nothing. And without it perf is not able

Re: [Sender Address Forgery]Re: pg_(total_)relation_size and partitioned tables

On 2018/01/22 11:44, Michael Paquier wrote: > On Sun, Jan 21, 2018 at 07:16:38PM +1300, David Rowley wrote: >> On 20 January 2018 at 23:14, Michael Paquier >> wrote: >>> If pg_partition_tree_tables() uses the top of the partition as input, >>> all the tree should show

relispartition for index partitions

Hi. I noticed that relispartition isn't set for index's partitions. create table p (a int) partition by list (a); create table p12 partition of p for values in (1, 2); create index on p (a); select relname, relkind from pg_class where relnamespace = 'public'::regnamespace and relispartition is

Re: JIT compiling with LLVM v9.0

Hi, Thanks for testing things out! On 2018-01-26 10:44:24 +0300, Konstantin Knizhnik wrote: > Also I noticed that parallel execution didsables JIT. Oh, oops, I broke that recently by moving where the decisition about whether to jit or not is. There actually is JITing, but only in the leader.

Rewriting the test of pg_upgrade as a TAP test - take two

Hi all, As promised on a recent thread, here is a second tentative to switch pg_upgrade's test.sh into a TAP infrastructure. This is a continuation of the following thread: https://www.postgresql.org/message-id/CAB7nPqRdaN1A1YNjxNL9T1jUEWct8ttqq29dNv8W_o37%2Be8wfA%40mail.gmail.com To begin