[HACKERS] Misleading bgworker log message
A handful of rather surprising errors showed up in our log extract this morning, along the lines of: 2017-03-17 05:01:55 CDT [5400]: [1-1] @ FATAL: 57P01: terminating connection due to administrator command After a moment of more than a little astonishment, a look at the full log revealed that, while the message implies some type of intervention from a human , what's actually happening is the termination of parallel query worker threads due to a statement timeout. I gather this is happening because the same mechanism is used to cancel the process in both cases. If that's indeed the case, I wonder if it's possible to re phrase the termination message so it does not suggest an admin was involved. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] removing tsearch2
There is also a mechanism for the results of the Perl module's "make test" to be reported to a site which aggregates and reports them by Perl version and OS - a sort of distributed build farm. See for example http://matrix.cpantesters.org/?dist=DBD-Pg+3.5.3 __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Mon, Feb 27, 2017 at 4:02 PM, David E. Wheeler wrote: > On Feb 27, 2017, at 1:53 PM, Bruce Momjian wrote: > > > Oh, does CPAN distribute compiled modules or requires users to compile > > them. > > Like PGXN, it formally does not care, but its implementation expects > source code distributions what will be built and installed by users. Note > that the vast majority of those modules, -- even pure Perl modules -- are > built with make. > > So users typically get their Perl modules in one of these ways: > > 1. As binaries from their distribution’s package manager. These tend to be > updated manually by volunteers and not integrated into CPAN, though there > are solutions such as [rpmcpan](https://github.com/iovation/rpmcpan) and > [PPM](http://www.activestate.com/activeperl/ppm-perl-modules) which do > regular distro package builds. > > 2. As source code from CPAN, from which they are compiled (when > necessary), built, and installed by the user or a build system such as > [Homebrew](https://brew.sh). > > Best, > > David > >
Re: [HACKERS] application_name in process name?
On Sun, Jul 17, 2016 at 10:34 AM, Tom Lane wrote: > It occurs to me that we could also remove the update_process_title GUC: > what you would do is configure a process_title pattern that doesn't > include the %-escape for current command tag, and the infrastructure > could notice that that escape isn't present and skip unnecessary updates. > The same kind of trick could be used for other potentially-expensive > items like the lock "waiting" flag. > This seems like an interesting project for learning my way around gucs and logging. Could you elaborate a little on the cost considerations?
[HACKERS] application_name in process name?
There are times when it would be useful to have the application_name connection parameter displayed in the process name - and thus in ps and pg_top - in addition to the user and database name. Would there be any downside to this? If it were done, are there any suggestions on how it could be added the process name so as to minimize impact on anything that might be trying to parse that line? __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.
On Thu, Nov 19, 2015 at 10:05 AM, Robert Haas wrote: > On Wed, Nov 18, 2015 at 10:21 AM, Alvaro Herrera > wrote: > > In my days of Perl, it was starting to become frowned upon to call > > subroutines without parenthesizing arguments. Is that no longer the > > case? > As I understand it, there are several reasons not to make function calls in Perl without parenthesis. Whether they are good reasons is a question for the user. Modern Perl <http://onyxneon.com/books/modern_perl/> chapter 5 covers most of them. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
[HACKERS] warning: HS_KEY redefined (9.5 beta2)
Google says this was present in beta1. ( http://www.postgresql.org/message-id/5596a162.30...@dunslane.net) Still seems broken, at least for me. Built with Perl 5.22. uname -m = x86_64 uname -r = 2.6.32-504.12.2.el6.x86_64 __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] Within CF app, "Bug Fixes" should be "Bug Fixes/Refactoring"
On Thu, Oct 29, 2015 at 2:45 PM, Michael Paquier wrote: > On Thu, Oct 29, 2015 at 8:33 PM, Peter Geoghegan wrote: > > I think that within the CF app, we should either rename the patch > > topic "Bug Fixes" to "Bug Fixes/Refactoring", or introduce a new > > "Refactoring" topic. I prefer the first approach. > > I would vote for the second approach, with a separate category for > refactoring. > > So if a bug fix involved some refactoring, which would you put it under? Or would you expect separate CF entries for the refactoring and the fix? ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] perlcritic
David wrote: > I believe there are ways to get perlcritic to keep quiet about things > we don't find relevant. Maybe that's a better way to use it. > There are indeed. A .perlcriticrc file can suppress (or add) either individual rules or groups of rules. I use one to ignore the ones I disagree with, along with the comment form to ignore specific cases. I see perlcritic as functioning for me along the same lines as a style guide, giving a consistency that helps with long term maintainability. It also helps keep me from straying into golf. ^_^
Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.
Something like that would be helpful. I just had to stop one after an hour and have no idea how much longer it would have taken. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Fri, Jul 24, 2015 at 5:41 PM, Josh Berkus wrote: > On 07/24/2015 11:06 AM, Jim Nasby wrote: > > On 7/23/15 5:18 AM, Thakur, Sameer wrote: > >> Hello, > >>> >logged > 25 times > >> Sorry, it is much lower at 7 times. Does not change overall point though > > > > I think it's related to the problem of figuring out how many dead tuples > > you expect to find in the overall heap, which you need to do to have any > > hope of this being a comprehensive estimate. > > What about just reporting scanned pages/total pages ? That would be > easy and cheap to track. It would result in some herky-jerky > "progress", but would still be an improvement over the feedback we don't > have now. > > -- > Josh Berkus > PostgreSQL Experts Inc. > http://pgexperts.com > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] pg_upgrade + Ubuntu
On Fri, Jul 10, 2015 at 2:10 PM, Alvaro Herrera wrote: > Joshua D. Drake wrote: > > > > On 07/10/2015 11:01 AM, Mike Blackwell wrote: > > >Does pg_config show the correct location? > > Good idea but: > > > > postgres@ly19:~$ pg_config > > You need to install postgresql-server-dev-X.Y for building a server-side > > extension or libpq-dev for building a client-side application. > > > > Which is worse having to install yet another package or having a command > > line option? > > It seems to me that this is a Debian packaging issue, not an upstream > issue, isn't it? If you want to fix the problem in this way, then > surely whatever package contains pg_upgrade should also contain > pg_config. > > Indeed. An interesting packaging choice. I'd think it belongs to an admin category along with pg_upgrade, pg_dump, etc., rather than a development package. Surely it could be useful for admin scripts? __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/>
Re: [HACKERS] pg_upgrade + Ubuntu
Does pg_config show the correct location? If so, perhaps pg_upgrade could get the .conf location the same way rather than requiring a command line option. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Fri, Jul 10, 2015 at 12:40 PM, Joshua D. Drake wrote: > > Hackers, > > Simple problem (I think): > > 9.4 version of pg_upgrade said: > > "/usr/lib/postgresql/9.4/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D > "/var/lib/postgresql/9.4/main" -o "-p 9400 -b -c synchronous_commit=off -c > fsync=off -c full_page_writes=off -c listen_addresses='' -c > unix_socket_permissions=0700 -c > unix_socket_directories='/var/lib/postgresql'" start > > postgres cannot access the server configuration file > "/var/lib/postgresql/9.4/main/postgresql.conf": No such file or directory > > The issue is Debian/Ubuntu etc... don't have a postgresql.conf in the > cluster. They keep it separately in /etc/postgresql. > > Could we get a flag that allows us to specifically point to where the conf > filesare? > > JD > -- > Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564 > PostgreSQL Centered full stack support, consulting and development. > Announcing "I'm offended" is basically telling the world you can't > control your own emotions, so everyone else should do it for you. > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
Ah. I see. Thanks for the clarification. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Tue, May 19, 2015 at 1:44 PM, Bruno Harbulot wrote: > > > On Tue, May 19, 2015 at 7:22 PM, Tom Lane wrote: > >> Mike Blackwell writes: >> > See for example >> > http://docs.oracle.com/cd/B19306_01/text.102/b14218/cqoper.htm#i997330, >> > Table 3-1, third row, showing the precedence of '?'. Further down the >> > page, under "Fuzzy" see "Backward Compatibility Syntax". >> >> If I'm reading that right, that isn't a SQL-level operator but an operator >> in their text search query language, which would only appear in SQL >> queries within string literals (compare tsquery's query operators in PG). >> So it wouldn't be a hazard for ?-substitution, as long as the substituter >> was bright enough to not change string literals. >> >> regards, tom lane >> > > That's how I read it too. I've tried this little test: > http://sqlfiddle.com/#!4/7436b/4/0 > > CREATE TABLE test_table ( > id INTEGER PRIMARY KEY, > name VARCHAR(100) > ); > > INSERT INTO test_table (id, name) VALUES (1, 'Nicole'); > INSERT INTO test_table (id, name) VALUES (2, 'Nicholas'); > INSERT INTO test_table (id, name) VALUES (3, 'Robert'); > INSERT INTO test_table (id, name) VALUES (4, 'Michael'); > INSERT INTO test_table (id, name) VALUES (5, 'Nicola'); > > CREATE INDEX idx_test_table_name ON test_table(name) INDEXTYPE IS > CTXSYS.CONTEXT; > > SELECT * FROM test_table WHERE CONTAINS(name, '?Nicolas', 1) > 0; > > > Fuzzy matching works indeed, but the question mark is part of the literal > (similarly to % when using LIKE). > > Best wishes, > > Bruno. > >
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
See for example http://docs.oracle.com/cd/B19306_01/text.102/b14218/cqoper.htm#i997330, Table 3-1, third row, showing the precedence of '?'. Further down the page, under "Fuzzy" see "Backward Compatibility Syntax". ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Tue, May 19, 2015 at 12:45 PM, Bruno Harbulot < br...@distributedmatter.net> wrote: > > > On Tue, May 19, 2015 at 6:15 PM, Mike Blackwell > wrote: > >> A Google search suggests Oracle 9.x supports a unary '?' operator (fuzzy >> match), so the use of '?' in an operator name is not without precedent. >> >> > Interesting. Do you have any specific link? I'm probably not using the > right Google search, but the nearest reference I've found is for Oracle 10, > and it seems to use the tilde (~) operator for fuzzy matching: > http://www.oracle.com/technetwork/search/oses/overview/new-query-features-in-10-1-8-2-1-132287.pdf > > Best wishes, > > Bruno. >
Re: [HACKERS] Problems with question marks in operators (JDBC, ECPG, ...)
A Google search suggests Oracle 9.x supports a unary '?' operator (fuzzy match), so the use of '?' in an operator name is not without precedent. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Tue, May 19, 2015 at 10:03 AM, Bruno Harbulot < br...@distributedmatter.net> wrote: > > > On Tue, May 19, 2015 at 3:23 PM, Kevin Grittner wrote: > >> David G. Johnston wrote: >> > On Mon, May 18, 2015 at 3:31 PM, Bruno Harbulot < >> br...@distributedmatter.net>wrote: >> >> >> In the discussion on the OpenJDK JDBC list two years ago >> >> ( >> http://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2013-February/50.html >> ), >> >> Lance Andersen said "There is nothing in the SQL standard that >> >> would support the use of an '?' as anything but a parameter >> >> marker.". >> >> > "CREATE OPERATOR is a PostgreSQL extension. There are no >> > provisions for user-defined operators in the SQL standard." >> >> Exactly. The standard specifies the characters to use for the >> predicates that it defines, and provides no mechanism for adding >> additional predicates; but who in the world would want to exclude >> all extensions to the standard? >> > > I was certainly not suggesting custom operators should be excluded. I was > suggesting using something that was actually not incompatible with the SQL > standards (and, even with standards aside, the expectations implementors > have regarding the question mark, since it affects other tools too). > > > >> > And by extension if indeed the standard does require the use of >> > "?" for parameters we are in violation there because the backend >> > protocol deals with $# placeholders and not "?" >> >> We're talking about a different specification that has question >> marks as parameter placeholders. That's in the Java Database >> Connector (JDBC) specification. (It is apparently also specified >> in other documents, although I'm not familiar enough with those to >> comment.) Note that it would create all sorts of pain if both the >> SQL statements and a connector issuing them used the same >> convention for substituting parameters; it is a *good* thing that >> plpgsql and SQL function definitions use a different convention >> than JDBC! >> > > Actually, we were not just talking about JDBC. I don't know the > specifications in details, but the SQL:201x (preliminary) documents linked > from > https://wiki.postgresql.org/wiki/Developer_FAQ#Where_can_I_get_a_copy_of_the_SQL_standards.3F > seem to have some information. The Foundation document (Section 4.25 > Dynamic SQL concepts) says that dynamic parameters are represented by a > question mark. > > In addition, the BNF grammar available at > http://www.savage.net.au/SQL/sql-2003-2.bnf.html#dynamic%20parameter%20specification > also says: > ::= > > I'm not familiar enough with these documents to know whether I'm missing > some context, but it would seem that the question mark is a reserved > character, beyond the scope of JDBC (at the very least, it seems > incompatible with Dynamic SQL and its implementation in ECPG). > > Best wishes, > > Bruno. > >
Re: [HACKERS] RangeType internal use
On Fri, Feb 13, 2015 at 3:13 PM, Jim Nasby wrote: > > If we exclude the issue of needing one or two oddball partitions for +/- > infinity, I expect that fixed sized partitions would actually cover 80-90% > of cases. That would not be true in our case. The data is not at all evenly distributed over the partitioning key. We would need something more like: values a, b, and c get their own partitions and everything else goes in partition d.
Re: [HACKERS] Proposal: knowing detail of config files via SQL
This would default to being available to superusers only, right? Details of the file system shouldn't be available to any random user. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Fri, Jan 30, 2015 at 9:50 AM, Sawada Masahiko wrote: > On Sat, Jan 31, 2015 at 12:24 AM, David Fetter wrote: > > On Fri, Jan 30, 2015 at 09:38:10PM +0900, Sawada Masahiko wrote: > >> On Tue, Jan 27, 2015 at 3:34 AM, Robert Haas > wrote: > >> > On Thu, Jan 22, 2015 at 5:19 PM, Tom Lane wrote: > >> >> David Johnston writes: > >> >>> On Thu, Jan 22, 2015 at 3:04 PM, Tom Lane > wrote: > >> >>>> Is that a requirement, and if so why? Because this proposal > doesn't > >> >>>> guarantee any such knowledge AFAICS. > >> >> > >> >>> The proposal provides for SQL access to all possible sources of > variable > >> >>> value setting and, ideally, a means of ordering them in priority > order, so > >> >>> that a search for TimeZone would return two records, one for > >> >>> postgresql.auto.conf and one for postgresql.conf - which are > numbered 1 and > >> >>> 2 respectively - so that in looking at that result if the > >> >>> postgresql.auto.conf entry were to be removed the user would know > that what > >> >>> the value is in postgresql.conf that would become active. > Furthermore, if > >> >>> postgresql.conf has a setting AND there is a mapping in an > #included file > >> >>> that information would be accessible via SQL as well. > >> >> > >> >> I know what the proposal is. What I am questioning is the use-case > that > >> >> justifies having us build and support all this extra mechanism. How > often > >> >> does anyone need to know what the "next down" variable value would > be? > >> > > >> > I believe I've run into situations where this would be useful. I > >> > wouldn't be willing to put in the effort to do this myself, but I > >> > wouldn't be prepared to vote against a high-quality patch that someone > >> > else felt motivated to write. > >> > > >> > >> Attached patch adds new view pg_file_settings (WIP version). > >> This view behaves like followings, > >> [postgres][5432](1)=# select * from pg_file_settings ; > >> name| setting | > >> sourcefile > >> > ++ > >> max_connections| 100| > >> /home/masahiko/pgsql/master/3data/postgresql.conf > >> shared_buffers | 128MB | > >> /home/masahiko/pgsql/master/3data/postgresql.conf > > > > This looks great! > > > > Is there a reason not to have the sourcefile as a column in > > pg_settings? > > > > pg_file_settings view also has source file column same as pg_settings. > It might was hard to confirm that column in previous mail. > I put example of pg_file_settings again as follows. > > [postgres][5432](1)=# select * from pg_file_settings where name = > 'work_mem'; > -[ RECORD 1 ]-- > name | work_mem > setting| 128MB > sourcefile | /home/masahiko/pgsql/master/3data/hoge.conf > -[ RECORD 2 ]-- > name | work_mem > setting| 64MB > sourcefile | /home/masahiko/pgsql/master/3data/postgresql.auto.conf > > Regards, > > --- > Sawada Masahiko > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] [PATCH] explain sortorder
Tom, Thanks for the comments on what you ended up changing. It helps point out the kind of things I should be looking for. I'll try to let less slip through in the future. Mike ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Mon, Jan 19, 2015 at 10:09 AM, Tom Lane wrote: > "Timmer, Marius" writes: > > We think, you wanted to switch to DESC behavior > > (print out NULLS FIRST) in cases, where > > „USING“ uses an operator which is considered to be > > a DESC operator. > > Right, because that's how addTargetToSortList() would parse it. > > > But get_equality_op_for_ordering_op is called in > > cases, where reverse is false, but > > the part > > if (reverse) > > *reverse = (strategy == BTGreaterStrategyNumber); > > never changes this to true? > > Sorry, not following? It's true that what I added to explain.c doesn't > worry too much about the possibility of get_ordering_op_properties() > failing --- that really shouldn't happen for something that was previously > accepted as a sorting operator. But if it does, "reverse" will just be > left as false, so the behavior will anyway be unsurprising I think. > We could alternatively make it throw a "cache lookup failed" error but > I'm not sure how that makes anyone's life better. > > regards, tom lane > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] [PATCH] explain sortorder (fwd)
> > > From: "Timmer, Marius" > > Hi, > > attached is version 8, fixing remaining issues, adding docs and tests as > requested/agreed. > > > Marius & Arne > > This looks good to me. Test coverage seems complete. Doc updates are included. Output format looks like it should be acceptable to Heikki. I'll mark this as ready for committer. Thanks for the patch! Mike
Re: [HACKERS] [PATCH] explain sortorder
V6 of this patch applies, builds and checks against the current HEAD. The areas below could use some attention. In explain.c: malloc() should not be called directly here. palloc() would be the correct call, I believe, but the functions in stringinfo.h are probably your best choice as they remove the necessity for dealing with buffer size and overflow. There is leftover commented out code from the previous patch version in the T_Sort case. In show_sort_group_keys(), the splitting of the existing declaration and initialization of the keyresno and target seems unnecessary and against the style of surrounding code. Multi-line comments should follow the existing format. There are no tests for the "... is LC_COLLATE" and "COLLATE..." cases. Section 14.1 of the documentation may need to be updated. Mike. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Wed, Jan 7, 2015 at 10:17 AM, Timmer, Marius < marius.tim...@uni-muenster.de> wrote: > Hi, > > we have spent the last days to realize your suggestions in the patch. > It affects the result of a EXPLAIN-Statement (even in non-verbose-mode). > Now you will get the order-information for every single sort-key which is > not ordered by the defaults. > > > best regards, > > Marius > > > > > --- > Marius Timmer > Zentrum für Informationsverarbeitung > Westfälische Wilhelms-Universität Münster > Einsteinstraße 60 > > mtimm...@uni-muenster.de >
Re: [HACKERS] [PATCH] explain sortorder
Looking forward to the new patch. I'll give it a more complete testing when you post it. Mike ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Tue, Dec 23, 2014 at 5:11 AM, Arne Scheffer < arne.schef...@uni-muenster.de> wrote: > >Heikki Linnakangas writes: > >> I would suggest just adding the information to the Sort Key line. As > >> long as you don't print the modifiers when they are defaults (ASC and > >> NULLS LAST), we could print the information even in non-VERBOSE mode. > > >+1. I had assumed without looking that that was what it did already, > >else I'd have complained too. > > > regards, tom lane > > We will change the patch according to Heikkis suggestions. > > A nice Christmas & all the best in the New Year > > Arne Scheffer > > http://www.uni-muenster.de/ZIV/Mitarbeiter/ArneScheffer.shtml >
Re: [HACKERS] Commitfest problems
On Tue, Dec 16, 2014 at 10:15 AM, Mark Cave-Ayland < mark.cave-ayl...@ilande.co.uk> wrote: > > Well as I mentioned in my last email, practically all developers will > rebase and run "make check" on their patched tree before submitting to > the list. Even when this is true, and with people new to the project submitting patches I'm not sure it can be assumed, time passes and things can change between submission and review. I've seen a fair number of "Needs rebase" comments on patches, through no fault of the original submitter.
Re: [HACKERS] [PATCH] explain sortorder
Initial review:
Patch applies cleanly to current head, although it appears to have
soft/hard tab and trailing space issues.
make check fails with the output below. The expected collation clause is
not present.
--
-- Test explain feature: sort order
--
CREATE TABLE sortordertest (n1 char(1), n2 int4);
-- Insert values by which should be ordered
INSERT INTO sortordertest(n1, n2) VALUES ('d', 5), ('b', 3), ('a', 1),
('e', 2), ('c', 4);
-- Display sort order when explain analyze and verbose are true.
EXPLAIN (VERBOSE, COSTS OFF) SELECT * FROM sortordertest ORDER BY n1
COLLATE "C" DESC, n2;
QUERY PLAN
Sort
Output: n1, n2, ((n1)::character(1))
Sort Key: sortordertest.n1, sortordertest.n2
Sort Order: ASC NULLS LAST, ASC NULLS LAST
-> Seq Scan on public.sortordertest
Output: n1, n2, n1
(6 rows)
DROP TABLE sortordertest;
______
*Mike Blackwell | Technical Analyst, Distribution Services/Rollout
Management | RR Donnelley*
1750 Wallace Ave | St Charles, IL 60174-3401
Office: 630.313.7818
mike.blackw...@rrd.com
http://www.rrdonnelley.com
Re: [HACKERS] proposal: plpgsql - Assert statement
> > >> On 18 November 2014 21:19, Petr Jelinek wrote: >> >> Personally, I see this as natural extension of the conditional block >>> control >>> which we already have for loops with CONTINUE WHEN and EXIT WHEN. This >>> basically extends it to any block and it seems quite natural to have it >>> for >>> me... >>> >> This seems to me like a step in the direction of APL, where every statement is a conditional. Perl has the option of putting the conditional on the end of a statement as suggested here for ASSERT. My experience has been that while it may "look prettier" to some, the conditional is overlooked in reviews, etc., more often than one would expect, giving a net loss in the overall risk/productivity analysis. As a code maintainer, I would be opposed to adding something like this for no other reason than perceived aesthetics. Your mileage may, of course, vary.
Re: [HACKERS] Congrats Andres Freund, the newest PostgreSQL Commiter!
Congrats Andres! Mike __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Thu, May 22, 2014 at 9:24 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > Hi All, > > At the Developer Meeting that occurred 21th May 2014 was announced a new > PostgreSQL commiter [1], Mr. Andres Freund. > > I had the opportunity to work and be mentored by him. He deserves very > much this confidence, for the excellent work that has been doing for the > community. > > Thank you and Congrats Andres! > > > [1] > https://wiki.postgresql.org/wiki/PgCon_2014_Developer_Meeting#New_Committer > > -- > Fabrízio de Royes Mello > Consultoria/Coaching PostgreSQL > >> Timbira: http://www.timbira.com.br > >> Blog sobre TI: http://fabriziomello.blogspot.com > >> Perfil Linkedin: http://br.linkedin.com/in/fabriziomello > >> Twitter: http://twitter.com/fabriziomello >
Re: [HACKERS] tds_fdw for Sybase and MS SQL Server
Excellent! I have an application for this. I'll give it a look. Thanks! Mike ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Sun, Apr 6, 2014 at 9:03 AM, Geoff Montee wrote: > If anyone is interested, I've developed a foreign data wrapper that can be > used to connect to Sybase databases and Microsoft SQL Server. You can get > it on GitHub here: > > https://github.com/GeoffMontee/tds_fdw > > I did my testing with FreeTDS, an open source TDS library. > > I've talked to Greg Smith about this in the past. I don't really have the > expertise or resources to develop and test this much more than I already > have. If anyone likes it and would like to take over "ownership" of the > code, feel free to do so. > > I actually developed this over a year ago, so it doesn't support write > operations added in PostgreSQL 9.3. > > By the way, thanks to everyone who spoke at PGConf NYC 2014. It was very > informative. > > Geoff Montee > >
[HACKERS] 9.3.2 Client-only installation per docs fails creating directory.
Per the docs (15.4 Installation Procedure, Client-only installation), after running make, the following should work: gmake -C src/bin installgmake -C src/include installgmake -C src/interfaces installgmake -C doc install However, it fails on the first command: [postgresql-9.3.2]$ sudo gmake -C src/bin install gmake: Entering directory `/opt/postgresql-9.3.2/src/bin' gmake -C initdb install gmake[1]: Entering directory `/opt/postgresql-9.3.2/src/bin/initdb' gmake -C ../../../src/port all [successful stuff cut here] /bin/mkdir -p '/usr/local/pgsql-9.3.2/bin' /usr/bin/install -c psql '/usr/local/pgsql-9.3.2/bin/psql' /usr/bin/install -c -m 644 ./psqlrc.sample '/usr/local/pgsql-9.3.2/share/psqlrc.sample' /usr/bin/install: cannot create regular file `/usr/local/pgsql-9.3.2/share/psqlrc.sample': No such file or directory gmake[1]: *** [install] Error 1 gmake[1]: Leaving directory `/opt/postgresql-9.3.2/src/bin/psql' gmake: *** [install-psql-recurse] Error 2 gmake: Leaving directory `/opt/postgresql-9.3.2/src/bin' The directory 'share' does not exist, which seem to be the issue. Is there a missing dependency somewhere? It appears the doc install correctly creates 'share', so installing src/bin last works. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] why semicolon after begin is not allowed in postgresql?
I believe the section you are reading refers to the BEGIN keyword in the procedural language plpgsql, not the SQL 'BEGIN' command. The issue stems from confusing two distinct languages both of which, along with several more procedural languages, are documented in the same manual. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Fri, Nov 22, 2013 at 4:24 PM, AK wrote: > I am reading the following in the documentation: "Tip: A common mistake is > to > write a semicolon immediately after BEGIN. This is incorrect and will > result > in a syntax error." > > So, "common mistake" means semicolons after BEGIN seem consistent to many > people - it seems consistent to me as well. If PostgreSql allowed them, we > would have one less rule to memorize, shorter documentation, less mistakes > and so on. In other words, without this limitation PostgreSql would be > slightly more useful, right? > > What am I missing? Why do we need this rule? How is it making PostgreSql > better? > > > > -- > View this message in context: > http://postgresql.1045698.n5.nabble.com/why-semicolon-after-begin-is-not-allowed-in-postgresql-tp5779905.html > Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] stats for network traffic WIP
This patch looks good to me. It applies, builds, and runs the regression
tests. Documentation is included and it seems to do what it says. I don't
consider myself a code expert, but as far as I can see it looks fine. This
is a pretty straightforward enhancement to the existing pg_stat_* code.
If no one has any objections, I'll mark it ready for committer.
Mike
______
*Mike Blackwell | Technical Analyst, Distribution Services/Rollout
Management | RR Donnelley*
1750 Wallace Ave | St Charles, IL 60174-3401
Office: 630.313.7818
mike.blackw...@rrd.com
http://www.rrdonnelley.com
<http://www.rrdonnelley.com/>
* *
On Thu, Nov 14, 2013 at 11:29 PM, Nigel Heron wrote:
> On Wed, Nov 13, 2013 at 11:27 PM, Peter Eisentraut
> wrote:
> > On Fri, 2013-11-08 at 10:01 -0500, Nigel Heron wrote:
> >> here's v4 of the patch. I added documentation and a new global view
> >> called "pg_stat_socket" (includes bytes_sent, bytes_received and
> >> stats_reset time)
> >
> > Your patch needs to be rebased:
> >
> > CONFLICT (content): Merge conflict in src/test/regress/expected/rules.out
> >
>
> Hi,
> here's a rebased patch with some additions.
>
> an overview of it's current state...
>
> a new pg_stat_socket global view:
> - total bytes sent and received
> - bytes sent and received for user backends
> - bytes sent and received for wal senders
> - total connection attempts
> - successful connections to user backends
> - successful connections to wal senders
> - stats reset time
> pg_stat_reset_shared('socket') resets the counters
>
> added to pg_stat_database view:
> - bytes sent and received per db
> - successful connections per db
> pg_stat_reset() resets the counters
>
> added to pg_stat_activity view:
> - bytes sent and received per backend
>
> added to pg_stat_replication view:
> - bytes sent and received per wal sender
>
> using the existing track_counts guc to enable/disable these stats.
> -nigel.
>
Re: [HACKERS] additional json functionality
On Wed, Nov 13, 2013 at 4:16 PM, Josh Berkus wrote:
>
> > Putting it all together, I'd consider:
> > *) dropping json_object (although maybe there is a case I'm not
> thinking about)
> > *) changing json_build function names to get the json prefix
> > *) adding a json object constructor that takes two parallel arrays as
> > arguments.
>
> I was with you until the third idea. Huh?
>
>
I actually had a use case for this today, though with hstore, importing a
fixed length record with something along the lines of:
hstore(
ARRAY['field 1', 'field 2', 'field 3'],
regexp_matches(fixed_field,'(.{4})(.{10})(.{5})')
)
__
*Mike Blackwell | Technical Analyst, Distribution Services/Rollout
Management | RR Donnelley*
1750 Wallace Ave | St Charles, IL 60174-3401
Office: 630.313.7818
mike.blackw...@rrd.com
http://www.rrdonnelley.com
<http://www.rrdonnelley.com/>
* *
Re: [HACKERS] stats for network traffic WIP
Also still to be tested: performance impact. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Fri, Nov 8, 2013 at 9:33 AM, Mike Blackwell wrote: > Patch applies and builds against git HEAD (as of 6790e738031089d5). "make > check" runs cleanly as well. > > The new features appear to work as advertised as far as I've been able to > check. > > The code looks good as far as I can see. Documentation patches are > included for the new features. > > Still to be tested: > the counts for streaming replication (no replication setup here to test > against yet). > > > __ > *Mike Blackwell | Technical Analyst, Distribution Services/Rollout > Management | RR Donnelley* > 1750 Wallace Ave | St Charles, IL 60174-3401 > Office: 630.313.7818 > mike.blackw...@rrd.com > http://www.rrdonnelley.com > > > <http://www.rrdonnelley.com/> > * * > > > On Fri, Nov 8, 2013 at 9:01 AM, Nigel Heron wrote: > >> On Tue, Oct 29, 2013 at 11:26 AM, Nigel Heron >> wrote: >> >> >> >> So, for now, the counters only track sockets created from an inbound >> >> (client to server) connection. >> > >> > here's v3 of the patch (rebase and cleanup). >> > >> >> Hi, >> here's v4 of the patch. I added documentation and a new global view >> called "pg_stat_socket" (includes bytes_sent, bytes_received and >> stats_reset time) >> >> thanks, >> -nigel. >> > >
Re: [HACKERS] stats for network traffic WIP
Patch applies and builds against git HEAD (as of 6790e738031089d5). "make check" runs cleanly as well. The new features appear to work as advertised as far as I've been able to check. The code looks good as far as I can see. Documentation patches are included for the new features. Still to be tested: the counts for streaming replication (no replication setup here to test against yet). __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Fri, Nov 8, 2013 at 9:01 AM, Nigel Heron wrote: > On Tue, Oct 29, 2013 at 11:26 AM, Nigel Heron > wrote: > >> > >> So, for now, the counters only track sockets created from an inbound > >> (client to server) connection. > > > > here's v3 of the patch (rebase and cleanup). > > > > Hi, > here's v4 of the patch. I added documentation and a new global view > called "pg_stat_socket" (includes bytes_sent, bytes_received and > stats_reset time) > > thanks, > -nigel. >
Re: [HACKERS] stats for network traffic WIP
On Wed, Oct 23, 2013 at 2:10 PM, Atri Sharma wrote: > > Adding myself as the co reviewer specifically for the testing > purposes, if its ok with you. > It's perfectly fine with me. Please do! ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] stats for network traffic WIP
On Wed, Oct 23, 2013 at 1:58 PM, Atri Sharma wrote: > > IMO, the idea is pretty good. Its just that we need to do some wide > spectrum performance testing. Thats only my thought though. I'm looking at trying to do some performance testing on this. Any suggestions on test scenarios, etc? ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] stats for network traffic WIP
Sounds good. I personally don't have any interest in log file i/o counters, but that's just me. I wonder if stats collector counters might be useful... I seem to recall an effort to improve that area. Maybe not enough use to take the performance hit on a regular basis, though. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Wed, Oct 23, 2013 at 1:44 PM, Nigel Heron wrote: > Hi, thanks, I'm still actively working on this patch. I've gotten the > traffic counters working when using SSL enabled clients (includes the > ssl overhead now) but I still have the walsender transfers under SSL > to work on. > I'll post an updated patch when i have it figured out. > Since the patch changes some views in pg_catalog, a regression test > fails .. i'm not sure what to do next. Change the regression test in > the patch, or wait until the review phase? > > I was also thinking of adding global counters for the stats collector > (pg_stat* file read/write bytes + packets lost) and also log file io > (bytes written for txt and csv formats) .. any interest? > > -nigel. > > On Wed, Oct 23, 2013 at 12:50 PM, Mike Blackwell > wrote: > > I added this to the current CF, and am starting to review it as I have > time. > > > > > __ > > Mike Blackwell | Technical Analyst, Distribution Services/Rollout > Management > > | RR Donnelley > > 1750 Wallace Ave | St Charles, IL 60174-3401 > > Office: 630.313.7818 > > mike.blackw...@rrd.com > > http://www.rrdonnelley.com > > > > > > > > > > On Mon, Oct 21, 2013 at 11:32 AM, Stephen Frost > wrote: > >> > >> Nigel, > >> > >> * Nigel Heron (nhe...@querymetrics.com) wrote: > >> > Hi, I've been using postgres for many years but never took the time to > >> > play > >> > with the code until now. As a learning experience i came up with this > >> > WIP > >> > patch to keep track of the # of bytes sent and received by the server > >> > over > >> > it's communication sockets. Counters are kept per database, per > >> > connection > >> > and globally/shared. > >> > >> Very neat idea. Please add it to the current commitfest > >> (http://commitfest.postgresql.org) and, ideally, someone will get in > and > >> review it during the next CM. > >> > >> Thanks! > >> > >> Stephen > > > > >
Re: [HACKERS] stats for network traffic WIP
I added this to the current CF, and am starting to review it as I have time. __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Mon, Oct 21, 2013 at 11:32 AM, Stephen Frost wrote: > Nigel, > > * Nigel Heron (nhe...@querymetrics.com) wrote: > > Hi, I've been using postgres for many years but never took the time to > play > > with the code until now. As a learning experience i came up with this WIP > > patch to keep track of the # of bytes sent and received by the server > over > > it's communication sockets. Counters are kept per database, per > connection > > and globally/shared. > > Very neat idea. Please add it to the current commitfest > (http://commitfest.postgresql.org) and, ideally, someone will get in and > review it during the next CM. > > Thanks! > > Stephen >
[HACKERS] RULE regression test fragility?
While reviewing the Network Stats Traffic patch I discovered the current regression test for rules depends on the system view definitions not changing: -- -- Check that ruleutils are working -- SELECT viewname, definition FROM pg_views WHERE schemaname <> 'information_schema' ORDER BY viewname; In this particular case new fields have been added to the view, breaking this apparently unrelated test. Is checking the definition of all views necessary for this test? Would it possibly be better to create a temporary view for this check, or is something else going on here? __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] Commitfest II CLosed
Actually, I did call them out in the thread announcing the CF Wrap Up ( http://www.postgresql.org/message-id/CAESHdJonURj3i9HR2w4e=ohep5hx7snqyydsgyweqqa+a3d...@mail.gmail.com). Looking back, it may have been better to post it as a separate thread, but I'm not confident that would have made much difference. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Mon, Oct 21, 2013 at 9:45 AM, Hannu Krosing wrote: > On 10/21/2013 03:56 PM, Heikki Linnakangas wrote: > > > > I feel guilty to complain, while not actually volunteering to be a > > commitfest manager myself, but I wish the commitfest manager would be > > more aggressive in nagging, pinging and threatening people to review > > stuff. If nothing else, always feel free to nag me :-). Josh tried > > that with the infamous Slacker List, but that backfired. Rather than > > posting a public list of shame, I think it would work better to send > > short off-list nag emails, or chat via IM. Something like "Hey, you've > > signed up to review this. Any progress?". Or "Hey, could you take a > > look at X please? No-one else seems to care about it." > Or maybe even nag publicly with "list of orphans" - hey people, do you > *really* think that this patch is not needed ? > > > > -- > Hannu Krosing > PostgreSQL Consultant > Performance, Scalability and High Availability > 2ndQuadrant Nordic OÜ > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Re: [HACKERS] CF 2013-09 Wrap Up
On Tue, Oct 15, 2013 at 1:39 AM, Noah Misch wrote: > On Mon, Oct 14, 2013 at 01:56:42PM -0500, Mike Blackwell wrote:> Any > patches marked Needs Review will be automatically moved to the next CF. > > We will try to make sure that all patches in the current CF have > received > > at least one review. > > The combined effect of those two statements is not clear to me. Does that > mean you'll retain never-reviewed patches and automatically move patches > that > have received at least one review? > Yes on the latter part. We will try to get a quick review for not-yet-reviewed patches and move or return them based on the result of that review. If we fail to find a reviewer, the patches will get moved to the next CF. For those following along, here are the patches still needing a first look. They are for the most part performance or internals patches and could use the eye of someone more experienced. Please consider a quick review of one of them if you fit that description. We'd like everyone to get a fair shake here. ^_^ HStore Gin Speedup<https://commitfest.postgresql.org/action/patch_view?id=1203> Performance Improvement by reducing WAL for Update Operation<https://commitfest.postgresql.org/action/patch_view?id=1209> [PoC] pgstattuple2: block sampling to reduce physical read<https://commitfest.postgresql.org/action/patch_view?id=1226> ECPG cursor readahead<https://commitfest.postgresql.org/action/patch_view?id=1195>
[HACKERS] CF 2013-09 Wrap Up
CF 2013-09 will be wrapping up this week. As a reminder, beginning on the official CF end date (11/15), patches Waiting for Author will be Returned with Feedback. Authors are welcome to add their patch to the next CF (2013-11). Any patches marked Needs Review will be automatically moved to the next CF. We will try to make sure that all patches in the current CF have received at least one review. If you have any questions or comments about the CF process, feel free to send a note to me or David Fetter (CFM). ^_^ Thanks to all who have submitted or reviewed patches this time around! Mike Blackwell
Re: [HACKERS] Patch for reserved connections for replication users
I'd received an email from Gibheer suggesting it be move due to lack of time to
work on it. I can certainly move it back if that's no longer the case.
On Oct 9, 2013, at 23:25, Amit Kapila wrote:
> On Thu, Oct 10, 2013 at 3:17 AM, Gibheer wrote:
> On Mon, 7 Oct 2013 11:39:55 +0530
> Amit Kapila wrote:
>> Robert Haas wrote:
>> On Mon, Aug 5, 2013 at 2:04 AM, Andres Freund
>> wrote:
> Hmm. It seems like this match is making MaxConnections no longer
> mean the maximum number of connections, but rather the maximum
> number of non-replication connections. I don't think I support
> that definitional change, and I'm kinda surprised if this is
> sufficient to implement it anyway (e.g. see InitProcGlobal()).
>>>
I don't think the implementation is correct, but why don't you
like the definitional change? The set of things you can do from
replication connections are completely different from a normal
connection. So using separate "pools" for them seems to make sense.
That they end up allocating similar internal data seems to be an
implementation detail to me.
>>
>>> Because replication connections are still "connections". If I tell
>>> the system I want to allow 100 connections to the server, it should
>>> allow 100 connections, not 110 or 95 or any other number.
>>
>> I think that to reserve connections for replication, mechanism similar
>> to superuser_reserved_connections be used rather than auto vacuum
>> workers or background workers.
>> This won't change the definition of MaxConnections. Another thing is
>> that rather than introducing new parameter for replication reserved
>> connections, it is better to use max_wal_senders as it can serve the
>> purpose.
>>
>> Review for replication_reserved_connections-v2.patch, considering we
>> are going to use mechanism similar to superuser_reserved_connections
>> and won't allow definition of MaxConnections to change.
>>
>> 1. /* the extra unit accounts for the autovacuum launcher */
>> MaxBackends = MaxConnections + autovacuum_max_workers + 1 +
>> - + max_worker_processes;
>> + + max_worker_processes + max_wal_senders;
>>
>> Above changes are not required.
>>
>> 2.
>> + if ((!am_superuser && !am_walsender) &&
>> ReservedBackends > 0 &&
>> !HaveNFreeProcs(ReservedBackends))
>>
>> Change the check as you have in patch-1 for
>> ReserveReplicationConnections
>>
>> if (!am_superuser &&
>> (max_wal_senders > 0 || ReservedBackends > 0) &&
>> !HaveNFreeProcs(max_wal_senders + ReservedBackends))
>> ereport(FATAL,
>> (errcode(ERRCODE_TOO_MANY_CONNECTIONS),
>> errmsg("remaining connection slots are reserved for replication and
>> superuser connections")));
>>
>> 3. In guc.c, change description of max_wal_senders similar to
>> superuser_reserved_connections at below place:
>> {"max_wal_senders", PGC_POSTMASTER, REPLICATION_SENDING,
>> gettext_noop("Sets the maximum number of simultaneously running WAL
>> sender processes."),
>>
>> 4. With this approach, there is no need to change InitProcGlobal(), as
>> it is used to keep track bgworkerFreeProcs and autovacFreeProcs, for
>> others it use freeProcs.
>>
>> 5. Below description in config.sgml needs to be changed for
>> superuser_reserved_connections to include the effect of max_wal_enders
>> in reserved connections.
>> Whenever the number of active concurrent connections is at least
>> max_connections minus superuser_reserved_connections, new connections
>> will be accepted only for superusers, and no new replication
>> connections will be accepted.
>>
>> 6. Also similar description should be added to max_wal_senders
>> configuration parameter.
>>
>> 7. Below comment needs to be updated, as it assumes only superuser
>> reserved connections as part of MaxConnections limit.
>> /*
>> * ReservedBackends is the number of backends reserved for superuser
>> use.
>> * This number is taken out of the pool size given by MaxBackends so
>> * number of backend slots available to non-superusers is
>> * (MaxBackends - ReservedBackends). Note what this really means is
>> * "if there are <= ReservedBackends connections available, only
>> superusers
>> * can make new connections" --- pre-existing superuser connections
>> don't
>> * count against the limit.
>> */
>> int ReservedBackends;
>>
>> 8. Also we can add comment on top of definition for max_wal_senders
>> similar to ReservedBackends.
>>
>>
>> With Regards,
>> Amit Kapila.
>> EnterpriseDB: http://www.enterprisedb.com
>
> Hi,
>
> I took the time and reworked the patch with the feedback till now.
> Thank you very much Amit!
Is there any specific reason why you moved this patch to next CommitFest?
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] File_fdw documentation patch to clarify OPTIONS clause
Robert, Thanks for the reply. I have no objections to clarifying the note. Attached is a patch with the text you suggested. Mike __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Mon, Sep 23, 2013 at 11:42 AM, Robert Haas wrote: > On Fri, Sep 20, 2013 at 4:26 PM, Mike Blackwell > wrote: > > The file_fdw documentation for the OPTIONS clause references the COPY > > command's documentation. In the case of COPY, the value for some options > > (e.g. HEADER, OIDS, ...) is optional. While the file_fdw documentation > > makes no mention of it, the values for all options are required. > > > > Attached is a patch to add a note to the docs mentioning this fact. > > I think this would be a good thing to document, but it took me a > minute to properly understand what you were saying. So I'd like to > suggest slightly different wording: "Note that while COPY allows > options such as OIDS and HEADER to be specified without a > corresponding value, the foreign data wrapper syntax requires a value > to be present in all cases. To activate COPY options normally > supplied without a value, you can instead pass the value TRUE." > > Other suggestions welcome if you don't like that text... > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > file-fdw-doc.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] File_fdw documentation patch to clarify OPTIONS clause
The file_fdw documentation for the OPTIONS clause references the COPY command's documentation. In the case of COPY, the value for some options (e.g. HEADER, OIDS, ...) is optional. While the file_fdw documentation makes no mention of it, the values for all options are required. Attached is a patch to add a note to the docs mentioning this fact. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * file-fdw-doc.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation
The only environment I have available at the moment is a virtual box. That's probably not going to be very helpful for performance testing. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Mon, Jul 8, 2013 at 11:09 PM, Amit Kapila wrote: > > On Tuesday, July 09, 2013 2:52 AM Mike Blackwell wrote: > > > I can't comment on further direction for the patch, but since it was > marked as Needs Review in the CF app I took a quick look at it. > Thanks for looking into it. > > Last time Heikki has found test scenario's where the original patch was > not performing good. > He has also proposed a different approach for WAL encoding and sent the > modified patch which has comparatively less negative performance impact and > asked to check if the patch can reduce the performance impact for the > scenario's mentioned by him. > After that I found that with some modification's (use new tuple data for > encoding) in his approach, it eliminates the negative performance impact > and > have WAL reduction for more number of cases. > > I think the first thing to verify is whether the results posted can be > validated in some other environment setup by another person. > The testcase used is posted at below link: > http://www.postgresql.org/message-id/51366323.8070...@vmware.com > > > > > It patches and compiles clean against the current Git HEAD, and 'make > check' runs successfully. > > > Does it need documentation for the GUC variable > 'wal_update_compression_ratio'? > > This variable has been added to test the patch for different > compression_ratio during development test. > It was not decided to have this variable permanently as part of this > patch, so currently there is no documentation for it. > However if the decision comes out to be that it needs to be part of > patch, then documentation for same can be updated. > > With Regards, > Amit Kapila. > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
[HACKERS] [REVIEW] row level security (v3)
The most recent patch (v3) applies and builds cleanly and passes make check. Documentation on the new SQL syntax and catalog changes is included with the patch and looks good to me. The regression tests look pretty complete. In addition to the included tests, dropping and altering the data type on a column referenced in the security clause work as expected, rejecting the change with a dependency error. Renaming a column succeeds as expected. pg_dump and restore properly was also successful. I noticed that the security clause is visible to any user via psql \dt+, as well as in the pg_rowsecurity view. Perhaps this should be mentioned in the relevant section of user-manag.sgml so users realize any sensitive information in the security clause isn't secure. What I've checked looks good. I don't feel qualified to do a code review so that's still outstanding. I believe Atri will be looking at that. ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation
I can't comment on further direction for the patch, but since it was marked as Needs Review in the CF app I took a quick look at it. It patches and compiles clean against the current Git HEAD, and 'make check' runs successfully. Does it need documentation for the GUC variable 'wal_update_compression_ratio'? ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * * On Tue, Jul 2, 2013 at 2:26 AM, Hari Babu wrote: > On Friday, June 07, 2013 5:07 PM Amit Kapila wrote: > >On Wednesday, March 06, 2013 2:57 AM Heikki Linnakangas wrote: > >> On 04.03.2013 06:39, Amit Kapila wrote: > >> > On Sunday, March 03, 2013 8:19 PM Craig Ringer wrote: > >> >> On 02/05/2013 11:53 PM, Amit Kapila wrote: > >> >>>> Performance data for the patch is attached with this mail. > >> >>>> Conclusions from the readings (these are same as my previous > >> patch): > >> >>>> > >> > >> The attached patch also just adds overhead in most cases, but the > >> overhead is much smaller in the worst case. I think that's the right > >> tradeoff here - we want to avoid scenarios where performance falls off > >> the cliff. That said, if you usually just get a slowdown, we certainly > >> can't make this the default, and if we can't turn it on by default, > >> this probably just isn't worth it. > >> > >> The attached patch contains the variable-hash-size changes I posted in > >> the "Optimizing pglz compressor". But in the delta encoding function, > >> it goes further than that, and contains some further micro- > >> optimizations: > >> the hash is calculated in a rolling fashion, and it uses a specialized > >> version of the pglz_hist_add macro that knows that the input can't > >> exceed 4096 bytes. Those changes shaved off some cycles, but you could > >> probably do more. One idea is to only add every 10 bytes or so to the > >> history lookup table; that would sacrifice some compressibility for > >> speed. > >> > >> If you could squeeze pglz_delta_encode function to be cheap enough > >> that we could enable this by default, this would be pretty cool patch. > >> Or at least, the overhead in the cases that you get no compression > >> needs to be brought down, to about 2-5 % at most I think. If it can't > >> be done easily, I feel that this probably needs to be dropped. > > >After trying some more on optimizing pglz_delta_encode(), I found that if > we use new data also in history, then the results of compression and cpu > utilization >are much better. > > >In addition to the pg lz micro optimization changes, following changes are > done in modified patch > > >1. The unmatched new data is also added to the history which can be > referenced later. > >2. To incorporate this change in the lZ algorithm, 1 extra control bit is > needed to indicate if data is from old or new tuple > > The patch is rebased to use the new PG LZ algorithm optimization changes > which got committed recently. > > Performance Data > - > > Head code: > > testname | wal_generated | duration > > -+---+-- > > two short fields, no change |1232911016 | 35.1784930229187 > two short fields, one changed |1240322016 | 35.0436308383942 > two short fields, both changed |1235318352 | 35.4989421367645 > one short and one long field, no change |1042332336 | 23.4457180500031 > ten tiny fields, all changed|1395194136 | 41.9023628234863 > hundred tiny fields, first 10 changed | 626725984 | 21.2999589443207 > hundred tiny fields, all changed| 621899224 | 21.6676609516144 > hundred tiny fields, half changed | 623998272 | 21.2745981216431 > hundred tiny fields, half nulled| 557714088 | 19.5902800559998 > > > pglz-with-micro-optimization-compress-using-newdata-2: > > testname | wal_generated | duration > > -+---+-- > > two short fields, no change |1232903384 | 35.0115969181061 > two short fields, one changed |12329
Re: [HACKERS] [v9.4] row level security
With the current HEAD and v3 patch I'm seeing the following error with "make check". -- == creating temporary installation== == initializing database system == pg_regress: initdb failed Examine /usr/local/src/postgres.patched.v3/src/test/regress/log/initdb.log for the reason. Command was: "/usr/local/src/postgres.patched.v3/src/test/regress/./tmp_check/install//usr/local/pgsql/bin/initdb" -D "/usr/local/src/postgres.patched.v3/src/test/regress/./tmp_check/data" -L "/usr/local/src/postgres.patched.v3/src/test/regress/./tmp_check/install//usr/local/pgsql/share" --noclean --nosync > "/usr/local/src/postgres.patched.v3/src/test/regress/log/initdb.log" 2>&1 make[1]: *** [check] Error 2 make[1]: Leaving directory `/usr/local/src/postgres.patched.v3/src/test/regress' make: *** [check] Error 2 ______ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout Management | RR Donnelley* 1750 Wallace Ave | St Charles, IL 60174-3401 Office: 630.313.7818 mike.blackw...@rrd.com http://www.rrdonnelley.com <http://www.rrdonnelley.com/> * *
