Re: [HACKERS] WAL consistency check facility

2016-09-10 Thread Amit Kapila
om the clarity perspective, this option sounds good, but I am slightly afraid that it might be inconvenient for users to set the different values for these two parameters. > Or maybe skip the second one and just decree that > standbys will always validate if the necessary informatio

Re: [HACKERS] WAL consistency check facility

2016-09-10 Thread Amit Kapila
; I'd like to hear extra opinions about that, but IMO just having an > ERROR would be fine for the first implementation. Once you've bumped > into an ERROR, you are likely going to fix it first. > +1 for just an ERROR to detect the inconsistency. I think adding additional GUC ju

Re: [HACKERS] pg_sequence catalog

2016-09-10 Thread Amit Kapila
On Sun, Sep 11, 2016 at 12:39 AM, Andres Freund wrote: > On 2016-09-10 17:23:21 +0530, Amit Kapila wrote: >> > >> >> I may be missing something here, but why would it contend on a lock, >> as per locking scheme proposed by Alvaro, access to sequence object >> w

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-10 Thread Amit Kapila
mjeem+qn0jzx31-obx...@mail.gmail.com [2] - https://www.postgresql.org/message-id/cad__ouhj29cebif_flge4t9vj_-cfxbwcxhjo+d_16txbem...@mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] pg_sequence catalog

2016-09-10 Thread Amit Kapila
nding on a single lock. > I may be missing something here, but why would it contend on a lock, as per locking scheme proposed by Alvaro, access to sequence object will need a share lock on buffer page. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsq

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

2016-09-10 Thread Amit Kapila
On Tue, Aug 30, 2016 at 8:01 AM, Andres Freund wrote: > On 2016-08-30 07:57:19 +0530, Amit Kapila wrote: >> I will write such a test case either in this week or early next week. > > Great. > Okay, attached patch adds some simple tests for pg_stat_statements. One thing to not

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2016-09-09 Thread Amit Kapila
o, I can create a > separate patch to be considered for a separate commit. > +1 to have such a function. I often need something like that whenever I debug the optimizer code. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-08 Thread Amit Kapila
ql.org/message-id/CAE9k0PkPumi4iWFuD%2BjHHkpcxn531%3DDJ8uH0dctsvF%2BdaZY6yQ%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-07 Thread Amit Kapila
od! > Thanks for verifying the issue and doing further testing of the patch. It is really helpful. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.pos

Re: [HACKERS] Hash Indexes

2016-09-07 Thread Amit Kapila
On Wed, Sep 7, 2016 at 11:49 PM, Jeff Janes wrote: > On Thu, Sep 1, 2016 at 8:55 PM, Amit Kapila wrote: >> >> >> I have fixed all other issues you have raised. Updated patch is >> attached with this mail. > > > I am finding the comments (particularly README

Re: [HACKERS] WAL consistency check facility

2016-09-07 Thread Amit Kapila
g the flags and set them during replay. Note - Please post your replies inline rather than top posting them. It breaks the discussion link, if you top post it. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-07 Thread Amit Kapila
On Wed, Sep 7, 2016 at 1:08 AM, Tomas Vondra wrote: > On 09/06/2016 04:49 AM, Amit Kapila wrote: >> On Mon, Sep 5, 2016 at 11:34 PM, Tomas Vondra >> wrote: >>> >>> >>> On 09/05/2016 06:03 AM, Amit Kapila wrote: >>>> So, in short we have to

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-07 Thread Amit Kapila
On Wed, Sep 7, 2016 at 3:28 PM, Amit Kapila wrote: > > Okay, I have fixed this issue as explained above. Apart from that, I > have fixed another issue reported by Mark Kirkwood upthread and few > other issues found during internal testing by Ashutosh Sharma. > Forgot to mention t

Re: [HACKERS] condition variables

2016-09-06 Thread Amit Kapila
On Tue, Sep 6, 2016 at 5:29 PM, Robert Haas wrote: > On Mon, Sep 5, 2016 at 3:17 PM, Amit Kapila wrote: >> On Mon, Aug 15, 2016 at 10:35 PM, Robert Haas wrote: >>>>> Don't you need to set proc->cvSleeping = false in ConditionVariableSignal? >>>> &g

Re: [HACKERS] Cache Hash Index meta page.

2016-09-05 Thread Amit Kapila
y based on patch in the below mail: https://www.postgresql.org/message-id/CAA4eK1J6b8O4PcEPqRxNYbLVbfToNMJEEm%2Bqn0jZX31-obXrJw%40mail.gmail.com It is better if we can provide the link for a patch on which the current patch is based on, that will help people to easily identify the dependent patches. -- With

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-05 Thread Amit Kapila
On Mon, Sep 5, 2016 at 11:34 PM, Tomas Vondra wrote: > > > On 09/05/2016 06:03 AM, Amit Kapila wrote: >> So, in short we have to compare three >> approaches here. >> >> 1) Group mode to reduce CLOGControlLock contention >> 2) Use granular locking mode

Re: [HACKERS] condition variables

2016-09-05 Thread Amit Kapila
re setting the latch. > > Right, OK. > I have independently faced this problem while using your patch and for now I have updated my local copy. If possible, please send an updated patch as this patch could be used for development of various parallelism projects. -- With Regards, Amit

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-05 Thread Amit Kapila
e > results because that might influence which approach we choose. > > Does that sound correct? > Sounds correct to me. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-04 Thread Amit Kapila
howing the interest and let me know if something is still un-clear or you need more information to proceed. [1] - https://www.postgresql.org/message-id/CAA4eK1%2B8gQTyGSZLe1Rb7jeM1Beh4FqA4VNjtpZcmvwizDQ0hw%40mail.gmail.com [2] - https://www.postgresql.org/message-id/20160330230914.GH13305%40awo

Re: [HACKERS] Hash Indexes

2016-09-01 Thread Amit Kapila
On Thu, Sep 1, 2016 at 11:33 PM, Jesper Pedersen wrote: > On 08/05/2016 07:36 AM, Amit Kapila wrote: > > Needs a rebase. > Done. > > + if (blkno == P_NEW) > + elog(ERROR, "hash AM does not use P_NEW"); > > Left over ? > No. We nee

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
On Thu, Sep 1, 2016 at 9:43 AM, Peter Geoghegan wrote: > On Wed, Aug 31, 2016 at 8:26 PM, Amit Kapila wrote: >>> As far as I am understanding things, we are aiming at something that >>> could be used on production systems. >>> >> >> I don't t

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
On Thu, Sep 1, 2016 at 8:02 AM, Amit Kapila wrote: > On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: >> On 27 August 2016 at 12:09, Kuntal Ghosh wrote: >> >>>>> * wal_consistency_mask = 511 /* Enable consistency check mask bit*/ >>>> >>>&

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
On Thu, Sep 1, 2016 at 8:30 AM, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 11:32 AM, Amit Kapila wrote: >> On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: >>> On 27 August 2016 at 12:09, Kuntal Ghosh wrote: >>> >>>>>> * wal_consistency_mas

Re: [HACKERS] WAL consistency check facility

2016-08-31 Thread Amit Kapila
in narrowing down the problem, if we have facility to enable it at RMGR ID level. Having said that, I think this must have the facility to enable it for all the RMGR ID's (say ALL) and probably that should be default. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com --

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

2016-08-29 Thread Amit Kapila
;t any correlation between >>> what code called RequestAddinLWLocks() and what code called >>> LWLockAssign(); when multiple modules are in use, it could become >>> quite difficult to troubleshoot problems where LWLockAssign() ran out >>> of locks. To fix, create a c

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-29 Thread Amit Kapila
On Tue, Aug 30, 2016 at 3:40 AM, Alvaro Herrera wrote: > Amit Kapila wrote: > >> How about attached? > > That works; pushed. Thanks. > (I removed a few #includes from the new file.) > oops, copied from hash.h and forgot to remove those. >> If you want, I think we

Re: [HACKERS] Sample configuration files

2016-08-29 Thread Amit Kapila
e, it > doesn't try to install them. That's partly because I think it would > need an extension version bump and that doesn't seem worth it. > What is the use case and how these files suppose to work? Do you expect these to be loaded as we do for postgresql.conf? --

Re: [HACKERS] WIP: Covering + unique indexes.

2016-08-27 Thread Amit Kapila
ert keytup as P_HIKEY. */ + PageIndexTupleDelete(opage, P_HIKEY); + + if (!_bt_pgaddtup(opage, IndexTupleSize(keytup), keytup, P_HIKEY)) + elog(ERROR, "failed to rewrite compressed item in index \"%s\"", + RelationGetRelationName(wstate->index)); + } + .. .. -- With Regards

Re: [HACKERS] WAL consistency check facility

2016-08-27 Thread Amit Kapila
HAS_GARBAGE, it seems we should mask BTP_SPLIT_END as that is just used to save some processing for vaccum and won't be set after crash recovery or on standby after WAL replay. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (p

Re: [HACKERS] shm_mq_set_sender() crash

2016-08-27 Thread Amit Kapila
cxt->worker[i].bgwhandle)) .. } Can some reordering impact the above code? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WAL consistency check facility

2016-08-26 Thread Amit Kapila
reads the same. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-26 Thread Amit Kapila
On Thu, Aug 25, 2016 at 6:54 PM, Alvaro Herrera wrote: > Amit Kapila wrote: >> On Wed, Aug 24, 2016 at 11:46 PM, Alvaro Herrera >> wrote: > >> > Can you split the new xlog-related stuff to a new file, say hash_xlog.h, >> > instead of cramming it in has

Re: [HACKERS] increasing the default WAL segment size

2016-08-25 Thread Amit Kapila
On Fri, Aug 26, 2016 at 9:04 AM, Michael Paquier wrote: > On Fri, Aug 26, 2016 at 12:25 PM, Amit Kapila wrote: >> If we change the default to 64MB, then I think it won't allow to use >> old databases as-is because we store it in pg_control (I think one >> will g

Re: [HACKERS] increasing the default WAL segment size

2016-08-25 Thread Amit Kapila
, then I think it won't allow to use old databases as-is because we store it in pg_control (I think one will get below error [1] for old databases, if we just change default and don't do anything else). Do you have way to address it or you think it is okay? [1] - FATAL: database fi

Re: [HACKERS] Why is a newly created index contains the invalid LSN?

2016-08-25 Thread Amit Kapila
e WAL for those indexes as well when wal_level < REPLICA. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-08-25 Thread Amit Kapila
but that variable is mainly used in pgstat.c/.h only. > Would it be worth making that change. > > -1 for the proposed change. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-24 Thread Amit Kapila
On Wed, Aug 24, 2016 at 11:46 PM, Alvaro Herrera wrote: > Amit Kapila wrote: >> $SUBJECT will make hash indexes reliable and usable on standby. > > Nice work. > > Can you split the new xlog-related stuff to a new file, say hash_xlog.h, > instead of cramming it in hash.h

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-23 Thread Amit Kapila
case in the regression suite as well after fix. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com test_hash.sql Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Some tests to cover hash_index"

2016-08-23 Thread Amit Kapila
n's patch even if those new tests helps to cover the code path which is required for wal-hash-index patch. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-23 Thread Amit Kapila
the fix is to use REGBUF_NO_IMAGE as we use for vm buffers. If you can send me the detailed steps for how you have produced the problem, then I can verify after fixing whether you are seeing the same problem or something else. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-23 Thread Amit Kapila
On Wed, Aug 24, 2016 at 9:53 AM, Mark Kirkwood wrote: > On 24/08/16 15:36, Amit Kapila wrote: >> >> On Wed, Aug 24, 2016 at 8:54 AM, Mark Kirkwood >> wrote: >>> >> >> Can you get the call stacks? >> > > For every stuck backend? (just dou

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-23 Thread Amit Kapila
On Wed, Aug 24, 2016 at 8:54 AM, Mark Kirkwood wrote: > On 24/08/16 12:09, Mark Kirkwood wrote: >> >> On 23/08/16 15:24, Amit Kapila wrote: >>> >>> >>> Thoughts? >>> >>> Note - To use this patch, first apply latest version

Re: [HACKERS] Block level parallel vacuum WIP

2016-08-23 Thread Amit Kapila
inappropriate balance of work among workers. Suppose one worker is able to finish it's work, it won't be able to do more. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] Proposal for CSN based snapshots

2016-08-23 Thread Amit Kapila
ires ProcArrayLock in shared mode in GetSnapshotData() although duration is reduced. I think it is worth to try the workload where we are seeing ProcArrayLock as bottleneck, but not sure if it can drastically improves the situation, because as per our initial analysis, lock acquisition and relea

Re: [HACKERS] WAL consistency check facility

2016-08-22 Thread Amit Kapila
also going to be very quite confusing for > users. You should not need to apply some maths to set up this > parameter, a list of rmgr names may be more adapted if this level of > tuning is needed, > Yeah, that can be better. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enter

Re: [HACKERS] WAL consistency check facility

2016-08-22 Thread Amit Kapila
nager list as defined in rmgrlist.h? Another thing that needs some thoughts is the UI of this patch, currently it is using integer mask which might not be best way, but again as it is intended to be mainly used for tests, it might be okay. Do we want to enable some tests in the regression

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-08-22 Thread Amit Kapila
On Tue, Aug 23, 2016 at 8:54 AM, Amit Kapila wrote: > $SUBJECT will make hash indexes reliable and usable on standby. > AFAIU, currently hash indexes are not recommended to be used in > production mainly because they are not crash-safe and with this patch, > I hope we can address tha

Re: [HACKERS] dsm_unpin_segment

2016-08-22 Thread Amit Kapila
On Mon, Aug 22, 2016 at 6:06 PM, Thomas Munro wrote: > On Tue, Aug 23, 2016 at 12:07 AM, Amit Kapila wrote: >> + int control_slot = -1; >> ... >> + if (control_slot == -1) >> + elog(ERROR, "cannot unpin unknown segment handle"); >> >> Isn't

Re: [HACKERS] Re: [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-08-22 Thread Amit Kapila
sically in similar situation, the count for nonExclusiveBackups will be decremented and if it hits pg_start_backup_callback(), the following Assertion can fail. pg_start_backup_callback() { .. Assert(XLogCtl->Insert.nonExclusiveBackups > 0); .. } -- With Regards, Amit Kapila. EnterpriseDB: http://www.e

Re: [HACKERS] dsm_unpin_segment

2016-08-22 Thread Amit Kapila
On Mon, Aug 22, 2016 at 5:24 AM, Thomas Munro wrote: > On Sat, Aug 20, 2016 at 11:37 PM, Amit Kapila wrote: >> On Tue, Aug 9, 2016 at 10:07 AM, Thomas Munro >> wrote: >>> On Tue, Aug 9, 2016 at 12:53 PM, Tom Lane wrote: >>>> The larger picture here is that

Re: [HACKERS] dsm_unpin_segment

2016-08-20 Thread Amit Kapila
On Sat, Aug 20, 2016 at 6:13 PM, Robert Haas wrote: > On Sat, Aug 20, 2016 at 7:37 AM, Amit Kapila wrote: >> 2. >> + if (dsm_control->item[seg->control_slot].pinned) >> + elog(ERROR, "cannot pin a segment that is already pinned"); >> >> Shou

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-20 Thread Amit Kapila
On Sat, Aug 20, 2016 at 9:58 PM, Claudio Freire wrote: > On Sat, Aug 20, 2016 at 4:27 AM, Amit Kapila wrote: >> >> That makes sense, but this means there is a chance that the searches >> could lead to different buffers in case of uniqueness checks (the >> search wi

Re: [HACKERS] dsm_unpin_segment

2016-08-20 Thread Amit Kapila
should be set to false after dsm_impl_unpin_segment(). Do you need a check if (control_slot >= 0)? In the code just above you have as Assert to ensure that it is >=0. 2. + if (dsm_control->item[seg->control_slot].pinned) + elog(ERROR, "cannot pin a segment that is already pinned&q

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-20 Thread Amit Kapila
On Sat, Aug 20, 2016 at 10:23 AM, Claudio Freire wrote: > On Sat, Aug 20, 2016 at 1:05 AM, Amit Kapila wrote: >> On Thu, Aug 18, 2016 at 8:24 AM, Claudio Freire >> wrote: >>> >>> A couple of points make me uneasy about this patch, yet I can think of >

Re: [HACKERS] CLUSTER, reform_and_rewrite_tuple(), and parallelism

2016-08-19 Thread Amit Kapila
CPU intensive is probably a good bet to push down to workers (provided it is safe). -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [WIP] [B-Tree] Keep indexes sorted by heap physical location

2016-08-19 Thread Amit Kapila
to use it. [1] - https://www.postgresql.org/message-id/caa4ek1lkq_udism-z2dq6cuvjh3db5fnfnnezzbpsrjw0ha...@mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Should we cacheline align PGXACT?

2016-08-19 Thread Amit Kapila
On Fri, Aug 19, 2016 at 8:24 PM, Alexander Korotkov wrote: > On Fri, Aug 19, 2016 at 3:19 PM, Amit Kapila > wrote: >> >> On Fri, Aug 19, 2016 at 11:42 AM, Alexander Korotkov >> wrote: >> > Hackers, >> > >> > originally this idea was prop

Re: [HACKERS] Should we cacheline align PGXACT?

2016-08-19 Thread Amit Kapila
ws huge benefit in this case. > For sure, we should validate that it doesn't cause performance regression in > other cases. At least we should test read-write and smaller machines. > Any other ideas? > may be test on Power m/c as well. -- With Regards, Amit Kapila. Enterp

Re: [HACKERS] LWLocks in DSM memory

2016-08-19 Thread Amit Kapila
h difference in performance, > small variance in perf can be attributed to variance in probability of > drawing the particular built-in script. > > Can you specify the m/c details as Andres wants tests to be conducted on some high socket m/c? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Pluggable storage

2016-08-18 Thread Amit Kapila
alog information, they need to be aware of HeapTuple and other required stuff like syscache? Again, if they need to update some stats or something like that, they need to be aware of heap tuple format. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-ha

Re: [HACKERS] [PATCH] bigint txids vs 'xid' type, new txid_recent(bigint) => xid

2016-08-18 Thread Amit Kapila
xact_commit_timestamp > I think there is a value in exposing such a variant which takes bigint and internally converts it to xid. I am not sure the semantics for the other proposal txid_recent() is equivalent to what we have for txid_current(). One thing which is different is that txid_curren

Re: [HACKERS] parallel.c is not marked as test covered

2016-08-17 Thread Amit Kapila
le file suggesting that adding more scripts to that group >> would be bad. But yes, perhaps putting this test into its own standalone >> group would be enough of a fix. > > Maybe now would be a good time to address this by applying the attached > patch to master and seeing wh

Re: [HACKERS] [sqlsmith] Failed assertion in joinrels.c

2016-08-17 Thread Amit Kapila
pgsql/plperl/plpython) functions. Is there a reason for changing the *_validator() function? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-13 Thread Amit Kapila
just commits the previous transaction after the user turns the Autocommit to on and the first command is complete. I am not saying that we should blindly follow that behaviour, but just to indicate that it should be okay for users if we don't try to define multiple behaviours here based on va

Re: [HACKERS] new autovacuum criterion for visible pages

2016-08-13 Thread Amit Kapila
On Thu, Aug 11, 2016 at 9:29 PM, Jeff Janes wrote: > On Thu, Aug 11, 2016 at 8:32 AM, Amit Kapila wrote: >> On Thu, Aug 11, 2016 at 2:09 AM, Jeff Janes wrote: >>> I wanted to create a new relopt named something like >>> autovacuum_vacuum_pagevisible_factor whic

Re: [HACKERS] [parallel query] random server crash while running tpc-h query on power2

2016-08-13 Thread Amit Kapila
readnext() instead of calling > ExecShutdownGatherWorkers(). > > > Attaching patch, which fix the issue with approach 1). > AFAICS, your patch seems to be the right fix for this issue, unless we need the instrumentation information during execution (other than for explain) for some p

Re: [HACKERS] new autovacuum criterion for visible pages

2016-08-11 Thread Amit Kapila
; Why would it more convenient for a user to set such a parameter as compare to existing parameters (autovacuum_vacuum_threshold + autovacuum_vacuum_scale_factor)? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Amit Kapila
On Mon, Aug 8, 2016 at 9:56 PM, Bruce Momjian wrote: > On Mon, Aug 8, 2016 at 06:34:46PM +0530, Amit Kapila wrote: >> I think here expensive part would be recheck for the cases where the >> index value is changed to a different value (value which doesn't exist >> in

Re: [HACKERS] [sqlsmith] Failed assertion in joinrels.c

2016-08-09 Thread Amit Kapila
On Mon, Aug 8, 2016 at 6:00 PM, Dilip Kumar wrote: > On Mon, Aug 8, 2016 at 5:23 PM, Amit Kapila wrote: >> >> >> Did you consider to use ERRCODE_UNDEFINED_COLUMN with error messages >> like: "type %u does not exit" or "type id %u does not exit"? Er

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Amit Kapila
l be worth > trying. > I think here expensive part would be recheck for the cases where the index value is changed to a different value (value which doesn't exist in WARM chain). You anyway have to add the new entry (key,TID) in index, but each time traversing the WARM chain would be an addit

Re: [HACKERS] [sqlsmith] Failed assertion in joinrels.c

2016-08-08 Thread Amit Kapila
On Mon, Aug 8, 2016 at 5:11 PM, Amit Kapila wrote: > On Mon, Aug 8, 2016 at 4:58 PM, Dilip Kumar wrote: >> >> On Wed, Aug 3, 2016 at 5:35 AM, Robert Haas wrote: >>> >>> I think that's just making life difficult. If nothing else, sqlsmith >>> hun

Re: [HACKERS] [sqlsmith] Failed assertion in joinrels.c

2016-08-08 Thread Amit Kapila
ing such errors. > I think this is closest error code among all existing error codes, other > options can be (ERRCODE_WRONG_OBJECT_TYPE). > Your other options and the option you choose are same. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers

Re: [HACKERS] Wait events monitoring future development

2016-08-08 Thread Amit Kapila
n care by using configurable knob. I am not aware if any other part of the design has been discussed in detail to conclude whether it has any obvious problem. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-08-06 Thread Amit Kapila
[2] - "That approach would not suit CREATE INDEX, because the approach's great strength is that the workers can run in parallel for the entire duration, since there is no merge bottleneck (this assumes good partition boundaries, which is of a bit risky assumption)" -- With Regar

Re: [HACKERS] [sqlsmith] Crash in GetOldestSnapshot()

2016-08-06 Thread Amit Kapila
* "old snapshot" logic. */ snapshot->lsn = GetXLogInsertRecPtr(); snapshot->whenTaken = GetSnapshotCurrentTimestamp(); MaintainOldSnapshotTimeMapping(snapshot->whenTaken, xmin); } -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Possible duplicate release of buffer lock.

2016-08-05 Thread Amit Kapila
why can't we ensure that we don't release the lock on target buffer in that loop or we can check in the caller such that if target buffer is valid (for a failure case), then release it. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers ma

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-05 Thread Amit Kapila
On Thu, Aug 4, 2016 at 7:46 PM, Pavel Stehule wrote: > 2016-08-04 15:37 GMT+02:00 Amit Kapila : >> >> > I dislike automatic commit or rollback here. >> > >> >> What problem you see with it, if we do so and may be mention the same >> in docs as well.

Re: [HACKERS] "Some tests to cover hash_index"

2016-08-05 Thread Amit Kapila
appropriate, how about hash_split_heap* or something like that. Register your patch in latest CF (https://commitfest.postgresql.org/10/) -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Cache Hash Index meta page.

2016-08-05 Thread Amit Kapila
t; Good idea. - if (retry) + if (opaque->hasho_prevblkno <= metap->hashm_maxbucket) This code seems to be problematic with respect to upgrades, because hasho_prevblkno will be initialized to 0x without the patch. -- With Regards, Amit Kapila. EnterpriseDB: http://www.en

Re: [HACKERS] Hash Indexes

2016-08-05 Thread Amit Kapila
and pin on bucket buffer multiple times. I have fixed it such that we release the pin on bucket buffers after we scan the last overflow page in bucket. Attached patch fixes the issue for me, let me know if you still see the issue. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enter

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-04 Thread Amit Kapila
or leave the transaction open as is currently being done in psql) and make it consistent across all clients. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] old_snapshot_threshold allows heap:toast disagreement

2016-08-03 Thread Amit Kapila
eems to be fine to just insist that we > always get a snapshot back from the snapmgr and use that to initialize > the TOAST snapshot. So I did it that way in the attached version. > > New patch attached. > Code looks good to me. I have not tested the patch. -- With Regards, A

Re: [HACKERS] old_snapshot_threshold allows heap:toast disagreement

2016-08-02 Thread Amit Kapila
On Tue, Aug 2, 2016 at 9:10 AM, Robert Haas wrote: > On Sat, Jul 30, 2016 at 8:17 AM, Amit Kapila wrote: >> On Fri, Jul 29, 2016 at 1:10 AM, Robert Haas wrote: >>> On Wed, Jul 27, 2016 at 7:26 PM, Andres Freund wrote: >>> >>> New version

Re: [HACKERS] Broken order-of-operations in parallel query latch manipulation

2016-08-02 Thread Amit Kapila
On Mon, Aug 1, 2016 at 8:45 PM, Tom Lane wrote: > Amit Kapila writes: >> On Mon, Aug 1, 2016 at 1:58 AM, Tom Lane wrote: >>> I believe this is wrong and the CHECK_FOR_INTERRUPTS needs to be before >>> or after the two latch operations. As-is, if the reason somebody s

Re: [HACKERS] Why we lost Uber as a user

2016-08-02 Thread Amit Kapila
e won't be able to reuse the line pointer till it is used in any one of the indexes and those could be reused once we make next non-HOT update). However, this should allow us not to update the indexes for which the corresponding column in tuple is not updated. I think it is a basic premise th

Re: [HACKERS] Reviewing freeze map code

2016-08-02 Thread Amit Kapila
On Tue, Aug 2, 2016 at 11:19 AM, Noah Misch wrote: > On Sat, Jul 23, 2016 at 01:25:55PM +0530, Amit Kapila wrote: >> On Mon, Jul 18, 2016 at 2:03 PM, Andres Freund wrote: >> > On 2016-07-18 10:02:52 +0530, Amit Kapila wrote: >> >> Consider the below scenario. >&g

Re: [HACKERS] Broken order-of-operations in parallel query latch manipulation

2016-07-31 Thread Amit Kapila
er places, but want to understand the problem you are seeing with current coding pattern in nodeGather.c -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] old_snapshot_threshold allows heap:toast disagreement

2016-07-30 Thread Amit Kapila
sn = lsn; +} Here, don't you need to initialize whenTaken as that is also used in TestForOldSnapshot_impl() to report error "snapshot too old". -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] pg_basebackup wish list

2016-07-29 Thread Amit Kapila
On Thu, Jul 28, 2016 at 7:34 PM, Fujii Masao wrote: > On Thu, Jul 28, 2016 at 10:16 PM, Amit Kapila wrote: > >> I think there is some value in providing >> .tar for -Z 0, > > I was thinking that "-Ft -Z0" is something like an alias of "-Ft". > T

Re: [HACKERS] System load consideration before spawning parallel workers

2016-07-29 Thread Amit Kapila
need some form of throttling w.r.t assignment of parallel workers to avoid system overload. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [Patch] RBTree iteration interface improvement

2016-07-28 Thread Amit Kapila
gt; ``` > extern void rb_begin_iterate(RBTree *rb, RBOrderControl ctrl); > extern RBNode *rb_iterate(RBTree *rb); > ``` > > As you see it doesn't allow to do multiple iterations over a single > tree. I believe it's a major flaw for a general-purpose container. > Can you explain

Re: [HACKERS] pg_basebackup wish list

2016-07-28 Thread Amit Kapila
Z 0 gets ignored or throw error or do something else? If first, then I think it is better to mention the same in docs. [1] - https://www.postgresql.org/message-id/CAMkU%3D1zzj0et2x9fCqxMGJ6XP-FtMSUwtNQGwF01698FRWQ6uA%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterpri

Re: [HACKERS] pg_basebackup wish list

2016-07-28 Thread Amit Kapila
? > > Yes, I think. What about the attached patch? > What if user tries to use -Z 0 with format as tar, won't it generate base.tar without any compression? I am not sure if that is what Jeff intends to say in his proposal. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] copyParamList

2016-07-27 Thread Amit Kapila
On Wed, Jul 27, 2016 at 6:46 PM, Robert Haas wrote: > On Wed, Jul 27, 2016 at 2:20 AM, Amit Kapila wrote: >> Okay, that makes sense to me apart from minor issue reported by >> Andrew. I think we might want to slightly rephrase the comments on >> top of copyParamList() which

Re: [HACKERS] copyParamList

2016-07-26 Thread Amit Kapila
On Wed, Jul 27, 2016 at 2:03 AM, Robert Haas wrote: > On Fri, Jul 8, 2016 at 2:28 AM, Amit Kapila wrote: >> On Tue, May 31, 2016 at 10:10 PM, Robert Haas wrote: >>> On Fri, May 27, 2016 at 6:07 PM, Andrew Gierth >>> wrote: >>>> copyParamList does not re

Re: [HACKERS] Reviewing freeze map code

2016-07-26 Thread Amit Kapila
On Wed, Jul 27, 2016 at 3:24 AM, Robert Haas wrote: > On Sat, Jul 23, 2016 at 3:55 AM, Amit Kapila wrote: >> Attached patch fixes the problem for me. Note, I have not tried to >> reproduce the problem for heap_lock_updated_tuple_rec(), but I think >> if you are convinced w

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2016-07-26 Thread Amit Kapila
case is important enough to proceed with one of those options or may be consider some another approach. Does any body else see the use case reported by Chapman important enough that we try to have some solution for it in-core? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterpris

Re: [HACKERS] Curing plpgsql's memory leaks for statement-lifespan values

2016-07-25 Thread Amit Kapila
On Sun, Jul 24, 2016 at 9:17 PM, Tom Lane wrote: > Amit Kapila writes: > >> Wouldn't it be better, if each nested sub-block (which is having an >> exception) has a separate "SPI Proc", "SPI Exec" contexts which would >> be destroyed at sub-block

Re: [HACKERS] Curing plpgsql's memory leaks for statement-lifespan values

2016-07-24 Thread Amit Kapila
and we then switch to "SPI Proc" for the memory that is required for longer duration. I think that might not be good, if we have to switch at many places, but OTOH the same will be required for a new statement-level execution context as well. In short, why do you think it is better to create

Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility

2016-07-23 Thread Amit Kapila
dToWAL()) and doesn't look to impact other parts of system, although some more analysis is needed to confirm the same, but I think the point to make it optional doesn't seem convincing to me. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent

<    5   6   7   8   9   10   11   12   13   14   >