Re: TopoSort() fix

[Rui Hai Jiang]

I'll try to figure out some  scenarios to do the test. A parallel process
group is needed for the test.

Actually I was trying to do some testing against the locking mechanism. I
happened to see this issue.

On Wed, Jul 3, 2019 at 9:38 PM Robert Haas  wrote:

> On Tue, Jul 2, 2019 at 11:23 AM Tom Lane  wrote:
> > Rui Hai Jiang  writes:
> > > I think I found an issue in the TopoSort() function.
> >
> > This indeed seems like a live bug introduced by a1c1af2a.
> > Robert?
>
> This is pretty thoroughly swapped out of my head, but it looks like
> that analysis might be correct.
>
> Is it practical to come up with a test case that demonstrates the problem?
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: TopoSort() fix

[Rui Hai Jiang]

Could the attached patch fix this issue? Or does any one else plan to fix
it?

If people are busy and have not  time, I can go ahead to fix it.  To fix
this issue, do we need a patch for each official branch?


Regards,
Ruihai

On Tue, Jul 2, 2019 at 11:23 PM Tom Lane  wrote:

> Rui Hai Jiang  writes:
> > I think I found an issue in the TopoSort() function.
>
> This indeed seems like a live bug introduced by a1c1af2a.
> Robert?
>
> regards, tom lane
>

TopoSort() fix

[Rui Hai Jiang]

Hi Hackers,

I think I found an issue in the TopoSort() function.

As the comments say,

/* .
 * .. If there are any other processes
 * in the same lock group on the queue, set their number of
 * beforeConstraints to -1 to indicate that they should be
emitted
 * with their groupmates rather than considered separately.
 */

If the line "break;" exists, there is no chance to set beforeConstraints to
-1 for other processes in the same lock group.

So, I think we need delete the line "break;" . See the patch.

I just took a look, and I found all the following versions have this line .


postgresql-12beta2, postgresql-12beta1, postgresql-11.4,
postgresql-11.3,postgresql-11.0,
postgresql-10.9,postgresql-10.5, postgresql-10.0


Thanks,
Ruihai


TopoSort.patch
Description: Binary data

Re: How to produce a Soft Block case of Deadlock Detection?

[Rui Hai Jiang]

I finally found this.

https://www.postgresql.org/message-id/29104.1182785028%40sss.pgh.pa.us

This is very useful to understand the Soft Block.

On Wed, Jun 19, 2019 at 7:18 PM Rui Hai Jiang  wrote:

> Hello, hackers.
>
> Any body know how to produce a Soft Block case of Deadlock Detection?
> I have produced the Hard Block case, but can't produce the Soft Block case.
>
>
> I read the design: src/backend/storage/lmgr/README. It reads,
>
> "If a process A is behind a process B in some lock's wait queue, and
> their requested locks conflict, then we must say that A waits for B, since
> ProcLockWakeup will never awaken A before B.  This creates additional
> edges in the WFG.  We call these "soft" edges, as opposed to the "hard"
> edges induced by locks already held.  Note that if B already holds any
> locks conflicting with A's request, then their relationship is a hard edge
> not a soft edge."
>
>
> But after trying many testing, I couldn't figure out how to produce a Soft
> Block.
>
> Following is what I did.
>
> * Hard Block Case
>
> ** Prepare
>
> create table t1 ( id int primary key, test int );
> create table t2 ( id int primary key, test int );
>
> insert into t1 values (10,10);
> insert into t2 values (20,20);
>
> ** test
>
> step1/backend1:
> begin;
> update t1 set test=11 where id=10;
>
> step2/backend2:
>   begin;
>   update t2 set test=21 where id=20;
>
> step3/backend1:
>   update t2 set test=21 where id=20;
>
> step4/process2: /*deadlock detected*/
>   update t1 set test=11 where id=10;
>
>
>
>
> * Soft Block Case
>
> ** Prepare
>
> create table t1 ( id int primary key, test int );
>
> create table t2 ( id int primary key, test int );
>
> create table t3 ( id int primary key, test int );
>
> insert into t1 values (10,10);
> insert into t2 values (20,20);
> insert into t3 values (30,30);
>
> ** test
>
> step1/backend1: /*lock t1.row1*/
> begin;
> select * from t1 where id=10 for update;
>
>
> step2/backend2: /*lock t2.row1*/
> begin;
> select * from t2 where id=20 for no key update;
>
> step3/backend3: /*lock t2.row1*/
> begin;
> select * from t2 where id=20 for key share;
>
> step4/backend4:/*lock t3.row1*/
> begin;
> select * from t3 where id=30 for update;
>
> step5/backend4：/*try to lock t1.row1*/
> update t1 set id=11 where id=10;
>
> step6/backend3：/*try to lock t3.row1*/
> update t3 set id=31 where id=30;
>
> step7/backend5：/*try to lock t2.row1, hope to create a soft edge*/
> begin;
> update t2 set id=21 where id=20;
>
> step8/backend1：/*try to lock t2.row1*/   /*Expect to detect soft block,
> but there seems no soft block*/
> update t2 set test=21 where id=20;
>
>

How to produce a Soft Block case of Deadlock Detection?

[Rui Hai Jiang]

Hello, hackers.

Any body know how to produce a Soft Block case of Deadlock Detection?
I have produced the Hard Block case, but can't produce the Soft Block case.


I read the design: src/backend/storage/lmgr/README. It reads,

"If a process A is behind a process B in some lock's wait queue, and
their requested locks conflict, then we must say that A waits for B, since
ProcLockWakeup will never awaken A before B.  This creates additional
edges in the WFG.  We call these "soft" edges, as opposed to the "hard"
edges induced by locks already held.  Note that if B already holds any
locks conflicting with A's request, then their relationship is a hard edge
not a soft edge."


But after trying many testing, I couldn't figure out how to produce a Soft
Block.

Following is what I did.

* Hard Block Case

** Prepare

create table t1 ( id int primary key, test int );
create table t2 ( id int primary key, test int );

insert into t1 values (10,10);
insert into t2 values (20,20);

** test

step1/backend1:
begin;
update t1 set test=11 where id=10;

step2/backend2:
  begin;
  update t2 set test=21 where id=20;

step3/backend1:
  update t2 set test=21 where id=20;

step4/process2: /*deadlock detected*/
  update t1 set test=11 where id=10;




* Soft Block Case

** Prepare

create table t1 ( id int primary key, test int );

create table t2 ( id int primary key, test int );

create table t3 ( id int primary key, test int );

insert into t1 values (10,10);
insert into t2 values (20,20);
insert into t3 values (30,30);

** test

step1/backend1: /*lock t1.row1*/
begin;
select * from t1 where id=10 for update;


step2/backend2: /*lock t2.row1*/
begin;
select * from t2 where id=20 for no key update;

step3/backend3: /*lock t2.row1*/
begin;
select * from t2 where id=20 for key share;

step4/backend4:/*lock t3.row1*/
begin;
select * from t3 where id=30 for update;

step5/backend4：/*try to lock t1.row1*/
update t1 set id=11 where id=10;

step6/backend3：/*try to lock t3.row1*/
update t3 set id=31 where id=30;

step7/backend5：/*try to lock t2.row1, hope to create a soft edge*/
begin;
update t2 set id=21 where id=20;

step8/backend1：/*try to lock t2.row1*/   /*Expect to detect soft block, but
there seems no soft block*/
update t2 set test=21 where id=20;

Is it possible and worthy to optimize scanRTEForColumn()?

[Rui Hai Jiang]

Hello,

When I run a select query, e.g. select id from t,  all columns in table "t" are 
checked to see if a column named "id" exists or not, and a Var is created for 
"id" if the column does exist.

Function scanRTEForColumn() does this job.

But I see in scanRTEForColumn(), the loop does not stop when a match is found, 
it continues to compare all other columns. And this will waste lots of 
computing.


I guess there may be some reasons for this. But I don't know yet.  I just 
wonder if it is possible and worthy to optimize this. And please note, "select 
*" does not call this function.


Node *
scanRTEForColumn(ParseState *pstate, RangeTblEntry *rte, char *colname,
 int location, int fuzzy_rte_penalty,
 FuzzyAttrMatchState *fuzzystate)
{
  foreach(c, rte->eref->colnames)
{
const char *attcolname = strVal(lfirst(c));
attnum++;
if (strcmp(attcolname, colname) == 0)
{
 if (result)
ereport(ERROR,

(errcode(ERRCODE_AMBIGUOUS_COLUMN),
 errmsg("column reference 
\"%s\" is ambiguous",
colname),
 parser_errposition(pstate, 
location)));
var = make_var(pstate, rte, attnum, location);
/* Require read access to the column */
markVarForSelectPriv(pstate, var, rte);
result = (Node *) var;
}

/* Updating fuzzy match state, if provided. */
if (fuzzystate != NULL)
updateFuzzyAttrMatchState(fuzzy_rte_penalty, fuzzystate,
  rte, 
attcolname, colname, attnum);
}

/*
 * If we have a unique match, return it.  Note that this allows a user
 * alias to override a system column name (such as OID) without error.
 */
if (result)
return result;


.
}

RE: How is the PostgreSQL debuginfo file generated

[Rui Hai Jiang]

Thank you for your help.



I have tried many ways you suggested.  I was not familiar with this area, so I 
did spend lots of time trying.



At last, I found the easiest way to do this is to build a new SRPM package, 
then build other RPMs from the new SRPM.



Thanks a lot,

Ruihai




From: Tom Lane <t...@sss.pgh.pa.us>
Sent: Friday, November 24, 2017 12:50:37 AM
To: Rui Hai Jiang
Cc: pgsql-hack...@postgresql.org
Subject: Re: How is the PostgreSQL debuginfo file generated

Rui Hai Jiang <ruihaiji...@msn.com> writes:
> I’m wondering how to build the debuginfo package from  the PostgreSQL source.
> For example to generate something like this one :   
> postgresql91-debuginfo.x86_64.

On platforms where such things exist, that's handled by the packaging
system, not by PG proper.  You should proceed by making a new SRPM
and building RPMs from that.

regards, tom lane