RE: [SLUG] Maximum process ID

[Robert Collins]

On Mon, 2004-10-18 at 13:12 +1000, Michael Kraus wrote:
> No, this definetly opens up race conditions and lots of rollbacks and
> reprocessing.
> 
> Eg (in pseudocode).
> 
> id = execute("SELECT MAX(id) FROM tablename") + 1;
> ...
> result = execute("INSERT INTO tablename VALUE(id, )");
> if (!result) { rollback(); repeat(); }
> 
> 
> Either that or it would cause the database performance to slow down
> whilst it holds a that table. Better just to have another unique
> identifier in my reckoning.

Have you profiled the performance of nextseq() in postgresql ? Or are
you just speculating?

-Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Robert Collins]

On Mon, 2004-10-18 at 12:27 +1000, Michael Kraus wrote:
>  Err, how is there a race condition? (Is there something I'm missing?)

Yes. processes can die and be spawned again very quickly. The
granularity of your timestamp must be > the maximum time to spawn a
process that was inserting that row. pids are not unique - they are
reused, and timestamps just narrow the race, they don't eliminate it. To
eliminate a race condition, you usually need an atomic operation of some
sort. (A la unique row id's as already discussed by other contributors
to this thread).

> >From my reasoning - every process running at a particular point in time
> has a unique process ID (i.e. no two processes running at the same time
> will have the same PID), therefore a time/date stamp coupled with a PID
> gives a unique identifier. 

datetime stamp != point in time.
multiple machines being introduced would also break this assumption.

> (For convenience and storage sake, a separate
> primary key is used as well, but as that is created by the database
> system itself, we don't know what that is at time of insertion.)

Well, you do - see the other responses again.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[James Gregory]

On Mon, Oct 18, 2004 at 01:19:35PM +1000, Jamie Wilkinson wrote:
> This one time, at band camp, Michael Kraus wrote:
> >
> >No, this definetly opens up race conditions and lots of rollbacks and
> >reprocessing.
> >
> >Eg (in pseudocode).
> >
> >id = execute("SELECT MAX(id) FROM tablename") + 1;
> >...
> >result = execute("INSERT INTO tablename VALUE(id, )");
> >if (!result) { rollback(); repeat(); }
> 
> Your +1 is broken.
> 
> id = select nextval(sequence);
> insert into tablename (id, ...) values ($id, ...)

So, the key part that I feel isn't being explained here is that this is
a common problem when you're dealing with databases, and any decent
database will already have mechanisms to deal with it that have been
thoroughly tested and will likely be tuned for good performance too.

Postgres has the notion of "sequences", which are entities distinct from
any table that you can use to get a new, unique number, atomically.

Now, in the case of just fetching a value from a sequence, there's no
race-condition. It's an atomic operation (as far as I'm aware). Note
that once a number is grabbed from the sequence it will *never* appear
again. Not even if you roll the transaction back.

Now, someone will probably correct my rather rash use of the word
"atomic", but I think you follow what I mean.

HTH,

James.

-- 
"Now, there are no problems  only opportunities. However, this seemed to be an
insurmountable opportunity."
 - http://www.surfare.net/~toolman/temp/diagram.html
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Jamie Wilkinson]

This one time, at band camp, Michael Kraus wrote:
>I already have that covered. Now consider how in embedded SQL you are
>going to know the value of id when executing "INSERT INTO tablename
>VALUES(null, ...)", where the table structure is (id INT UNIQUE PRIMARY
>KEY AUTO_INCREMENT, ...)
>
>Hence I need an identifier that the inserting process knows about to
>grab a handle back to that exact row.

Our code does this by selecting the next value from the sequence, then
inserting in to the table with that sequence as the ID.  Thus, the
uniqueness of the row ID is guaranteed.

-- 
[EMAIL PROTECTED]   http://spacepants.org/jaq.gpg
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Jamie Wilkinson]

This one time, at band camp, Michael Kraus wrote:
>
>No, this definetly opens up race conditions and lots of rollbacks and
>reprocessing.
>
>Eg (in pseudocode).
>
>id = execute("SELECT MAX(id) FROM tablename") + 1;
>...
>result = execute("INSERT INTO tablename VALUE(id, )");
>if (!result) { rollback(); repeat(); }

Your +1 is broken.

id = select nextval(sequence);
insert into tablename (id, ...) values ($id, ...)

-- 
[EMAIL PROTECTED]   http://spacepants.org/jaq.gpg
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Benno]

On Mon Oct 18, 2004 at 13:12:59 +1000, Michael Kraus wrote:
>
>No, this definetly opens up race conditions and lots of rollbacks and
>reprocessing.
>
>Eg (in pseudocode).
>
>id =3D execute("SELECT MAX(id) FROM tablename") + 1;
>...
>result =3D execute("INSERT INTO tablename VALUE(id, )");
>if (!result) { rollback(); repeat(); }
>
>
>Either that or it would cause the database performance to slow down
>whilst it holds a that table. Better just to have another unique
>identifier in my reckoning.

No, not max(id). I'm talking from postgres, but you don't select max(id),
you use a sequence to get the next index and then use that to insert.

It seems like Glen came up with the way to do it in MySQL.

Benno
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

No, this definetly opens up race conditions and lots of rollbacks and
reprocessing.

Eg (in pseudocode).

id = execute("SELECT MAX(id) FROM tablename") + 1;
...
result = execute("INSERT INTO tablename VALUE(id, )");
if (!result) { rollback(); repeat(); }


Either that or it would cause the database performance to slow down
whilst it holds a that table. Better just to have another unique
identifier in my reckoning.

Regards,

 

Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net

 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: Benno [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 1:09 PM
To: torquemada
Cc: Michael Kraus; [EMAIL PROTECTED]
Subject: Re: [SLUG] Maximum process ID

On Mon Oct 18, 2004 at 12:58:42 +1000, torquemada wrote:
>
>OID's, (object-ID's) are made exactly for this purpose.
>if your database vendor does not provide this feature, lobby for it.
>ie PostgreSQL supports OID's and they are pretty much guaranteed to be 
>unique

Well, the better idea is to start a transaction, get the next id from a
seqeunce, and then use this as the UID. I'm not sure if MySQL has
sequences, but I'm *sure* that this is a problem that is solved in some
way in the MySQL.

Benno
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Benno]

On Mon Oct 18, 2004 at 13:06:30 +1000, Michael Kraus wrote:
>G'day,
>
>Yes - but that isn't for me as a user of MySQL to have to worry about -
>the developers are the ones concerned about the internals. I only need
>to use x amount of space, and so I only choose to use x about of space
>not x+y amount if I can help it.
>
>Hrmm... Ask a simple question - get criticisism for choice of
>implementation.=20
>

Hrm... Provide a useful answer to a question - get criticised for
trying to provide a rational answer.


-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Glen Turner]

Michael Kraus wrote:
I already have that covered. Now consider how in embedded SQL you are
going to know the value of id when executing "INSERT INTO tablename
VALUES(null, ...)", where the table structure is (id INT UNIQUE PRIMARY
KEY AUTO_INCREMENT, ...)
Hence I need an identifier that the inserting process knows about to
grab a handle back to that exact row.
See LAST_INSERT_ID(), which returns the value of the newly-created
autoincrement field.
Cheers,
Glen
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Benno]

On Mon Oct 18, 2004 at 12:58:42 +1000, torquemada wrote:
>
>OID's, (object-ID's) are made exactly for this purpose.
>if your database vendor does not provide this feature, lobby for it.
>ie PostgreSQL supports OID's and they are pretty much guaranteed to be
>unique

Well, the better idea is to start a transaction, get the next id
from a seqeunce, and then use this as the UID. I'm not sure if MySQL has
sequences, but I'm *sure* that this is a problem that is solved in some
way in the MySQL.

Benno
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

See my reply to Glenn.


Regards,

 
Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net

 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: torquemada [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 12:59 PM
To: Michael Kraus
Cc: [EMAIL PROTECTED]
Subject: RE: [SLUG] Maximum process ID


OID's, (object-ID's) are made exactly for this purpose.
if your database vendor does not provide this feature, lobby for it.
ie PostgreSQL supports OID's and they are pretty much guaranteed to be
unique

cordially,
Torquemada

On Mon, 18 Oct 2004, Michael Kraus wrote:

> G'day again...
>
> Actually, I want the PID for neither of the reasons you've said. 
> Rather, I'm using the PID and the date/time as a unique identifier to 
> a database row. The main contents of the row could occur more than 
> once in the table I'm working with - however we want to link this 
> exact row with rows in other tables in the database. Initially I 
> thought about using the date/time as an identifier (which I was 
> storing anyway), but remembered that two instances of the script could

> be running concurrently, so decided to incorporate the PID to ensure
uniqueness.
>
>
> Regards,
>
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

G'day,

Yes - but that isn't for me as a user of MySQL to have to worry about -
the developers are the ones concerned about the internals. I only need
to use x amount of space, and so I only choose to use x about of space
not x+y amount if I can help it.

Hrmm... Ask a simple question - get criticisism for choice of
implementation. 


Regards,

 
Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net


The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: Benno [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 12:14 PM
To: Michael Kraus
Cc: Benno; Robert Collins; Glen Turner; [EMAIL PROTECTED]
Subject: Re: [SLUG] Maximum process ID

On Mon Oct 18, 2004 at 11:56:02 +1000, Michael Kraus wrote:
>G'day...
>
>Not necessarily, MEDIUMINT is probably (I don't know I haven't checked 
>the code) stored as its own defined type, and may be stored differently

>on disk and in memory than an INT value (which is 2^32).

Well, I'm sure it is stored differently on disk, but in memory it is
going to be an int. Unless they want things to be really slow.

>I expect that MySQL has either implemented some difference and hence 
>that is why they provide a variety of data structures. Being the good 
>database student that I was, I'm wanting to use the most appropriate 
>integer structure for my purpose.
>
>The base data types implemented in a language (in this case we are 
>referring to C, however I can't say if MySQL is programmed completely 
>in C, but I could take a look at the source if I really needed to know)

>influence rather than dictate the data types used by the program 
>(MySQL). (Especially when we are talking about C as it's a language 
>that allows you to define your own datatypes and how they are handled.)

Well, since sizeof(pid_t) gives you 4, why not use an int in your
database?

Benno
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[torquemada]

OID's, (object-ID's) are made exactly for this purpose.
if your database vendor does not provide this feature, lobby for it.
ie PostgreSQL supports OID's and they are pretty much guaranteed to be
unique

cordially,
Torquemada

On Mon, 18 Oct 2004, Michael Kraus wrote:

> G'day again...
>
> Actually, I want the PID for neither of the reasons you've said. Rather,
> I'm using the PID and the date/time as a unique identifier to a database
> row. The main contents of the row could occur more than once in the
> table I'm working with - however we want to link this exact row with
> rows in other tables in the database. Initially I thought about using
> the date/time as an identifier (which I was storing anyway), but
> remembered that two instances of the script could be running
> concurrently, so decided to incorporate the PID to ensure uniqueness.
>
>
> Regards,
>
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

I already have that covered. Now consider how in embedded SQL you are
going to know the value of id when executing "INSERT INTO tablename
VALUES(null, ...)", where the table structure is (id INT UNIQUE PRIMARY
KEY AUTO_INCREMENT, ...)

Hence I need an identifier that the inserting process knows about to
grab a handle back to that exact row.

Regards,
 

Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net
 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: Glen Turner [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 12:41 PM
To: Michael Kraus
Cc: Robert Collins; [EMAIL PROTECTED]
Subject: Re: [SLUG] Maximum process ID

Michael Kraus wrote:
> G'day again... 
> 
> Actually, I want the PID for neither of the reasons you've said. 
> Rather, I'm using the PID and the date/time as a unique identifier to 
> a database row. The main contents of the row could occur more than 
> once in the table I'm working with - however we want to link this 
> exact row with rows in other tables in the database. Initially I 
> thought about using the date/time as an identifier (which I was 
> storing anyway), but remembered that two instances of the script could

> be running concurrently, so decided to incorporate the PID to ensure
uniqueness.

If your database can't hand you a unique ID for a row of a table then
find another one.  Doesn't MySQL have autoincrement fields which do
this:

   CREATE TABLE t (id INT  NOT NULL  PRIMARY KEY  AUTO_INCREMENT,
   data VARCHAR(100));

   INSERT INTO t (id, data) VALUES (NULL, 'data');

where the NULL value results in the insertion of a unique number one
greater than any other value in that field?

Note that there's no race condition, as if both clients hand up NULL
simultaneously the DB server will assign a value to one client's insert
and then the other client's insert.

Cheers,
Glen
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Glen Turner]

Michael Kraus wrote:
G'day again... 

Actually, I want the PID for neither of the reasons you've said. Rather,
I'm using the PID and the date/time as a unique identifier to a database
row. The main contents of the row could occur more than once in the
table I'm working with - however we want to link this exact row with
rows in other tables in the database. Initially I thought about using
the date/time as an identifier (which I was storing anyway), but
remembered that two instances of the script could be running
concurrently, so decided to incorporate the PID to ensure uniqueness.
If your database can't hand you a unique ID for a row of a table then
find another one.  Doesn't MySQL have autoincrement fields which do
this:
  CREATE TABLE t (id INT  NOT NULL  PRIMARY KEY  AUTO_INCREMENT,
  data VARCHAR(100));
  INSERT INTO t (id, data) VALUES (NULL, 'data');
where the NULL value results in the insertion of a unique number one
greater than any other value in that field?
Note that there's no race condition, as if both clients hand up NULL
simultaneously the DB server will assign a value to one client's insert
and then the other client's insert.
Cheers,
Glen
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

 
Err, how is there a race condition? (Is there something I'm missing?)

>From my reasoning - every process running at a particular point in time
has a unique process ID (i.e. no two processes running at the same time
will have the same PID), therefore a time/date stamp coupled with a PID
gives a unique identifier. (For convenience and storage sake, a separate
primary key is used as well, but as that is created by the database
system itself, we don't know what that is at time of insertion.)

Even though I'm not using the two as a primary key, they combination of
the two are a valid key candidate. A identity field isn't required for
the sake of having one. Some tables have a primary key consisting of the
whole row. (Eg. a table that provides a many to many relationship
between two other tables.)

However, I'm happy to be proven wrong - is there something I'm missing
here?

Regards,

 
Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net

 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: Robert Collins [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 12:03 PM
To: Michael Kraus
Cc: Glen Turner; [EMAIL PROTECTED]
Subject: RE: [SLUG] Maximum process ID

On Mon, 2004-10-18 at 11:57 +1000, Michael Kraus wrote:
> G'day again... 
> 
> Actually, I want the PID for neither of the reasons you've said. 
> Rather, I'm using the PID and the date/time as a unique identifier to 
> a database row. The main contents of the row could occur more than 
> once in the table I'm working with - however we want to link this 
> exact row with rows in other tables in the database. Initially I 
> thought about using the date/time as an identifier (which I was 
> storing anyway), but remembered that two instances of the script could

> be running concurrently, so decided to incorporate the PID to ensure
uniqueness.

Theres still a race condition there. If you want a unique id, use a
unique id - an identity field for instance.

Rob

--
GPG key available at: <http://www.robertcollins.net/keys.txt>.
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Benno]

On Mon Oct 18, 2004 at 11:56:02 +1000, Michael Kraus wrote:
>G'day...
>
>Not necessarily, MEDIUMINT is probably (I don't know I haven't checked
>the code) stored as its own defined type, and may be stored differently
>on disk and in memory than an INT value (which is 2^32).

Well, I'm sure it is stored differently on disk, but in memory it is going
to be an int. Unless they want things to be really slow.

>I expect that MySQL has either implemented some difference and hence
>that is why they provide a variety of data structures. Being the good
>database student that I was, I'm wanting to use the most appropriate
>integer structure for my purpose.
>
>The base data types implemented in a language (in this case we are
>referring to C, however I can't say if MySQL is programmed completely in
>C, but I could take a look at the source if I really needed to know)
>influence rather than dictate the data types used by the program
>(MySQL). (Especially when we are talking about C as it's a language that
>allows you to define your own datatypes and how they are handled.)

Well, since sizeof(pid_t) gives you 4, why not use an int in your database?

Benno
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Robert Collins]

On Mon, 2004-10-18 at 11:57 +1000, Michael Kraus wrote:
> G'day again... 
> 
> Actually, I want the PID for neither of the reasons you've said. Rather,
> I'm using the PID and the date/time as a unique identifier to a database
> row. The main contents of the row could occur more than once in the
> table I'm working with - however we want to link this exact row with
> rows in other tables in the database. Initially I thought about using
> the date/time as an identifier (which I was storing anyway), but
> remembered that two instances of the script could be running
> concurrently, so decided to incorporate the PID to ensure uniqueness.

Theres still a race condition there. If you want a unique id, use a
unique id - an identity field for instance.

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

G'day again... 

Actually, I want the PID for neither of the reasons you've said. Rather,
I'm using the PID and the date/time as a unique identifier to a database
row. The main contents of the row could occur more than once in the
table I'm working with - however we want to link this exact row with
rows in other tables in the database. Initially I thought about using
the date/time as an identifier (which I was storing anyway), but
remembered that two instances of the script could be running
concurrently, so decided to incorporate the PID to ensure uniqueness.


Regards,


Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]

ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net
 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: Robert Collins [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 11:33 AM
To: Michael Kraus
Cc: Glen Turner; [EMAIL PROTECTED]
Subject: RE: [SLUG] Maximum process ID

On Mon, 2004-10-18 at 11:06 +1000, Michael Kraus wrote:
> G'day...
> 
> Whilst this discussion is pretty great - I would like to note that the

> practical purpose of knowing the size of a PID is so that I can store 
> the current processes' PID in an MySQL database, and the process is a 
> Perl script.

The key element is why do you need the pid? There is usually only two
good reasons to know a process's pid:
1) to kill it
2) to tell it to cleanly shutdown.

And then you still need to check its the same command line you thought
it was.

Do you want the pid for one of those reasons, or another?... 
Is it so that you know what the current processs? (so what happens when
it dies and is replaced, with the same pid, with another process -
perhaps even the same script again). Is it for auditing? What happens if
the perl script runs twice at once ? 


Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

G'day...

Not necessarily, MEDIUMINT is probably (I don't know I haven't checked
the code) stored as its own defined type, and may be stored differently
on disk and in memory than an INT value (which is 2^32).

I expect that MySQL has either implemented some difference and hence
that is why they provide a variety of data structures. Being the good
database student that I was, I'm wanting to use the most appropriate
integer structure for my purpose.

The base data types implemented in a language (in this case we are
referring to C, however I can't say if MySQL is programmed completely in
C, but I could take a look at the source if I really needed to know)
influence rather than dictate the data types used by the program
(MySQL). (Especially when we are talking about C as it's a language that
allows you to define your own datatypes and how they are handled.)

So, I neither think that MEDIUMINTS are handled internally as a 2^32 (or
if it is, the other 8 bits may be used snazzily for something else) nor
that the MySQL developers are on crack. (Who knows, maybe it gives them
the ability to think in high level abstraction, but I somehow doubt it!)

Regards,

 

Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net

 
The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: Benno [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 11:17 AM
To: Michael Kraus
Cc: Robert Collins; Glen Turner; [EMAIL PROTECTED]
Subject: Re: [SLUG] Maximum process ID

On Mon Oct 18, 2004 at 11:06:34 +1000, Michael Kraus wrote:
>G'day...
>
>Whilst this discussion is pretty great - I would like to note that the 
>practical purpose of knowing the size of a PID is so that I can store 
>the current processes' PID in an MySQL database, and the process is a 
>Perl script.
>
>Based on the discussion so far, I'm making a pretty safe bet that a 
>MEDIUMINT value (2^24) unsigned (0 - 16,777,215) will be more than 
>sufficient. I don't know of a /running/ process that could have a 
>negative value for its PID. (Any reasons why this wouldn't be
>sufficient?)
>

Why not just use an int (2^32). It's storage requirement are going to be
no different to those required by (2^24) anyway. Internally 2^24 are
going to be implemented as int anyway. (Assuming the MySQL developers
weren't on crack.)

Benno
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Robert Collins]

On Mon, 2004-10-18 at 11:06 +1000, Michael Kraus wrote:
> G'day...
> 
> Whilst this discussion is pretty great - I would like to note that the
> practical purpose of knowing the size of a PID is so that I can store
> the current processes' PID in an MySQL database, and the process is a
> Perl script.

The key element is why do you need the pid? There is usually only two
good reasons to know a process's pid:
1) to kill it
2) to tell it to cleanly shutdown.

And then you still need to check its the same command line you thought
it was.

Do you want the pid for one of those reasons, or another?... 
Is it so that you know what the current processs? (so what happens when
it dies and is replaced, with the same pid, with another process -
perhaps even the same script again). Is it for auditing? What happens if
the perl script runs twice at once ? 


Rob
-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Benno]

On Mon Oct 18, 2004 at 11:06:34 +1000, Michael Kraus wrote:
>G'day...
>
>Whilst this discussion is pretty great - I would like to note that the
>practical purpose of knowing the size of a PID is so that I can store
>the current processes' PID in an MySQL database, and the process is a
>Perl script.
>
>Based on the discussion so far, I'm making a pretty safe bet that a
>MEDIUMINT value (2^24) unsigned (0 - 16,777,215) will be more than
>sufficient. I don't know of a /running/ process that could have a
>negative value for its PID. (Any reasons why this wouldn't be
>sufficient?)
>

Why not just use an int (2^32). It's storage requirement are going to
be no different to those required by (2^24) anyway. Internally 2^24 are
going to be implemented as int anyway. (Assuming the MySQL developers 
weren't on crack.)

Benno
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

RE: [SLUG] Maximum process ID

[Michael Kraus]

G'day...

Whilst this discussion is pretty great - I would like to note that the
practical purpose of knowing the size of a PID is so that I can store
the current processes' PID in an MySQL database, and the process is a
Perl script.

Based on the discussion so far, I'm making a pretty safe bet that a
MEDIUMINT value (2^24) unsigned (0 - 16,777,215) will be more than
sufficient. I don't know of a /running/ process that could have a
negative value for its PID. (Any reasons why this wouldn't be
sufficient?)

(Having said all this, it's still quite fun and educational to watch all
this great discussion.)

Thanks!

Regards,
 
Michael S. E. Kraus
Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]

ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
http://www.wildtechnology.net

 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.


-Original Message-
From: Robert Collins [mailto:[EMAIL PROTECTED] 
Sent: Monday, 18 October 2004 10:53 AM
To: Glen Turner
Cc: Michael Kraus; [EMAIL PROTECTED]
Subject: Re: [SLUG] Maximum process ID

On Sat, 2004-10-16 at 14:37 +0930, Glen Turner wrote:
> Robert Collins wrote:
> > On Fri, 2004-10-15 at 15:33 +1000, Michael Kraus wrote:
> >
> >>I'm developing a database application that  uses the inserting 
> >>processes pid. Problem is that I'm wondering how big this pid should
be?
> > 
> > 
> > sizeof(pid_t) IIRC. Its system specific.
> 
> Technically, PID is also signed since
>pid_t f;
>f = fork();
> can return -1 on error.
> 
> Personally I'd go with a signed 32bit integer, since that is what the 
> GNU C library uses for pid_t. And its really the C library's 
> definition, rather than the kernel's, which counts.
> 
> Any kernel in it's right mind is going to avoid negative PIDs as that 
> isn't the user's expectation; I note the Amos' message saying that the

> Linux kernel currently allows PID values 1 to 4,194,303.

This kindof misses the point. pid_t is *hardware and software* platform
specific. The libc typedef on your platform is the only one that matters
- and its not the same everywhere.

Going with 32bit signed int may make sense if Michaels platform uses
that for pid_t, on all the machines doing inserts, consistently. But its
highly likely that whatever intended use the pid field in the database
has, its in appropriate. (Why is left as an exercise for the reader).

Rob

--
GPG key available at: <http://www.robertcollins.net/keys.txt>.
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Robert Collins]

On Sat, 2004-10-16 at 14:37 +0930, Glen Turner wrote:
> Robert Collins wrote:
> > On Fri, 2004-10-15 at 15:33 +1000, Michael Kraus wrote:
> >
> >>I'm developing a database application that  uses the inserting processes
> >>pid. Problem is that I'm wondering how big this pid should be?
> > 
> > 
> > sizeof(pid_t) IIRC. Its system specific.
> 
> Technically, PID is also signed since
>pid_t f;
>f = fork();
> can return -1 on error.
> 
> Personally I'd go with a signed 32bit integer, since that is what
> the GNU C library uses for pid_t. And its really the C library's
> definition, rather than the kernel's, which counts.
> 
> Any kernel in it's right mind is going to avoid negative PIDs as
> that isn't the user's expectation; I note the Amos' message saying
> that the Linux kernel currently allows PID values 1 to 4,194,303.

This kindof misses the point. pid_t is *hardware and software* platform
specific. The libc typedef on your platform is the only one that matters
- and its not the same everywhere.

Going with 32bit signed int may make sense if Michaels platform uses
that for pid_t, on all the machines doing inserts, consistently. But its
highly likely that whatever intended use the pid field in the database
has, its in appropriate. (Why is left as an exercise for the reader).

Rob

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Christopher Vance]

On Fri, Oct 15, 2004 at 03:33:57PM +1000, Michael Kraus wrote:
I'm developing a database application that  uses the inserting processes
pid. Problem is that I'm wondering how big this pid should be?
It depends on your platform:
traditional 3
Linux   32768
FreeBSD 9
Solaris tunable up to 99
Plan9   just uses 32 bits - I can't be bothered booting
to check if it's signed or not
(You might care to +/- 1 on the values above, as some may be
inclusive, and some exclusive.)
Given that this is mostly a Linux list, you know which number to use,
and why things will break if you move your stuff elsewhere.  :-)
--
Christopher Vance
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Glen Turner]

Robert Collins wrote:
On Fri, 2004-10-15 at 15:33 +1000, Michael Kraus wrote:
I'm developing a database application that  uses the inserting processes
pid. Problem is that I'm wondering how big this pid should be?

sizeof(pid_t) IIRC. Its system specific.
Technically, PID is also signed since
  pid_t f;
  f = fork();
can return -1 on error.
Personally I'd go with a signed 32bit integer, since that is what
the GNU C library uses for pid_t. And its really the C library's
definition, rather than the kernel's, which counts.
Any kernel in it's right mind is going to avoid negative PIDs as
that isn't the user's expectation; I note the Amos' message saying
that the Linux kernel currently allows PID values 1 to 4,194,303.
--
 Glen Turner Tel: (08) 8303 3936 or +61 8 8303 3936
 Australia's Academic & Research Network  www.aarnet.edu.au
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[amos]

On my 2.6.7 kernel (no tweaks to this value):
> cat /proc/sys/kernel/pid_max
32768
Since I'd expect databases to align such stuff to 32 bit
anyway, I think a 2^32 is a safe bet. And using this virtual
file will give you the correct answer for the current system.
Ref: proc(5):
/proc/sys/kernel/pid_max
This  file (new in Linux 2.5) specifies the value at which PIDs
wrap around (i.e., the value in this file is one  greater  than
the  maximum  PID).   The  default  value for this file, 32768,
results in the same range of PIDs as on earlier  kernels.   The
value  in  this  file  can  be  set  to  any  value  up to 2^22
(PID_MAX_LIMIT, approximately 4 million).
Cheers,
--Amos
Michael Kraus wrote:
G'day...
 
I'm developing a database application that  uses the inserting processes
pid. Problem is that I'm wondering how big this pid should be?
 
(OK, if I was a true guru, I'd know this magic number off the top of my
head...)
 
Is the maximum value of a pid on a UN*X system 65,535? (At least
generally speaking - I think on FreeBSD for example its at 30,000 by
default.)
 
I've been looking at a lot of web pages, but mostly they seem to deal
with the maximum number of concurrent processes. (Which is rather
defined by memory and sits at about 33,000 for modified Linux with an
average process -excluding the OS-being 30K, but only 4,090 unmodified.)
 
Regards,

 

Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net http://www.wildtechnology.net/> 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Robert Collins]

On Fri, 2004-10-15 at 15:33 +1000, Michael Kraus wrote:
> G'day...
>  
> I'm developing a database application that  uses the inserting processes
> pid. Problem is that I'm wondering how big this pid should be?

sizeof(pid_t) IIRC. Its system specific.

FWIW, I just wouldn't do that.

Rob
 

-- 
GPG key available at: .


signature.asc
Description: This is a digitally signed message part
-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Mike K]

G'day...

Thanks to Howard and Dave for their answers...! Greatly(!) appreciated.


On Fri, 2004-10-15 at 15:53, Dave Airlie wrote:
> #define PID_MAX_LIMIT (4*1024*1024)
> 
> so u32 is what you want to be safe..

Yep, and a u24 is just as safe I reckon... :)  (I.e. MEDIUMINT in
MySQL.)

Thanks heaps guys...

-Mike ... (now, to examine methods of Perl/Web Authentication
methods...)

-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Dave Airlie]

>
> I'm developing a database application that  uses the inserting processes
> pid. Problem is that I'm wondering how big this pid should be?

/*
 * This controls the default maximum pid allocated to a process
 */
#define PID_MAX_DEFAULT 0x8000

/*
 * A maximum of 4 million PIDs should be enough for a while:
 */
#define PID_MAX_LIMIT (4*1024*1024)

so u32 is what you want to be safe..

Dave.

-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

Re: [SLUG] Maximum process ID

[Howard Lowndes]

On Fri, 2004-10-15 at 15:33, Michael Kraus wrote:
> G'day...
>  
> I'm developing a database application that  uses the inserting processes
> pid. Problem is that I'm wondering how big this pid should be?
>  
> (OK, if I was a true guru, I'd know this magic number off the top of my
> head...)
>  
> Is the maximum value of a pid on a UN*X system 65,535? (At least
> generally speaking - I think on FreeBSD for example its at 30,000 by
> default.)

The max is 64k, but some *nices rotate it below 32k

>  
> I've been looking at a lot of web pages, but mostly they seem to deal
> with the maximum number of concurrent processes. (Which is rather
> defined by memory and sits at about 33,000 for modified Linux with an
> average process -excluding the OS-being 30K, but only 4,090 unmodified.)
>  
> Regards,
> 
>  
> 
> Michael S. E. Kraus
> Software Developer/Technical Support Specialist
> Wild Technology Pty Ltd
> [EMAIL PROTECTED]
> Direct Line 02-8306-0007 
> 
> 
> ABN 98 091 470 692
> Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
> Telephone 1300-13-9453 |  Facsimile 1300-88-9453
> http://www.wildtechnology.net http://www.wildtechnology.net/> 
> 
> The information contained in this email message and any attachments may
> be confidential information and may also be the subject of client legal
> - legal professional privilege. If you are not the intended recipient,
> any use, interference with, disclosure or copying of this material is
> unauthorised and prohibited.   This email and any attachments are also
> subject to copyright.  No part of them may be reproduced, adapted or
> transmitted without the written permission of the copyright owner.  If
> you have received this email in error, please immediately advise the
> sender by return email and delete the message from your system.
-- 
Howard.
LANNet Computing Associates;
Your Linux people 
--
"When you just want a system that works, you choose Linux;
when you want a system that just works, you choose Microsoft."
--
"Flatter government, not fatter government;
Get rid of the Australian states."


-- 
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html

[SLUG] Maximum process ID

[Michael Kraus]

G'day...
 
I'm developing a database application that  uses the inserting processes
pid. Problem is that I'm wondering how big this pid should be?
 
(OK, if I was a true guru, I'd know this magic number off the top of my
head...)
 
Is the maximum value of a pid on a UN*X system 65,535? (At least
generally speaking - I think on FreeBSD for example its at 30,000 by
default.)
 
I've been looking at a lot of web pages, but mostly they seem to deal
with the maximum number of concurrent processes. (Which is rather
defined by memory and sits at about 33,000 for modified Linux with an
average process -excluding the OS-being 30K, but only 4,090 unmodified.)
 
Regards,

 

Michael S. E. Kraus
Software Developer/Technical Support Specialist
Wild Technology Pty Ltd
[EMAIL PROTECTED]
Direct Line 02-8306-0007 


ABN 98 091 470 692
Level 4 Tiara, 306/9 Crystal Street, Waterloo NSW 2017, Australia
Telephone 1300-13-9453 |  Facsimile 1300-88-9453
http://www.wildtechnology.net http://www.wildtechnology.net/> 

The information contained in this email message and any attachments may
be confidential information and may also be the subject of client legal
- legal professional privilege. If you are not the intended recipient,
any use, interference with, disclosure or copying of this material is
unauthorised and prohibited.   This email and any attachments are also
subject to copyright.  No part of them may be reproduced, adapted or
transmitted without the written permission of the copyright owner.  If
you have received this email in error, please immediately advise the
sender by return email and delete the message from your system.

--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html