Re: [PATCH] mod_unique_id: use ap_random_insecure_bytes() to get unique ID

2013-07-07 Thread Daniel Lescohier
Actually, for stamp_fraction, one should choose either:

r->request_time & 0x /* microseconds modulus 65536 */

apr_uint16_t ui16;
htons(ui16 = apr_time_usec(r->request_time) >> 4) /* fraction of a second
with denominator 62500 */



On Sun, Jul 7, 2013 at 7:32 AM, Daniel Lescohier
wrote:

> Another option:
>
>
> typedef struct {
> apr_uint32_t stamp;
> apr_uint32_t counter;
> apr_uint16_t stamp_fraction;
> char root[ROOT_SIZE];
> } unique_id_rec;
>
> where ROOT_SIZE=8, and stamp_fraction is set on every request to
> htons(r->request_time & 0x).
>
> The child will be initialized with 12 bytes of random data (root +
> counter).  If one is not as confident of having a good random source, then
> it's better to also include the fractional second: there will be few
> requests happening in the same 1/65,536th of a second of the wall clock; if
> there are multiple requests in that fraction of a second, if the multiple
> requests are in the same child process, then the atomic increment
> (apr_atomic_inc32) of the counter alone will guarantee different ids;
> otherwise, for those few requests in that fraction of a second in multiple
> processes (on the same or different servers), the randomness initialized by
> the child in root+counter will be used to make different ids.
>
>
>
> On Sat, Jul 6, 2013 at 9:38 AM, Daniel Lescohier <
> daniel.lescoh...@cbsi.com> wrote:
>
>> Ah, I missed that.  I see it's in the doxygen docs for the random
>> module.  However, the sources aren't under random/, they're under misc/.  I
>> was switching between the doxygen docs and looking at the sources; maybe
>> when I was looking for it, I missed it because the sources were under misc/.
>>
>> For the standard module: you can keep the unique id size the same, but
>> still have a root of 10 bytes, by getting rid of the thread_index:
>> thread_index is very wasteful, using 32 bits, when only a few bits are used.
>>
>> Change:
>>
>> typedef struct {
>> unsigned int stamp;
>> unsigned int in_addr;
>> unsigned int pid;
>> unsigned short counter;
>> unsigned int thread_index;
>> } unique_id_rec;
>>
>> to:
>>
>> typedef struct {
>> apr_uint32_t stamp;
>> apr_uint32_t counter;
>> char root[ROOT_SIZE];
>> } unique_id_rec;
>>
>> Have the two ints first for alignment purposes, so there is no padding in
>> the struct.
>>
>> With the counter field using apr_uint32_t, you can use apr_atomic_inc32()
>> to do the increments.  There is less need of thread_index if you increment
>> the counter atomically.
>>
>> The initialization of the counter with random data also gives you more
>> process randomness than the 10 bytes of root.
>>
>>
>>
>> On Fri, Jul 5, 2013 at 8:04 PM, Stefan Fritsch  wrote:
>>
>>> On Wednesday 26 June 2013, Daniel Lescohier wrote:
>>> > When I looked into the ap random functions, I didn't like the
>>> > implementation, because I didn't see anywhere in the httpd codebase
>>> > that entropy is periodically added to the entropy pool.  After
>>> > reading the details of how the Linux entropy pool works
>>> > (https://lwn
>>> > .net/Articles/525204/), I decided to use /dev/urandom instead,
>>> > since Linux is periodically adding entropy to it.  This code is
>>> > not portable, but this was for a private Apache module that is
>>> > only used on Linux.
>>> >
>>> > To preserve entropy on the web server machine, I also only generate
>>> > a random number once per apache child, then increment an uint32
>>> > portion of it for each unique id call.  I also have seconds and
>>> > microseconds, so that's why I think it's OK to do increments from
>>> > the random base, instead of generating a new random id on each
>>> > request.
>>>
>>> The "insecure" in ap_random_insecure_bytes is there for a reason. But
>>> if you only use it once per process, anyway, it should be sufficient.
>>> The fact that several consumers (especially with multi-threaded mpms)
>>> pull from the same pool in undefined order adds some entropy, too.
>>>
>>> FWIW, there is apr_generate_random_bytes() which can do the reading of
>>> /dev/urandom for you.
>>>
>>
>>
>


Re: [PATCH] mod_unique_id: use ap_random_insecure_bytes() to get unique ID

2013-07-07 Thread Daniel Lescohier
Another option:

typedef struct {
apr_uint32_t stamp;
apr_uint32_t counter;
apr_uint16_t stamp_fraction;
char root[ROOT_SIZE];
} unique_id_rec;

where ROOT_SIZE=8, and stamp_fraction is set on every request to
htons(r->request_time & 0x).

The child will be initialized with 12 bytes of random data (root +
counter).  If one is not as confident of having a good random source, then
it's better to also include the fractional second: there will be few
requests happening in the same 1/65,536th of a second of the wall clock; if
there are multiple requests in that fraction of a second, if the multiple
requests are in the same child process, then the atomic increment
(apr_atomic_inc32) of the counter alone will guarantee different ids;
otherwise, for those few requests in that fraction of a second in multiple
processes (on the same or different servers), the randomness initialized by
the child in root+counter will be used to make different ids.



On Sat, Jul 6, 2013 at 9:38 AM, Daniel Lescohier
wrote:

> Ah, I missed that.  I see it's in the doxygen docs for the random module.
> However, the sources aren't under random/, they're under misc/.  I was
> switching between the doxygen docs and looking at the sources; maybe when I
> was looking for it, I missed it because the sources were under misc/.
>
> For the standard module: you can keep the unique id size the same, but
> still have a root of 10 bytes, by getting rid of the thread_index:
> thread_index is very wasteful, using 32 bits, when only a few bits are used.
>
> Change:
>
> typedef struct {
> unsigned int stamp;
> unsigned int in_addr;
> unsigned int pid;
> unsigned short counter;
> unsigned int thread_index;
> } unique_id_rec;
>
> to:
>
> typedef struct {
> apr_uint32_t stamp;
> apr_uint32_t counter;
> char root[ROOT_SIZE];
> } unique_id_rec;
>
> Have the two ints first for alignment purposes, so there is no padding in
> the struct.
>
> With the counter field using apr_uint32_t, you can use apr_atomic_inc32()
> to do the increments.  There is less need of thread_index if you increment
> the counter atomically.
>
> The initialization of the counter with random data also gives you more
> process randomness than the 10 bytes of root.
>
>
>
> On Fri, Jul 5, 2013 at 8:04 PM, Stefan Fritsch  wrote:
>
>> On Wednesday 26 June 2013, Daniel Lescohier wrote:
>> > When I looked into the ap random functions, I didn't like the
>> > implementation, because I didn't see anywhere in the httpd codebase
>> > that entropy is periodically added to the entropy pool.  After
>> > reading the details of how the Linux entropy pool works
>> > (https://lwn
>> > .net/Articles/525204/), I decided to use /dev/urandom instead,
>> > since Linux is periodically adding entropy to it.  This code is
>> > not portable, but this was for a private Apache module that is
>> > only used on Linux.
>> >
>> > To preserve entropy on the web server machine, I also only generate
>> > a random number once per apache child, then increment an uint32
>> > portion of it for each unique id call.  I also have seconds and
>> > microseconds, so that's why I think it's OK to do increments from
>> > the random base, instead of generating a new random id on each
>> > request.
>>
>> The "insecure" in ap_random_insecure_bytes is there for a reason. But
>> if you only use it once per process, anyway, it should be sufficient.
>> The fact that several consumers (especially with multi-threaded mpms)
>> pull from the same pool in undefined order adds some entropy, too.
>>
>> FWIW, there is apr_generate_random_bytes() which can do the reading of
>> /dev/urandom for you.
>>
>
>