Re: [OMPI devel] [EXTERNAL] glibc malloc hooks going away
ubuntu12 fedora16 the hooks are there, but marked as "deprecated" On Tue, Jun 11, 2013 at 4:05 AM, Jeff Squyres (jsquyres) wrote: > On Jun 10, 2013, at 10:29 AM, "Barrett, Brian W" > wrote: > > > At least they've finally come to that conclusion. I look forward to not > > shipping a memory allocator with our communication library ;). > > > +1 on that. > > That being said, that release note was for glibc 2.14. I just downloaded > and built 2.17; it looks like a) the hooks are still there, and b) they're > still installed by default. > > Mellanox: did you find a distro that disables the glibc hooks by default? > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel >
Re: [OMPI devel] [EXTERNAL] glibc malloc hooks going away
On Jun 11, 2013, at 9:37 AM, Mike Dubman wrote: > ubuntu12 > fedora16 > > the hooks are there, but marked as "deprecated" Does that mean they come up with a compiler warning? > > On Tue, Jun 11, 2013 at 4:05 AM, Jeff Squyres (jsquyres) > wrote: > On Jun 10, 2013, at 10:29 AM, "Barrett, Brian W" wrote: > > > At least they've finally come to that conclusion. I look forward to not > > shipping a memory allocator with our communication library ;). > > > +1 on that. > > That being said, that release note was for glibc 2.14. I just downloaded and > built 2.17; it looks like a) the hooks are still there, and b) they're still > installed by default. > > Mellanox: did you find a distro that disables the glibc hooks by default? > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] [EXTERNAL] glibc malloc hooks going away
yes On Tue, Jun 11, 2013 at 5:10 PM, Jeff Squyres (jsquyres) wrote: > On Jun 11, 2013, at 9:37 AM, Mike Dubman wrote: > > > ubuntu12 > > fedora16 > > > > the hooks are there, but marked as "deprecated" > > Does that mean they come up with a compiler warning? > > > > > > On Tue, Jun 11, 2013 at 4:05 AM, Jeff Squyres (jsquyres) < > jsquy...@cisco.com> wrote: > > On Jun 10, 2013, at 10:29 AM, "Barrett, Brian W" > wrote: > > > > > At least they've finally come to that conclusion. I look forward to > not > > > shipping a memory allocator with our communication library ;). > > > > > > +1 on that. > > > > That being said, that release note was for glibc 2.14. I just > downloaded and built 2.17; it looks like a) the hooks are still there, and > b) they're still installed by default. > > > > Mellanox: did you find a distro that disables the glibc hooks by default? > > > > -- > > Jeff Squyres > > jsquy...@cisco.com > > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > > > > ___ > > devel mailing list > > de...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/devel > > > > ___ > > devel mailing list > > de...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/devel > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel >
Re: [OMPI devel] [EXTERNAL] glibc malloc hooks going away
On Jun 10, 2013, at 9:05 PM, Jeff Squyres wrote: > That being said, that release note was for glibc 2.14. I just downloaded and > built 2.17; it looks like a) the hooks are still there, and b) they're still > installed by default. Ah -- I see what happened. glibc marked the hook interfaces with compiler attributes for "deprecated" starting with glibc 2.14. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] RFC: Add static initializer for opal_mutex_t
On Mon, Jun 10, 2013 at 06:53:36PM +0200, George Bosilca wrote: > > On Jun 10, 2013, at 17:18 , Nathan Hjelm wrote: > > > On Sat, Jun 08, 2013 at 12:28:02PM +0200, George Bosilca wrote: > >> All Windows objects that are managed as HANDLES can easily be modified to > >> have static initializer. A clean solution is attached to the question at > >> stackoverflow: > >> http://stackoverflow.com/questions/3555859/is-it-possible-to-do-static-initialization-of-mutexes-in-windows > > > > Not the cleanest solution (and I don't know how handles work) so I held off > > on proposing adding a static initializer until the windows code was gone. > > Nothing really fancy, a HANDLE is basically an untyped location storage (a > void*). > > >> That being said I think having a static initializer for a synchronization > >> object is a dangerous thing. It has many subtleties and too many hidden > >> limitations. As an example they can only be used on the declaration of the > >> object, and can't be safely used for locally static object (they must be > >> global). > > > > I have never seen any indication that a statically initialized mutex is not > > safe for static objecs. The man page for thread_mutex_init uses the static > > initializer on a static mutex: http://linux.die.net/man/3/pthread_mutex_init > > It is thread safe for global static objects, but might not be thread safe for > local static objects. > > >> What are the instances in the Open MPI code where such a statically > >> defined mutex need to be used before it has a chance of being correctly > >> initialized? > > > > MPI_T_thread_init may be called from any thread (or multiple threads at the > > same time). The current code uses atomics to protect the initialization of > > the mutex. I would prefer to declare the mpit lock like: > > > > opal_mutex_t mpit_big_lock = OPAL_MUTEX_STATIC_INIT; > > > > and remove the atomics. It would be much cleaner and should work fine on > > all currently supported platforms. > > OK, almost a corner-case. > > > how does mutex static initializer works > > A more detailed explanation in the "Static Initializers for Mutexes and > Condition Variables" part of the > http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html Interesting. We could add a caveat to the definition describing where static initialization might not be optimal. Either that or we could implement a opal_once to do the initialization in this case. I would have to look into the solaris thread case to see if a once function is possible there. -Nathan
Re: [OMPI devel] RFC: Add static initializer for opal_mutex_t
On Jun 11, 2013, at 9:09 AM, Nathan Hjelm wrote: > On Mon, Jun 10, 2013 at 06:53:36PM +0200, George Bosilca wrote: >> >> On Jun 10, 2013, at 17:18 , Nathan Hjelm wrote: >> >>> On Sat, Jun 08, 2013 at 12:28:02PM +0200, George Bosilca wrote: All Windows objects that are managed as HANDLES can easily be modified to have static initializer. A clean solution is attached to the question at stackoverflow: http://stackoverflow.com/questions/3555859/is-it-possible-to-do-static-initialization-of-mutexes-in-windows >>> >>> Not the cleanest solution (and I don't know how handles work) so I held off >>> on proposing adding a static initializer until the windows code was gone. >> >> Nothing really fancy, a HANDLE is basically an untyped location storage (a >> void*). >> That being said I think having a static initializer for a synchronization object is a dangerous thing. It has many subtleties and too many hidden limitations. As an example they can only be used on the declaration of the object, and can't be safely used for locally static object (they must be global). >>> >>> I have never seen any indication that a statically initialized mutex is not >>> safe for static objecs. The man page for thread_mutex_init uses the static >>> initializer on a static mutex: http://linux.die.net/man/3/pthread_mutex_init >> >> It is thread safe for global static objects, but might not be thread safe >> for local static objects. >> What are the instances in the Open MPI code where such a statically defined mutex need to be used before it has a chance of being correctly initialized? >>> >>> MPI_T_thread_init may be called from any thread (or multiple threads at the >>> same time). The current code uses atomics to protect the initialization of >>> the mutex. I would prefer to declare the mpit lock like: >>> >>> opal_mutex_t mpit_big_lock = OPAL_MUTEX_STATIC_INIT; >>> >>> and remove the atomics. It would be much cleaner and should work fine on >>> all currently supported platforms. >> >> OK, almost a corner-case. >> >>> how does mutex static initializer works >> >> A more detailed explanation in the "Static Initializers for Mutexes and >> Condition Variables" part of the >> http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html > > Interesting. We could add a caveat to the definition describing where static > initialization might not be optimal. Either that or we could implement a > opal_once to do the initialization in this case. I would have to look into > the solaris thread case to see if a once function is possible there. We don't support solaris threads any more - haven't for quite some time. > > -Nathan > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: Add static initializer for opal_mutex_t
On Tue, Jun 11, 2013 at 09:13:01AM -0700, Ralph Castain wrote: > > On Jun 11, 2013, at 9:09 AM, Nathan Hjelm wrote: > > > On Mon, Jun 10, 2013 at 06:53:36PM +0200, George Bosilca wrote: > >> > >> On Jun 10, 2013, at 17:18 , Nathan Hjelm wrote: > >> > >>> On Sat, Jun 08, 2013 at 12:28:02PM +0200, George Bosilca wrote: > All Windows objects that are managed as HANDLES can easily be modified > to have static initializer. A clean solution is attached to the question > at stackoverflow: > http://stackoverflow.com/questions/3555859/is-it-possible-to-do-static-initialization-of-mutexes-in-windows > >>> > >>> Not the cleanest solution (and I don't know how handles work) so I held > >>> off on proposing adding a static initializer until the windows code was > >>> gone. > >> > >> Nothing really fancy, a HANDLE is basically an untyped location storage (a > >> void*). > >> > That being said I think having a static initializer for a > synchronization object is a dangerous thing. It has many subtleties and > too many hidden limitations. As an example they can only be used on the > declaration of the object, and can't be safely used for locally static > object (they must be global). > >>> > >>> I have never seen any indication that a statically initialized mutex is > >>> not safe for static objecs. The man page for thread_mutex_init uses the > >>> static initializer on a static mutex: > >>> http://linux.die.net/man/3/pthread_mutex_init > >> > >> It is thread safe for global static objects, but might not be thread safe > >> for local static objects. > >> > What are the instances in the Open MPI code where such a statically > defined mutex need to be used before it has a chance of being correctly > initialized? > >>> > >>> MPI_T_thread_init may be called from any thread (or multiple threads at > >>> the same time). The current code uses atomics to protect the > >>> initialization of the mutex. I would prefer to declare the mpit lock like: > >>> > >>> opal_mutex_t mpit_big_lock = OPAL_MUTEX_STATIC_INIT; > >>> > >>> and remove the atomics. It would be much cleaner and should work fine on > >>> all currently supported platforms. > >> > >> OK, almost a corner-case. > >> > >>> how does mutex static initializer works > >> > >> A more detailed explanation in the "Static Initializers for Mutexes and > >> Condition Variables" part of the > >> http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html > > > > Interesting. We could add a caveat to the definition describing where > > static initialization might not be optimal. Either that or we could > > implement a opal_once to do the initialization in this case. I would have > > to look into the solaris thread case to see if a once function is possible > > there. > > We don't support solaris threads any more - haven't for quite some time. If that is the case would there be any objection to the removal of the solaris thread code from opal_mutex? -Nathan
Re: [OMPI devel] RFC: Add static initializer for opal_mutex_t
On Jun 11, 2013, at 9:16 AM, Nathan Hjelm wrote: > On Tue, Jun 11, 2013 at 09:13:01AM -0700, Ralph Castain wrote: >> >> On Jun 11, 2013, at 9:09 AM, Nathan Hjelm wrote: >> >>> On Mon, Jun 10, 2013 at 06:53:36PM +0200, George Bosilca wrote: On Jun 10, 2013, at 17:18 , Nathan Hjelm wrote: > On Sat, Jun 08, 2013 at 12:28:02PM +0200, George Bosilca wrote: >> All Windows objects that are managed as HANDLES can easily be modified >> to have static initializer. A clean solution is attached to the question >> at stackoverflow: >> http://stackoverflow.com/questions/3555859/is-it-possible-to-do-static-initialization-of-mutexes-in-windows > > Not the cleanest solution (and I don't know how handles work) so I held > off on proposing adding a static initializer until the windows code was > gone. Nothing really fancy, a HANDLE is basically an untyped location storage (a void*). >> That being said I think having a static initializer for a >> synchronization object is a dangerous thing. It has many subtleties and >> too many hidden limitations. As an example they can only be used on the >> declaration of the object, and can't be safely used for locally static >> object (they must be global). > > I have never seen any indication that a statically initialized mutex is > not safe for static objecs. The man page for thread_mutex_init uses the > static initializer on a static mutex: > http://linux.die.net/man/3/pthread_mutex_init It is thread safe for global static objects, but might not be thread safe for local static objects. >> What are the instances in the Open MPI code where such a statically >> defined mutex need to be used before it has a chance of being correctly >> initialized? > > MPI_T_thread_init may be called from any thread (or multiple threads at > the same time). The current code uses atomics to protect the > initialization of the mutex. I would prefer to declare the mpit lock like: > > opal_mutex_t mpit_big_lock = OPAL_MUTEX_STATIC_INIT; > > and remove the atomics. It would be much cleaner and should work fine on > all currently supported platforms. OK, almost a corner-case. > how does mutex static initializer works A more detailed explanation in the "Static Initializers for Mutexes and Condition Variables" part of the http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html >>> >>> Interesting. We could add a caveat to the definition describing where >>> static initialization might not be optimal. Either that or we could >>> implement a opal_once to do the initialization in this case. I would have >>> to look into the solaris thread case to see if a once function is possible >>> there. >> >> We don't support solaris threads any more - haven't for quite some time. > > If that is the case would there be any objection to the removal of the > solaris thread code from opal_mutex? Don't see why - even when Sun was still active, they agreed to standardize on pthreads as their compilers progressed to that point > > -Nathan > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
[OMPI devel] RFC: improve the hash function used by opal_hash_table_t
What: Implement a better hash function in opal_hash_table_t. The function is a
simple one-at-a-time Jenkin's hash (see
http://en.wikipedia.org/wiki/Jenkins_hash_function) and has good collision
rates and isn't overly complex or slow.
Why: I am preparing an update to the MCA variable system (adding performance
variables) which will include a hash-based lookup function (in preperation for
the inevitable MPI_T_cvar/pvar/category lookup functions-- MPI 3.0 errata). The
current hash function is not very good so now seems like a good time to update
it.
When: Will push this to trunk on Thursday if there are no objections.
Patch below
Index: opal/class/opal_hash_table.c
===
--- opal/class/opal_hash_table.c(revision 28609)
+++ opal/class/opal_hash_table.c(working copy)
@@ -356,14 +356,20 @@
static inline uint32_t opal_hash_value(size_t mask, const void *key,
size_t keysize)
{
-size_t h, i;
-const unsigned char *p;
-
-h = 0;
-p = (const unsigned char *)key;
-for (i = 0; i < keysize; i++, p++)
-h = HASH_MULTIPLIER*h + *p;
-return (uint32_t)(h & mask);
+const unsigned char *p = (const unsigned char *) key;
+uint32_t h = 0, i;
+
+for (i = 0 ; i < keysize ; ++i, ++p) {
+h += *p;
+h += h << 10;
+h ^= h >> 6;
+}
+
+h += h << 3;
+h ^= h >> 11;
+h += h << 15;
+
+return h & (uint32_t) mask;
}
int opal_hash_table_get_value_ptr(opal_hash_table_t* ht, const void* key,
-Nathan
Re: [OMPI devel] RFC: improve the hash function used byopal_hash_table_t
Hello,
one of the best hash functions I know is crc32. It is not very expensive,
especially with modern CPUs where it is implemented as the CPU instruction.
Hope this helps.
Kind regards,
Alex Granovsky
-Original Message-
From: Nathan Hjelm
Sent: Wednesday, June 12, 2013 1:32 AM
To: Open MPI Developers
Subject: [OMPI devel] RFC: improve the hash function used
byopal_hash_table_t
What: Implement a better hash function in opal_hash_table_t. The function is
a simple one-at-a-time Jenkin's hash (see
http://en.wikipedia.org/wiki/Jenkins_hash_function) and has good collision
rates and isn't overly complex or slow.
Why: I am preparing an update to the MCA variable system (adding performance
variables) which will include a hash-based lookup function (in preperation
for the inevitable MPI_T_cvar/pvar/category lookup functions-- MPI 3.0
errata). The current hash function is not very good so now seems like a good
time to update it.
When: Will push this to trunk on Thursday if there are no objections.
Patch below
Index: opal/class/opal_hash_table.c
===
--- opal/class/opal_hash_table.c (revision 28609)
+++ opal/class/opal_hash_table.c (working copy)
@@ -356,14 +356,20 @@
static inline uint32_t opal_hash_value(size_t mask, const void *key,
size_t keysize)
{
-size_t h, i;
-const unsigned char *p;
-
-h = 0;
-p = (const unsigned char *)key;
-for (i = 0; i < keysize; i++, p++)
-h = HASH_MULTIPLIER*h + *p;
-return (uint32_t)(h & mask);
+const unsigned char *p = (const unsigned char *) key;
+uint32_t h = 0, i;
+
+for (i = 0 ; i < keysize ; ++i, ++p) {
+h += *p;
+h += h << 10;
+h ^= h >> 6;
+}
+
+h += h << 3;
+h ^= h >> 11;
+h += h << 15;
+
+return h & (uint32_t) mask;
}
int opal_hash_table_get_value_ptr(opal_hash_table_t* ht, const void* key,
-Nathan
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: improve the hash function used by opal_hash_table_t
The one-at-the-time version computes on chars, if the performance of the hash
function is a critical element in the equation then you will be better off
avoiding its usage. I would suggest going with Murmur
(http://en.wikipedia.org/wiki/MurmurHash) instead, which is faster and perform
well in random distribution. Another interesting features is that there are
specialized derivative for strings based key, a feature that might prove
helpful with the MCA parameters and the MPI_T stuff.
George.
On Jun 11, 2013, at 23:32 , Nathan Hjelm wrote:
> What: Implement a better hash function in opal_hash_table_t. The function is
> a simple one-at-a-time Jenkin's hash (see
> http://en.wikipedia.org/wiki/Jenkins_hash_function) and has good collision
> rates and isn't overly complex or slow.
>
> Why: I am preparing an update to the MCA variable system (adding performance
> variables) which will include a hash-based lookup function (in preperation
> for the inevitable MPI_T_cvar/pvar/category lookup functions-- MPI 3.0
> errata). The current hash function is not very good so now seems like a good
> time to update it.
>
> When: Will push this to trunk on Thursday if there are no objections.
>
> Patch below
>
> Index: opal/class/opal_hash_table.c
> ===
> --- opal/class/opal_hash_table.c (revision 28609)
> +++ opal/class/opal_hash_table.c (working copy)
> @@ -356,14 +356,20 @@
> static inline uint32_t opal_hash_value(size_t mask, const void *key,
>size_t keysize)
> {
> -size_t h, i;
> -const unsigned char *p;
> -
> -h = 0;
> -p = (const unsigned char *)key;
> -for (i = 0; i < keysize; i++, p++)
> -h = HASH_MULTIPLIER*h + *p;
> -return (uint32_t)(h & mask);
> +const unsigned char *p = (const unsigned char *) key;
> +uint32_t h = 0, i;
> +
> +for (i = 0 ; i < keysize ; ++i, ++p) {
> +h += *p;
> +h += h << 10;
> +h ^= h >> 6;
> +}
> +
> +h += h << 3;
> +h ^= h >> 11;
> +h += h << 15;
> +
> +return h & (uint32_t) mask;
> }
>
> int opal_hash_table_get_value_ptr(opal_hash_table_t* ht, const void* key,
>
>
>
>
> -Nathan
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: improve the hash function used by opal_hash_table_t
I think as long as we are updating this we might as well add a performant hash
function. I see Murmur2 is 32-bit and has a public domain implementation:
https://code.google.com/p/smhasher/source/browse/trunk/MurmurHash2.cpp
Since Murmur2 hashes in 4-byte blocks I expect it probably would perform better
than either a nieve crc32 or Jenkins. Though a hardware accelerated crc32 (if
available) would probably work great as well. Both are good suggestions that I
will look into. I will also look into the string variant of Murmur though that
would require another set of opal_hash_table functions for string keys (I was
using the _ptr variety).
-Nathan
On Wed, Jun 12, 2013 at 12:10:42AM +0200, George Bosilca wrote:
> The one-at-the-time version computes on chars, if the performance of the hash
> function is a critical element in the equation then you will be better off
> avoiding its usage. I would suggest going with Murmur
> (http://en.wikipedia.org/wiki/MurmurHash) instead, which is faster and
> perform well in random distribution. Another interesting features is that
> there are specialized derivative for strings based key, a feature that might
> prove helpful with the MCA parameters and the MPI_T stuff.
>
> George.
>
>
> On Jun 11, 2013, at 23:32 , Nathan Hjelm wrote:
>
> > What: Implement a better hash function in opal_hash_table_t. The function
> > is a simple one-at-a-time Jenkin's hash (see
> > http://en.wikipedia.org/wiki/Jenkins_hash_function) and has good collision
> > rates and isn't overly complex or slow.
> >
> > Why: I am preparing an update to the MCA variable system (adding
> > performance variables) which will include a hash-based lookup function (in
> > preperation for the inevitable MPI_T_cvar/pvar/category lookup functions--
> > MPI 3.0 errata). The current hash function is not very good so now seems
> > like a good time to update it.
> >
> > When: Will push this to trunk on Thursday if there are no objections.
> >
> > Patch below
> >
> > Index: opal/class/opal_hash_table.c
> > ===
> > --- opal/class/opal_hash_table.c(revision 28609)
> > +++ opal/class/opal_hash_table.c(working copy)
> > @@ -356,14 +356,20 @@
> > static inline uint32_t opal_hash_value(size_t mask, const void *key,
> >size_t keysize)
> > {
> > -size_t h, i;
> > -const unsigned char *p;
> > -
> > -h = 0;
> > -p = (const unsigned char *)key;
> > -for (i = 0; i < keysize; i++, p++)
> > -h = HASH_MULTIPLIER*h + *p;
> > -return (uint32_t)(h & mask);
> > +const unsigned char *p = (const unsigned char *) key;
> > +uint32_t h = 0, i;
> > +
> > +for (i = 0 ; i < keysize ; ++i, ++p) {
> > +h += *p;
> > +h += h << 10;
> > +h ^= h >> 6;
> > +}
> > +
> > +h += h << 3;
> > +h ^= h >> 11;
> > +h += h << 15;
> > +
> > +return h & (uint32_t) mask;
> > }
> >
> > int opal_hash_table_get_value_ptr(opal_hash_table_t* ht, const void* key,
> >
> >
> >
> >
> > -Nathan
> > ___
> > devel mailing list
> > de...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] RFC: improve the hash function used by opal_hash_table_t
On Jun 12, 2013, at 00:22 , Nathan Hjelm wrote: > Though a hardware accelerated crc32 (if available) would probably work great > as well. http://google-opensource.blogspot.fr/2011/04/introducing-cityhash.html with code available under MIT @ https://code.google.com/p/cityhash/ George.
