Re: ubytes to ulong problem
On 12/21/2013 07:57 PM, Ali Çehreli wrote: On 12/21/2013 05:44 PM, Charles Hixson wrote: On 12/21/2013 03:52 PM, Ali Çehreli wrote: On 12/21/2013 03:13 PM, John Colvin wrote: Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes. No, it is not guaranteed to work unless the alignment is right. How about this, then: :) import std.array; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = block.front; block.popFront(); foreach (ub; block) { value = 8; value |= ub; } return value; } Ali Nice, but the block is longer than 8 bytes, so I should use a for (i = n; i n + 8; i++) rather than a foreach, and index off of i. I clearly need to redo the documentation a bit (even though it's form me of a few months from now). It needs to say something like Convert a 8 byte slice from a ubyte array starting at index n into a ulong. n should always be required to be specified, so I don't want a default value. (0 was used as a test case, because I'd made a really stupid mistake and used ^ for exponentiation, and then couldn't see what was going on, so I was simplifying everything...and I still couldn't see it. Actually the array starts with a ushort, which specifies the number of ulongs to follow before a bunch of bytes that are unintelligible data to the class that's using this function. (OTOH, it seems like something generally useful, so I'll probably put it in a utils.d file, with some other generally useful routines.) OTOH, if I'm going to consider this to be a general utility function, then I really don't want to make assumptions about where things start, etc. Perhaps I should throw an exception (other than assertion error) if the index is bad or the array is to short for the given index. I need to think about that a bit more. The alternative is to use enforce rather than assertions...though as long as I'm the only user assertions suffice. (It's not going to be separately compiled.) -- Charles Hixson
Re: ubytes to ulong problem
On 12/22/2013 01:04 AM, Charles Hixson wrote: Nice, but the block is longer than 8 bytes, so I should use a for (i = n; i n + 8; i++) rather than a foreach, and index off of i. Makes sense. That reminded me of the Phobos function that does exactly what you want. Have you considered std.bitmanip.read? Ali
Re: ubytes to ulong problem
On Sunday, 22 December 2013 at 03:57:38 UTC, Ali Çehreli wrote: On 12/21/2013 05:44 PM, Charles Hixson wrote: On 12/21/2013 03:52 PM, Ali Çehreli wrote: On 12/21/2013 03:13 PM, John Colvin wrote: Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes. No, it is not guaranteed to work unless the alignment is right. It's just an unaligned load. If your target cpu architecture can't do unaligned loads then you're either using something very small or very old.
Re: ubytes to ulong problem
On 12/22/2013 02:22 AM, Ali Çehreli wrote: On 12/22/2013 01:04 AM, Charles Hixson wrote: Nice, but the block is longer than 8 bytes, so I should use a for (i = n; i n + 8; i++) rather than a foreach, and index off of i. Makes sense. That reminded me of the Phobos function that does exactly what you want. Have you considered std.bitmanip.read? Ali No, thanks. That's precisely what I was looking for. -- Charles Hixson
ubytes to ulong problem
I was planning to ask if there were a better way to do this, but instead I need to ask what's my mistake? For some reason, if called with an uninitialized ubyte array, and an index of 0, it returns a value of 8, even though all the values in the array are 0. The error has to be somewhere in the ret = statement, but I sure don't see it. /**Convert 8 consecutive bytes sliced from a ubyte[] into a ulong * @paramblockThe array from which to slice. * @paramnThe starting index within the block*/ ulongubytesToUlong(ubyte[] block, int n) {ulongret; assert (n = 0); assert (n + 8 = block.length); writefln (n = %s, n); writefln (block[0] = %s, cast(ulong)block[0]); writefln (block[1] = %s, cast(ulong)block[1]); writefln (block[2] = %s, cast(ulong)block[2]); writefln (block[3] = %s, cast(ulong)block[3]); writefln (block[4] = %s, cast(ulong)block[4]); writefln (block[5] = %s, cast(ulong)block[5]); writefln (block[6] = %s, cast(ulong)block[6]); writefln (block[7] = %s, cast(ulong)block[7]); ret=cast(ulong)block[n] * 2^21 +cast(ulong)block[n+1] * 2^18 +cast(ulong)block[n+2] * 2^15 +cast(ulong)block[n+3] * 2^12 +cast(ulong)block[n+4] * 2^9 +cast(ulong)block[n+5] * 2^6 +cast(ulong)block[n+6] * 2^3 +cast(ulong)block[n+7] * 2^0; writefln (ret = %s, ret); returnret; } -- Charles Hixson
Re: ubytes to ulong problem
On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote: On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson wrote: I was planning to ask if there were a better way to do this, but instead I need to ask what's my mistake? For some reason, if called with an uninitialized ubyte array, and an index of 0, it returns a value of 8, even though all the values in the array are 0. The error has to be somewhere in the ret = statement, but I sure don't see it. /**Convert 8 consecutive bytes sliced from a ubyte[] into a ulong * @paramblockThe array from which to slice. * @paramnThe starting index within the block*/ ulongubytesToUlong(ubyte[] block, int n) {ulongret; assert (n = 0); assert (n + 8 = block.length); writefln (n = %s, n); writefln (block[0] = %s, cast(ulong)block[0]); writefln (block[1] = %s, cast(ulong)block[1]); writefln (block[2] = %s, cast(ulong)block[2]); writefln (block[3] = %s, cast(ulong)block[3]); writefln (block[4] = %s, cast(ulong)block[4]); writefln (block[5] = %s, cast(ulong)block[5]); writefln (block[6] = %s, cast(ulong)block[6]); writefln (block[7] = %s, cast(ulong)block[7]); ret=cast(ulong)block[n] * 2^21 +cast(ulong)block[n+1] * 2^18 +cast(ulong)block[n+2] * 2^15 +cast(ulong)block[n+3] * 2^12 +cast(ulong)block[n+4] * 2^9 +cast(ulong)block[n+5] * 2^6 +cast(ulong)block[n+6] * 2^3 +cast(ulong)block[n+7] * 2^0; writefln (ret = %s, ret); returnret; } Use the exponentiation operator which is spelled: ^^
Re: ubytes to ulong problem
On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson wrote: I was planning to ask if there were a better way to do this, but instead I need to ask what's my mistake? For some reason, if called with an uninitialized ubyte array, and an index of 0, it returns a value of 8, even though all the values in the array are 0. The error has to be somewhere in the ret = statement, but I sure don't see it. /**Convert 8 consecutive bytes sliced from a ubyte[] into a ulong * @paramblockThe array from which to slice. * @paramnThe starting index within the block*/ ulongubytesToUlong(ubyte[] block, int n) {ulongret; assert (n = 0); assert (n + 8 = block.length); writefln (n = %s, n); writefln (block[0] = %s, cast(ulong)block[0]); writefln (block[1] = %s, cast(ulong)block[1]); writefln (block[2] = %s, cast(ulong)block[2]); writefln (block[3] = %s, cast(ulong)block[3]); writefln (block[4] = %s, cast(ulong)block[4]); writefln (block[5] = %s, cast(ulong)block[5]); writefln (block[6] = %s, cast(ulong)block[6]); writefln (block[7] = %s, cast(ulong)block[7]); ret=cast(ulong)block[n] * 2^21 +cast(ulong)block[n+1] * 2^18 +cast(ulong)block[n+2] * 2^15 +cast(ulong)block[n+3] * 2^12 +cast(ulong)block[n+4] * 2^9 +cast(ulong)block[n+5] * 2^6 +cast(ulong)block[n+6] * 2^3 +cast(ulong)block[n+7] * 2^0; writefln (ret = %s, ret); returnret; } As pointed out before, you're using '^' which is xor, instead of ^^. Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly: /**Convert 8 consecutive bytes sliced from a ubyte[] * to a ulong using bigendian byte ordering. * @paramblockThe array from which to slice. * @paramnThe starting index within the block, default 0*/ ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { return ((cast(ulong)block[n ]) 56) | ((cast(ulong)block[n+1]) 48) | ((cast(ulong)block[n+2]) 40) | ((cast(ulong)block[n+3]) 32) | ((cast(ulong)block[n+4]) 24) | ((cast(ulong)block[n+5]) 16) | ((cast(ulong)block[n+6]) 8) | (cast(ulong)block[n+7]); } unittest { ubyte[8] a = [0,0,0,0,0,0,0,0]; assert(a[].ubytesToUlong() == 0); a[7] = 3; a[6] = 1; assert(a[].ubytesToUlong() == 259); }
Re: ubytes to ulong problem
On 12/21/2013 02:31 PM, ponce wrote: On Saturday, 21 December 2013 at 22:29:59 UTC, ponce wrote: On Saturday, 21 December 2013 at 22:22:09 UTC, Charles Hixson wrote: I was planning to ask if there were a better way to do this, but instead I need to ask what's my mistake? For some reason, if called with an uninitialized ubyte array, and an index of 0, it returns a value of 8, even though all the values in the array are 0. The error has to be somewhere in the ret = statement, but I sure don't see it. /**Convert 8 consecutive bytes sliced from a ubyte[] into a ulong * @paramblockThe array from which to slice. * @paramnThe starting index within the block*/ ulongubytesToUlong(ubyte[] block, int n) {ulongret; assert (n = 0); assert (n + 8 = block.length); writefln (n = %s, n); writefln (block[0] = %s, cast(ulong)block[0]); writefln (block[1] = %s, cast(ulong)block[1]); writefln (block[2] = %s, cast(ulong)block[2]); writefln (block[3] = %s, cast(ulong)block[3]); writefln (block[4] = %s, cast(ulong)block[4]); writefln (block[5] = %s, cast(ulong)block[5]); writefln (block[6] = %s, cast(ulong)block[6]); writefln (block[7] = %s, cast(ulong)block[7]); ret=cast(ulong)block[n] * 2^21 +cast(ulong)block[n+1] * 2^18 +cast(ulong)block[n+2] * 2^15 +cast(ulong)block[n+3] * 2^12 +cast(ulong)block[n+4] * 2^9 +cast(ulong)block[n+5] * 2^6 +cast(ulong)block[n+6] * 2^3 +cast(ulong)block[n+7] * 2^0; writefln (ret = %s, ret); returnret; } Use the exponentiation operator which is spelled: ^^ Thanks. I was *sure* it was something stupid on my part. It's 'good' to know that I was right about *that*. -- Charles Hixson
Re: ubytes to ulong problem
On 12/21/2013 03:13 PM, John Colvin wrote: Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali
Re: ubytes to ulong problem
On Saturday, 21 December 2013 at 23:52:05 UTC, Ali Çehreli wrote: On 12/21/2013 03:13 PM, John Colvin wrote: Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali Nevermind equally fast, that will be much faster. -10 brain points for me tonight...
Re: ubytes to ulong problem
On 12/21/2013 03:52 PM, Ali Çehreli wrote: On 12/21/2013 03:13 PM, John Colvin wrote: Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes. -- Charles Hixson
Re: ubytes to ulong problem
On 12/21/2013 05:44 PM, Charles Hixson wrote: On 12/21/2013 03:52 PM, Ali Çehreli wrote: On 12/21/2013 03:13 PM, John Colvin wrote: Ideally the compiler will optimise your version to be fast, but you may find you get better performance by doing the bit manipulations eplicitly: Assuming that the program needs to support only big endian and little endian systems (i.e. excluding systems where no D compiler exists :)), the following is less wordy and should be equally fast: import std.bitmanip; import std.system; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = *cast(ulong*)(block.ptr + n); if (std.system.endian == Endian.littleEndian) { return *cast(ulong*)(value.nativeToBigEndian.ptr); } else { return value; } } Ali Will that work even when the alignment is to odd bytes? Because that's the case I was really worried about. The ubyte array is a packed mixture of types, some of which are isolated bytes. No, it is not guaranteed to work unless the alignment is right. How about this, then: :) import std.array; ulong ubytesToUlong(ubyte[] block, size_t n = 0) in { assert (n = 0); assert (n + 8 = block.length); } body { ulong value = block.front; block.popFront(); foreach (ub; block) { value = 8; value |= ub; } return value; } Ali