RE: Byte Swapping (Re Post)
> -Original Message- > From: David Godsey [mailto:[EMAIL PROTECTED] > Sent: Friday, February 17, 2006 08:42 > To: Dirk Bremer > Cc: Gordon Bruce; [EMAIL PROTECTED]; gerald_clark; > mysql@lists.mysql.com; David Godsey > Subject: RE: Byte Swapping (Re Post) > > Endianess is byte ordering not bit ordering: > http://www.cs.umass.edu/~verts/cs32/endian.html > http://www.webopedia.com/TERM/b/big_endian.html Note that endian-ness can be expressed within a byte and within a word. It depends on how the data is delivered. If you were using a string representation of a number, it would be possible to reverse the bytes. If using a binary-form of a number that spans multiple bytes, simply reversing the bytes might not suffice. Again, it depends on the data. Dirk Bremer - Senior Systems Engineer - ESS/AMS - NISC Lake St. Louis MO - USA Central Time Zone 636-755-2652 fax 636-755-2503 [EMAIL PROTECTED] www.nisc.coop -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Byte Swapping (Re Post)
> -Original Message- > From: Gordon Bruce [mailto:[EMAIL PROTECTED] > Sent: Tuesday, February 14, 2006 15:42 > To: [EMAIL PROTECTED]; gerald_clark > Cc: mysql@lists.mysql.com; David Godsey > Subject: RE: Byte Swapping (Re Post) > > If the order of the bytes is opposite between big-endian and > little-endian, then if you can get the bytes in a string REVERSE() > should flip the order. > REVERSE would alter the order of the bytes. To convert between big-endian and little-endian, I believe that you need to reverse the order of the bits in either a byte or a word. Dirk Bremer - Senior Systems Engineer - ESS/AMS - NISC Lake St. Louis MO - USA Central Time Zone 636-755-2652 fax 636-755-2503 [EMAIL PROTECTED] www.nisc.coop -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Byte Swapping (Re Post)
If the order of the bytes is opposite between big-endian and little-endian, then if you can get the bytes in a string REVERSE() should flip the order. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 14, 2006 3:25 PM To: gerald_clark Cc: mysql@lists.mysql.com; David Godsey Subject: Re: Byte Swapping (Re Post) gerald_clark <[EMAIL PROTECTED]> wrote on 02/14/2006 03:59:21 PM: > [EMAIL PROTECTED] wrote: > > >"David Godsey" <[EMAIL PROTECTED]> wrote on 02/14/2006 03:28:41 PM: > > > > > > > >>Well, just thought I'd try one more time because I didn't get an answer > >> > >> > >to > > > > > >>my question last time. > >> > >>So what I have is a random data stream that is sent in raw form, and > >> > >> > >based > > > > > >>on some data definition, I can assemble with the correct data types and > >>such. One of my requirements is that I have to store the data in raw > >>form, and when I pull the data out, it displays based on the > >> > >> > >configuration > > > > > >>(with the correct data types and such). So floats and doubles are IEEE > >>standards so I don't have to worry about those, however with integer > >>types, I may need to do some byte swapping (because this data can come > >>from variouse systems that could be either big or little endian). So I > >> > >> > >am > > > > > >>singling out the data I need, but now I need to add the ability to byte > >>swap the data. > >> > >>Keep in mind that it would be best if I can do this in SQL so that it is > >>portable. I realize that it can easily be done in C, but that makes my > >>code less portable (which is also a requirement, to have it portable > >> > >> > >that > > > > > >>is). So does anybody know of a MySQL function that is already > >> > >> > >implemented > > > > > >>to do byte swapping? or know of a way to implement this in SQL? > >> > >>If not, is my only other option to write a UDF? > >> > >>Thanks for any help. > >> > >>Accomplishing the impossible means only that the boss will add it to > >> > >> > >your > > > > > >>regular duties. > >> > >>David Godsey > >> > >> > >> > > > >Native functions? No. Something you can cobble together? Yes. There > >should be several ways you can deal with your data as a string of binary > >characters. Just re-sequence those and you should have your bytes swapped. > > > >One idea is to use the substring functions directly on your BINARY string. > >Another is to use the substring functions in combination with > >HEX()/UNHEX() to work on an escaped version of your BINARY string. > > > > > Would not the first zero value character terminate the substring, > rendering it invalid? > > >Sorry or the lame ideas but usually things like this are not handled at > >the database layer but rather in the application layer. Depending on which > >version of MySQL you are using you may be able to define a FUNCTION (a > >different creature than a UDF) or a STORED PROCEDURE to do the swapping. > >Both will be pure SQL and should meet your compatibility needs. Neither > >will be as fast as creating and registering a UDF, though. > > > >Shawn Green > >Database Administrator > >Unimin Corporation - Spruce Pine > > I don't know if it will or if it won't. The original poster (David Godsey) seems to have no trouble extracting specific subsections of raw data from his blob fields. I just assume that working with chunks of raw data that contained zeroes in them was no problem for him. His need is to somehow binarily invert sections of each number (the INET_... functions could also help) in order to convert big-endian to little-endian and vice versa. I was just trying to help point him to some possible functions that may help him to do that. Hopefully he will post back with what works and what doesn't. Shawn Green Database Administrator Unimin Corporation - Spruce Pine -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Byte Swapping (Re Post)
gerald_clark <[EMAIL PROTECTED]> wrote on 02/14/2006 03:59:21 PM: > [EMAIL PROTECTED] wrote: > > >"David Godsey" <[EMAIL PROTECTED]> wrote on 02/14/2006 03:28:41 PM: > > > > > > > >>Well, just thought I'd try one more time because I didn't get an answer > >> > >> > >to > > > > > >>my question last time. > >> > >>So what I have is a random data stream that is sent in raw form, and > >> > >> > >based > > > > > >>on some data definition, I can assemble with the correct data types and > >>such. One of my requirements is that I have to store the data in raw > >>form, and when I pull the data out, it displays based on the > >> > >> > >configuration > > > > > >>(with the correct data types and such). So floats and doubles are IEEE > >>standards so I don't have to worry about those, however with integer > >>types, I may need to do some byte swapping (because this data can come > >>from variouse systems that could be either big or little endian). So I > >> > >> > >am > > > > > >>singling out the data I need, but now I need to add the ability to byte > >>swap the data. > >> > >>Keep in mind that it would be best if I can do this in SQL so that it is > >>portable. I realize that it can easily be done in C, but that makes my > >>code less portable (which is also a requirement, to have it portable > >> > >> > >that > > > > > >>is). So does anybody know of a MySQL function that is already > >> > >> > >implemented > > > > > >>to do byte swapping? or know of a way to implement this in SQL? > >> > >>If not, is my only other option to write a UDF? > >> > >>Thanks for any help. > >> > >>Accomplishing the impossible means only that the boss will add it to > >> > >> > >your > > > > > >>regular duties. > >> > >>David Godsey > >> > >> > >> > > > >Native functions? No. Something you can cobble together? Yes. There > >should be several ways you can deal with your data as a string of binary > >characters. Just re-sequence those and you should have your bytes swapped. > > > >One idea is to use the substring functions directly on your BINARY string. > >Another is to use the substring functions in combination with > >HEX()/UNHEX() to work on an escaped version of your BINARY string. > > > > > Would not the first zero value character terminate the substring, > rendering it invalid? > > >Sorry or the lame ideas but usually things like this are not handled at > >the database layer but rather in the application layer. Depending on which > >version of MySQL you are using you may be able to define a FUNCTION (a > >different creature than a UDF) or a STORED PROCEDURE to do the swapping. > >Both will be pure SQL and should meet your compatibility needs. Neither > >will be as fast as creating and registering a UDF, though. > > > >Shawn Green > >Database Administrator > >Unimin Corporation - Spruce Pine > > I don't know if it will or if it won't. The original poster (David Godsey) seems to have no trouble extracting specific subsections of raw data from his blob fields. I just assume that working with chunks of raw data that contained zeroes in them was no problem for him. His need is to somehow binarily invert sections of each number (the INET_... functions could also help) in order to convert big-endian to little-endian and vice versa. I was just trying to help point him to some possible functions that may help him to do that. Hopefully he will post back with what works and what doesn't. Shawn Green Database Administrator Unimin Corporation - Spruce Pine
Re: Byte Swapping (Re Post)
[EMAIL PROTECTED] wrote: "David Godsey" <[EMAIL PROTECTED]> wrote on 02/14/2006 03:28:41 PM: Well, just thought I'd try one more time because I didn't get an answer to my question last time. So what I have is a random data stream that is sent in raw form, and based on some data definition, I can assemble with the correct data types and such. One of my requirements is that I have to store the data in raw form, and when I pull the data out, it displays based on the configuration (with the correct data types and such). So floats and doubles are IEEE standards so I don't have to worry about those, however with integer types, I may need to do some byte swapping (because this data can come from variouse systems that could be either big or little endian). So I am singling out the data I need, but now I need to add the ability to byte swap the data. Keep in mind that it would be best if I can do this in SQL so that it is portable. I realize that it can easily be done in C, but that makes my code less portable (which is also a requirement, to have it portable that is). So does anybody know of a MySQL function that is already implemented to do byte swapping? or know of a way to implement this in SQL? If not, is my only other option to write a UDF? Thanks for any help. Accomplishing the impossible means only that the boss will add it to your regular duties. David Godsey Native functions? No. Something you can cobble together? Yes. There should be several ways you can deal with your data as a string of binary characters. Just re-sequence those and you should have your bytes swapped. One idea is to use the substring functions directly on your BINARY string. Another is to use the substring functions in combination with HEX()/UNHEX() to work on an escaped version of your BINARY string. Would not the first zero value character terminate the substring, rendering it invalid? Sorry or the lame ideas but usually things like this are not handled at the database layer but rather in the application layer. Depending on which version of MySQL you are using you may be able to define a FUNCTION (a different creature than a UDF) or a STORED PROCEDURE to do the swapping. Both will be pure SQL and should meet your compatibility needs. Neither will be as fast as creating and registering a UDF, though. Shawn Green Database Administrator Unimin Corporation - Spruce Pine -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Byte Swapping (Re Post)
"David Godsey" <[EMAIL PROTECTED]> wrote on 02/14/2006 03:28:41 PM: > Well, just thought I'd try one more time because I didn't get an answer to > my question last time. > > So what I have is a random data stream that is sent in raw form, and based > on some data definition, I can assemble with the correct data types and > such. One of my requirements is that I have to store the data in raw > form, and when I pull the data out, it displays based on the configuration > (with the correct data types and such). So floats and doubles are IEEE > standards so I don't have to worry about those, however with integer > types, I may need to do some byte swapping (because this data can come > from variouse systems that could be either big or little endian). So I am > singling out the data I need, but now I need to add the ability to byte > swap the data. > > Keep in mind that it would be best if I can do this in SQL so that it is > portable. I realize that it can easily be done in C, but that makes my > code less portable (which is also a requirement, to have it portable that > is). So does anybody know of a MySQL function that is already implemented > to do byte swapping? or know of a way to implement this in SQL? > > If not, is my only other option to write a UDF? > > Thanks for any help. > > Accomplishing the impossible means only that the boss will add it to your > regular duties. > > David Godsey > Native functions? No. Something you can cobble together? Yes. There should be several ways you can deal with your data as a string of binary characters. Just re-sequence those and you should have your bytes swapped. One idea is to use the substring functions directly on your BINARY string. Another is to use the substring functions in combination with HEX()/UNHEX() to work on an escaped version of your BINARY string. Sorry or the lame ideas but usually things like this are not handled at the database layer but rather in the application layer. Depending on which version of MySQL you are using you may be able to define a FUNCTION (a different creature than a UDF) or a STORED PROCEDURE to do the swapping. Both will be pure SQL and should meet your compatibility needs. Neither will be as fast as creating and registering a UDF, though. Shawn Green Database Administrator Unimin Corporation - Spruce Pine
