[Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
Chip Wachob wrote: > Hello, > > I was not able to find any answers in the archive on this one. > > I'm wondering if this task can be done in a better way than what I've > attempted.. > > I have an array of bytes. Up to 64, which makes for 512 bits. > > I am reading these bytes in serially, and once I have a collection of > them, I want to shift them by 'n' bits. The size of the array and the > number of bits are both variable up to the limit of 64/512. > > Now, I've played around with looping through the bytes and taking the > LSByte and shifting it by 'n' bits using >> or << and that works for > the first byte. But then I need to take the next byte in the sequence > and shift it in the opposite direction by 8-n bits using << or >> > (opposite the LSByte direction), respectively. Then I OR the two > bytes and save them into the location of the LSByte and then move to > the next byte in the sequence and so on. While this works most of the > time, I sometimes get strange behavior at the 'fringes' of the bytes. > Sometimes I end up with zero, or the shift seems to 'roll over'. > > I'm thinking that maybe there's a way to treat the array / list and > shift allowing the bits to transfer from byte to byte as needed. > Maybe by concatenating the bytes into one huge word and then breaking > it apart again? > > I'm thinking that you folks out there know of a built-in function, or, > an easier and more predictable way to accomplish the same. Here are two ways to implement the left shift: def bitshift(b, n, byteorder="big"): size = len(b) + (n + 7) // 8 shifted = int.from_bytes(b, byteorder) << n return shifted.to_bytes(size, byteorder) def bitshift2(b, n): nbytes, nbits = divmod(n, 8) if nbits: a = [0] for bb in b: hi, lo = divmod(bb << nbits, 256) a[-1] |= hi a.append(lo) b = bytes(a) return b + b"\x00" * nbytes assert bitshift(b"\xaa\xbb", 12) == b"\x0a\xab\xb0\x00" assert bitshift2(b"\xaa\xbb", 12) == b"\x0a\xab\xb0\x00" ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
All,
Sorry for not being more clear. I think I was nearing my fill of
Python for the day when I wrote.
Now, refreshed, let me see if I can fill in the blanks a bit more.
- Bits that are shifted in either direction off the end of the
64-byte(or 'n' size) collection are of no use. They are actually
phantom bits created by the hardware that I'm trying to get rid of by
using this shifting. And, the desired data needs to be properly
aligned so I can work with it as well.
- Any bits that are shifted 'in' from either end are zeros.
- What follows is a mini version, the array could be any size up to 64 bytes
input: 10010010 . 1110 . 01010011
shift 'x' (for example, I'll use 1)
output: 01001001 . 01110111 . 10101001
Some results that I've been getting appear as follows:
- 'stuck bits' output:
input : 01010011 . 01110001 . 1111
output : 00101001 . 00111000 . 1111
where the most significant bit of the word appears to 'stick'
regardless of the shifting. This isn't the best example.
- 'roll over' output"
input : 0001 . . 10001000
output : . 1000 . 01000100
The most significant byte here has the 'roll over' issue. I'm guess
this has something to do with sign extension of the byte in the larger
Python storage medium.
The above are not actual data that have been manipulated by the
software but illustrations of observations. The real data is 'random'
so I'm doing the best to recall from memory what happens.
The data is being returned from the hardware as a bytearray but since
I'm reading it a byte at a time, it ends up being stored as a list.
I've been using for loops to cycle through the list.
Ideally, I'd love to be able to say :
# here's the array / list
rx_data = []
# read through all the bytes
# stash the results into the list
for x in range (bytes_of_data):
rx_data[x] = read_data()[0]
# bit shift the entire lot
rx_data = rx_data >> bits_to_shift.
Which would provide me with the output described above.
Code I've been working with is as follows:
def bit_loop_report(disp, num_bits):
if(disp): print "++Begin Report"
num_bits = 0x00
num_bytes = 0x00
# define some constants to help in bit shifting
rt_mask = array ('B', [0x00, 0x01, 0x03, 0x07,
0x0F,
0x1F, 0x3F, 0x7F])
lt_mask = array ('B', [0xFF, 0x80, 0xC0, 0xE0,
0xF0,
0xF8, 0xFC, 0xFE])
# account for partial bytes worth of stops
# determine number of bytes
num_bytes = num_bits / 8
# if we have bits left over then save that count
# between 1 and 7 inclusive
if (num_bits > (num_bytes * 8)):
num_bits = num_bits - (num_bytes * 8)
print " num bits ", num_bits
# if the number of bits left over is non-zero...
if(num_bits != 0):
# increment byte count to cover the additional bits
num_bytes += 1
# calculate the actual number of extra bits
rs_extra_bits = 8 - (num_bits % 8)
print " extra bits ", rs_extra_bits
if(disp): print "Loop size bytes = ", num_bytes
# some vars we'll use
bit_on = False
disp_bit_on = "ON"
disp_bit_off = "OFF"
bit_state_human_read = ""
print "\n\n Use Ctrl-C to Exit \n\n"
# create an array to store stop states
# last read
last_read_ary = array('B', (0x00,)*num_bytes)
r_last_read_ary = array('B', (0x00,)*num_bytes)
# current read
curr_read_ary = array('B', (0x00,)*num_bytes)
r_curr_read_ary = array('B', (0x00,)*num_bytes)
# hold the stop change results
stop_chng_ary = array('B', (0x00,)*num_bytes)
# if num_bits is non-zero, we have to shift
# the data in the array accordingly
done_shft_ary = array('B', (0x00,)*num_bytes)
# clear everything out
bit_loop_flush(disp)
# clear the registers
bit_loop_clear(disp)
# latch the states
bit_loop_strobe(disp)
# read in the initial state of the loop
# bytes are read in backwards because the
# MSByte comes from the hardware first
# (MSByte -> LSByte) and MSbit first.
for first_read in range (0, num_bytes):
# read a byte and stash into the array
r_last_read_ary[first_read] = bit_loop_read(disp)
# reverse the byte order
last_read_ary = list(reversed(r_last_read_ary))
# now we enter a loop waiting for changes
# to the data in the loop
try:
while (True):
# zero out the variables
read_stop = 0x00
temp_log = 0x00
changed = 0x00
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On Mon, Oct 08, 2018 at 09:16:16AM -0400, Chip Wachob wrote: > - What follows is a mini version, the array could be any size up to 64 bytes > > input: 10010010 . 1110 . 01010011 > > shift 'x' (for example, I'll use 1) > > output: 01001001 . 01110111 . 10101001 The first two seem to be simple right bit-shift: 10010010 -> 01001001 1110 -> 01110111 but the third seems to be something completely different, neither a left nor a right shift: 01010011 -> 10101001 A left-bit shift would give this: 10100110 and a right shift would give this: 00101001 So now I don't know what you want. [...] > The above are not actual data that have been manipulated by the > software but illustrations of observations. The real data is 'random' > so I'm doing the best to recall from memory what happens. Seriously, "this is what I vaguely remember happening" is no way to debug software. Show us *actual data* with *actual results* and maybe we can help, otherwise we're just pissing into the wind here. [...] > Ideally, I'd love to be able to say : > > # here's the array / list > rx_data = [] > > # read through all the bytes > # stash the results into the list > for x in range (bytes_of_data): >rx_data[x] = read_data()[0] > > # bit shift the entire lot > rx_data = rx_data >> bits_to_shift. > > Which would provide me with the output described above. Take your byte-array returned from read_data, the *lot* of it, not just the first byte. Convert to an int, and shift the int. py> b = bytearray(4) # work with four bytes py> b[1] = 255 # fill in some non-zero values py> b[2] = 127 py> b bytearray(b'\x00\xff\x7f\x00') py> n = int.from_bytes(b, 'big') py> hex(n) '0xff7f00' py> bin(n >> 1) '0b0111000' py> bin(n << 1) '0b01110' Does that help? -- Steve ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 10/08/2018 07:16 AM, Chip Wachob wrote: > All, > > Sorry for not being more clear. I think I was nearing my fill of > Python for the day when I wrote. > > Now, refreshed, let me see if I can fill in the blanks a bit more. > > - Bits that are shifted in either direction off the end of the > 64-byte(or 'n' size) collection are of no use. They are actually > phantom bits created by the hardware that I'm trying to get rid of by > using this shifting. if you have stray bits, masking is usually a better solution than shifting. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
Chip Wachob wrote:
> All,
>
> Sorry for not being more clear. I think I was nearing my fill of
> Python for the day when I wrote.
>
> Now, refreshed, let me see if I can fill in the blanks a bit more.
You have posted code sprinkled with prints and irrelevant details.
Please rewrite the relevant part into a simple function,
def bitshift_right(data, nshift):
...
just the pure algorithm without any prints. Then find input data that
produces an AssertionError in the snippet below:
input = ...
rshift = ...
wanted_output = ...
assert bitshift_right(input, rshift) == wanted_output
I'm sure then someone is able and willing to point out the error in your
implementation.
In the mean time here's my effort which also provides an idea on how to wrap
the shifting into a simple class:
$ cat bitshift_oo.py
format_bits = "{:08b}".format
try:
format_bits(42)
except ValueError:
# python 2
def format_bits(b):
return bin(b)[2:].zfill(8)
def to_bits(s, sep=""):
return sep.join(map(format_bits, s))
def bitshift_right(b, n):
nbytes, nbits = divmod(n, 8)
length = len(b)
if nbits:
nbits = 8 - nbits
a = [0]
for bb in reversed(b):
hi, lo = divmod(bb << nbits, 256)
a[-1] |= lo
a.append(hi)
b = bytearray(a[::-1])
return (bytearray(nbytes) + b)[:length]
class Bytes(bytearray):
def __rshift__(self, n):
return Bytes(bitshift_right(self, n))
def __str__(self):
return to_bits(self, " ")
if __name__ == "__main__":
SAMPLE = Bytes(b"\xaa\xbb")
for i in range(12):
print(SAMPLE >> i)
$ python bitshift_oo.py
10101010 10111011
01010101 01011101
00101010 10101110
00010101 01010111
1010 10101011
0101 01010101
0010 10101010
0001 01010101
10101010
01010101
00101010
00010101
$
PS: An easier approach would be to match the to_bits() function with a
to_bytes() function. Then you can implement the shift as a simple string
manipulation:
def bitshift_right(b, n):
bits = to_bits(b)
shifted = ("0" * n + bits)[:len(bits)]
return to_bytes(shifted)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
First, Steven, Thank you for your insight. On 10/8/18, Steven D'Aprano wrote: > On Mon, Oct 08, 2018 at 09:16:16AM -0400, Chip Wachob wrote: > >> - What follows is a mini version, the array could be any size up to 64 >> bytes >> >> input: 10010010 . 1110 . 01010011 >> >> shift 'x' (for example, I'll use 1) >> >> output: 01001001 . 01110111 . 10101001 > > The first two seem to be simple right bit-shift: > > 10010010 -> 01001001 > 1110 -> 01110111 > > but the third seems to be something completely different, neither a left > nor a right shift: > > 01010011 -> 10101001 I _think_ that you got the input and output confused on the left-most byte. I checked to make sure that I had not made a typo, but I think that it should be 10101001 => 01010011 which is a right shift of one bit. > > A left-bit shift would give this: 10100110 > and a right shift would give this: 00101001 > > So now I don't know what you want. > > [...] >> The above are not actual data that have been manipulated by the >> software but illustrations of observations. The real data is 'random' >> so I'm doing the best to recall from memory what happens. > > Seriously, "this is what I vaguely remember happening" is no way to > debug software. Show us *actual data* with *actual results* and maybe we > can help, otherwise we're just pissing into the wind here. > I want to say also that I realize that my ethereal results were not very helpful. I'm having trouble actually capturing real data as it seems to be 'intermittent'. I fully expected the Brain Trust to come back with A) There's a much better way to do what you're doing, or B) Hey, there's a function for that, etc. I'm going to keep digging into my problem using these suggestions and get to the bottom of it. I'm sorry if you felt that I was wasting your time. > > [...] >> Ideally, I'd love to be able to say : >> >> # here's the array / list >> rx_data = [] >> >> # read through all the bytes >> # stash the results into the list >> for x in range (bytes_of_data): >>rx_data[x] = read_data()[0] >> >> # bit shift the entire lot >> rx_data = rx_data >> bits_to_shift. >> >> Which would provide me with the output described above. > > Take your byte-array returned from read_data, the *lot* of it, not just > the first byte. Convert to an int, and shift the int. I would be happy to take the 'lot' of it, except that the function returns only one byte at a time. It's a library / driver provided by the manufacturer. > > py> b = bytearray(4) # work with four bytes > py> b[1] = 255 # fill in some non-zero values > py> b[2] = 127 > py> b > bytearray(b'\x00\xff\x7f\x00') > py> n = int.from_bytes(b, 'big') I'm not familiar with this int.from_bytes. Being new to Python, I don't have a mental library of these nifty functions. How / where does one find a library of these wonderful functions so I can search to find something that might be really handy? Could I use the int.from_bytes on a 64-byte array, or would that end up breaking things? Then, I could shift the whole mess, but I would have to break it back apart to use the data. Which, I presume I could slice out of the larger 'word'. > py> hex(n) > '0xff7f00' > py> bin(n >> 1) > '0b0111000' > py> bin(n << 1) > '0b01110' > > Does that help? > It does help somewhat. > > > > -- > Steve > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
Mats, Thanks for the suggestion. Even if I did mask, I would still have to shift the data to line it up properly. And the shifting seems to be where I'm introducing my error(s). This bit stream is complicated in that I have to reverse the bytes as well since I'm reading off the 'end' of a bunch of registers. So, the MSByte comes to me first and goes into the [0] location and the extra bits get added to the '0th' end of the LSByte. It is a very unhandy hardware setup but one that I don't have any control over. All good insight. On 10/8/18, Mats Wichmann wrote: > On 10/08/2018 07:16 AM, Chip Wachob wrote: >> All, >> >> Sorry for not being more clear. I think I was nearing my fill of >> Python for the day when I wrote. >> >> Now, refreshed, let me see if I can fill in the blanks a bit more. >> >> - Bits that are shifted in either direction off the end of the >> 64-byte(or 'n' size) collection are of no use. They are actually >> phantom bits created by the hardware that I'm trying to get rid of by >> using this shifting. > > if you have stray bits, masking is usually a better solution than shifting. > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 09/10/18 02:48, Chip Wachob wrote: >> bytearray(b'\x00\xff\x7f\x00') >> py> n = int.from_bytes(b, 'big') > > I'm not familiar with this int.from_bytes. > > Being new to Python, I don't have a mental library of these nifty > functions. How / where does one find a library of these wonderful > functions so I can search to find something that might be really > handy? The interactive prompt. >>> dir(int) And the help() function >>> help(int) And finally the online documentation and search engine. But dir/help are usually faster. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 09/10/18 02:55, Chip Wachob wrote: > This bit stream is complicated in that I have to reverse the bytes as > well since I'm reading off the 'end' of a bunch of registers. So, the > MSByte comes to me first and goes into the [0] location and the extra > bits get added to the '0th' end of the LSByte. It is a very unhandy > hardware setup but one that I don't have any control over. I'm not sure I understand this fully. Do you mean that each byte has extra bits added or that only the last (LSByte) has extra bits? If it's the latter then it becomes a fairly trivial exercise of slicing the data off rather than shifting. But then you get left with a non-byte. If it's the former then I don't understand how you can determine "byte" boundaries! Also when you say you have to "reverse" the bytes do you mean the bits within each byte need to be reversed? Or merely that the bytes themselves come in the reverse order? If its the latter then the bytearray.reverse() method should suffice to fix that issue. Finally, just to clarify what's happening. It sounds like you have an API that delivers 8 bits(octets) at a time? (Not necessarily actual bytes, just groups of 8 bits, is that correct? And that potentially your actual data is not an even number of octets long (say 14 bits?) and thus you get a few random bits of cruft at the end. Is that the case? If so, how do you know the real length of the actual data? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
Alan, Thank you. Another related question. How do I know I want to do dir(int) vs.. I don't know dir(long)? And, when I do dir(int) I get the following results. I must not be seeing the from_bytes in this list... >>> dir(int) ['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__', '__delattr__', '__div__', '__divmod__', '__doc__', '__float__', '__floordiv__', '__format__', '__getattribute__', '__getnewargs__', '__hash__', '__hex__', '__index__', '__init__', '__int__', '__invert__', '__long__', '__lshift__', '__mod__', '__mul__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'bit_length', 'conjugate', 'denominator', 'imag', 'numerator', 'real'] On 10/9/18, Alan Gauld via Tutor wrote: > On 09/10/18 02:48, Chip Wachob wrote: > >>> bytearray(b'\x00\xff\x7f\x00') >>> py> n = int.from_bytes(b, 'big') >> >> I'm not familiar with this int.from_bytes. >> >> Being new to Python, I don't have a mental library of these nifty >> functions. How / where does one find a library of these wonderful >> functions so I can search to find something that might be really >> handy? > > The interactive prompt. dir(int) > > And the help() function > help(int) > > And finally the online documentation and search engine. > But dir/help are usually faster. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
I'm sorry that my explanations are obtuse... Hopefully this will clarify things. Again, not knowing how much information is too much... I know I can be too verbose and that's not always a good thing. I have a bank of registers that I read via SPI. The SPI API is from the IC manufacturer. The maximum size of that bank is 512 bits. Or, 64 bytes. The register bank can be any valid integer between 1 and 512 long and changes each time the program is run. There is nothing that limits it to byte-sized counts. To determine the size of the bank, I can saturate the register with ones, followed by a stream of 0x00's. When I see the reads from the bank come back as something other than 0xFF, then I know I've 'pushed' zeros through the entire length of the registers. I keep a running count of bytes that contain 0xFF when I do this and if the last byte is something other than 0xFF or 0x00 then I add one to my byte count. This is the byte count for this register until the program is run again. The next time, the bit count is likely to be different. The input to the register array is at the least significant bit. The output is connected to the most significant bit. When I read a register, the SPI pumps out the MSByte first, which goes into my '0'th location in my list (array). (The data is read as MSbit first BTW, which is accurate given this configuration). When the read is complete, I have something that looks like this: data_in[0] = MSByte data_in[1] = MSByte -1 ... data_in[n-1] = LSByte -1 data_in[n] = LSByte* *The data_in[n] (LSByte) is the byte that can have extra '1's in it's least significant bit positions due to the pull-up on the data line.. Only the LSByte has the 'extra' bits. The other bytes are 'normal'. To handle the data in what I'll call a 'normal' order, I use the list.reverse() function that I discovered somewhere on the web. That puts the bytes in what I would call the 'proper' order. So then I have : data_in[0] = LSByte data_in[1] = LSByte - 1 data_in[2] = LSByte - 2 ... data_in[n-1] = MSByte -1 data_in[n] = MSByte I still have the extra bits that were added to the lower bits of data[0], I need to right shift to remove the 'added' ones. If this were only a byte sized register, I would do the shift and that would be sufficient but that is not likely. So, in the example of just one additional bit that needs removing: data_in[0] >>= 1 but, I need the least significant bit from data_in[1] to move into the most significant bit of the now-shifted data_in[0]. This of course has a domino affect down the line until I reach data_in[n]. Since I know the length of the register from my initialization process, I can ignore / mask the most significant bit of data_in[n] in my other calculations. My initial solution was to take data_in[1] and _left_ shift it by the amount of 8 - shift_count. scratch = data_in[n] <<= (8 - #) This moved the LSbit to the MSbit location of scratch. I then OR the data_in[0] shifted result and the scratch result to get my final data_in[0] value. data_in[0] = data_in[0] | scratch I really hope that this makes sense. The API is providing me with a bytearray-formatted result, but the API only reads one whole byte, or eight bits, at a time. So, I've been using only the [0] of the bytearray for my data. I hope that this answers your question about octets. I'm not entirely sure I follow the question well enough to answer. On 10/9/18, Alan Gauld via Tutor wrote: > On 09/10/18 02:55, Chip Wachob wrote: > >> This bit stream is complicated in that I have to reverse the bytes as >> well since I'm reading off the 'end' of a bunch of registers. So, the >> MSByte comes to me first and goes into the [0] location and the extra >> bits get added to the '0th' end of the LSByte. It is a very unhandy >> hardware setup but one that I don't have any control over. > > > I'm not sure I understand this fully. > Do you mean that each byte has extra > bits added or that only the last (LSByte) > has extra bits? > > If it's the latter then it becomes a fairly > trivial exercise of slicing the data off > rather than shifting. But then you get left > with a non-byte. > > If it's the former then I don't understand > how you can determine "byte" boundaries! > > Also when you say you have to "reverse" the > bytes do you mean the bits within each byte > need to be reversed? Or merely that the bytes > themselves come in the reverse order? If > its the latter then the bytearray.reverse() > method should suffice to fix that issue. > > Finally, just to clarify what's happening. > It sounds like you have an API that delivers > 8 bits(octets) at a time? (Not necessarily actual > bytes, just groups of 8 bits, is that correct? > And that potentially your actual data is not > an even number of octets long (say 14 bits?) > and thus you get a few random bits of cruft > at the end. > > Is that the case? If so, how do you know the > real length of the actual data? > > -- > Alan G > Auth
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
Peter,
Thank you for taking the time to create the example.
I'm new to Python, and, as a result, about a quarter of your example
makes sense to me. The remainder I will have to start googling to see
if I can get my head around it.
On 10/9/18, Peter Otten <__pete...@web.de> wrote:
> Chip Wachob wrote:
>
>> All,
>>
>> Sorry for not being more clear. I think I was nearing my fill of
>> Python for the day when I wrote.
>>
>> Now, refreshed, let me see if I can fill in the blanks a bit more.
>
> You have posted code sprinkled with prints and irrelevant details.
> Please rewrite the relevant part into a simple function,
>
> def bitshift_right(data, nshift):
>...
>
> just the pure algorithm without any prints. Then find input data that
> produces an AssertionError in the snippet below:
>
> input = ...
> rshift = ...
> wanted_output = ...
> assert bitshift_right(input, rshift) == wanted_output
>
> I'm sure then someone is able and willing to point out the error in your
> implementation.
>
> In the mean time here's my effort which also provides an idea on how to wrap
>
> the shifting into a simple class:
>
> $ cat bitshift_oo.py
> format_bits = "{:08b}".format
> try:
> format_bits(42)
> except ValueError:
> # python 2
> def format_bits(b):
> return bin(b)[2:].zfill(8)
>
>
> def to_bits(s, sep=""):
> return sep.join(map(format_bits, s))
>
>
> def bitshift_right(b, n):
> nbytes, nbits = divmod(n, 8)
> length = len(b)
> if nbits:
> nbits = 8 - nbits
> a = [0]
> for bb in reversed(b):
> hi, lo = divmod(bb << nbits, 256)
> a[-1] |= lo
> a.append(hi)
> b = bytearray(a[::-1])
> return (bytearray(nbytes) + b)[:length]
>
>
> class Bytes(bytearray):
> def __rshift__(self, n):
> return Bytes(bitshift_right(self, n))
>
> def __str__(self):
> return to_bits(self, " ")
>
>
> if __name__ == "__main__":
> SAMPLE = Bytes(b"\xaa\xbb")
> for i in range(12):
> print(SAMPLE >> i)
> $ python bitshift_oo.py
> 10101010 10111011
> 01010101 01011101
> 00101010 10101110
> 00010101 01010111
> 1010 10101011
> 0101 01010101
> 0010 10101010
> 0001 01010101
> 10101010
> 01010101
> 00101010
> 00010101
> $
>
> PS: An easier approach would be to match the to_bits() function with a
> to_bytes() function. Then you can implement the shift as a simple string
> manipulation:
>
> def bitshift_right(b, n):
> bits = to_bits(b)
> shifted = ("0" * n + bits)[:len(bits)]
> return to_bytes(shifted)
>
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 09/10/18 13:45, Chip Wachob wrote: > Another related question. How do I know I want to do dir(int) vs.. I > don't know dir(long)? Because you want to convert the byte array into an int. And there is no long in Python... > And, when I do dir(int) I get the following results. I must not be > seeing the from_bytes in this list... > dir(int) > ['__abs__', '__add__', '__and__', '__class__', '__cmp__', Ah, I assume this means you are using Python v2.7? I assumed you were using v3. The bytes operations were, I think, added in v3. That complicates things, I think you are probably best converting your byte list into a string and then using the struct module to convert your bytearray into an int. (struct is a bit like C's sscanf function if that helps - I seem to recall you know C?) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 09/10/18 14:21, Chip Wachob wrote: > The API is providing me with a bytearray-formatted result, but the API > only reads one whole byte, or eight bits, at a time. Being slightly picky but an octet and a byte are subtly different. An octet is a group of 8 bits with no meaning attached. So an octet contains 8 bits which could be individual single bit flags, or they could be two groups of 4(nibbles) or a single group of 8(a byte). A byte is a more specific thing that only exists as a set of 8 bits joined together as a single unit of data. Most of the time it makes no significant difference so people often use the terms interchangeably but in this kind of scenario the difference is significant. You are reading octets and want to convert them into a bytearray - removing the extraneous bits in the process. The way I'd do that is convert to an int and then do a single shift right on the result. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 09/10/18 17:10, Alan Gauld via Tutor wrote: On 09/10/18 13:45, Chip Wachob wrote: Another related question. How do I know I want to do dir(int) vs.. I don't know dir(long)? Because you want to convert the byte array into an int. And there is no long in Python... Really? :) mark@mark-HP-15-Notebook-PC:~$ python Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> dir(long) ['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__', '__delattr__', '__div__', '__divmod__', '__doc__', '__float__', '__floordiv__', '__format__', '__getattribute__', '__getnewargs__', '__hash__', '__hex__', '__index__', '__init__', '__int__', '__invert__', '__long__', '__lshift__', '__mod__', '__mul__', '__neg__', '__new__', '__nonzero__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'bit_length', 'conjugate', 'denominator', 'imag', 'numerator', 'real'] -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 09/10/18 17:32, Mark Lawrence wrote: > On 09/10/18 17:10, Alan Gauld via Tutor wrote: >> On 09/10/18 13:45, Chip Wachob wrote: >> >>> Another related question. How do I know I want to do dir(int) vs.. I >>> don't know dir(long)? >> >> Because you want to convert the byte array into an int. >> And there is no long in Python... >> > > Really? :) Quite. I should, of course, have said there is no long in Python v3... We have since established that Chip is using 2.7 so there is indeed a long type. I pretty much stopped using v2 about 3 years ago so I tend to forget about these details. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
On 10/09/2018 12:13 PM, Alan Gauld via Tutor wrote: > Quite. > I should, of course, have said there is no long in Python v3... > > We have since established that Chip is using 2.7 so there > is indeed a long type. > > I pretty much stopped using v2 about 3 years ago so I > tend to forget about these details. > https://pythonclock.org/ tick tick ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] [spoiler] Re: Shifting arrays as though they are a 'word'
Alan, Yes, 2.7 here. I wanted to use 3, but the API only supports 2.7. And, I'm not about to get into rewriting an API. :) So, octets it is. Thank you for the clarification. Yes, my background is C, let me look into your suggested method. It is the shifting across those byte boundaries that is really messing me up. If only they were all just bytes.. but then that wouldn't be any 'fun' would it? Thank you, On 10/9/18, Alan Gauld via Tutor wrote: > On 09/10/18 13:45, Chip Wachob wrote: > >> Another related question. How do I know I want to do dir(int) vs.. I >> don't know dir(long)? > > Because you want to convert the byte array into an int. > And there is no long in Python... > >> And, when I do dir(int) I get the following results. I must not be >> seeing the from_bytes in this list... >> > dir(int) >> ['__abs__', '__add__', '__and__', '__class__', '__cmp__', > > Ah, I assume this means you are using Python v2.7? > I assumed you were using v3. The bytes operations were, > I think, added in v3. > > That complicates things, I think you are probably best > converting your byte list into a string and then using > the struct module to convert your bytearray into an int. > (struct is a bit like C's sscanf function if > that helps - I seem to recall you know C?) > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
