Re: [Tutor] truncated dictionary return
richard kappler gmail.com> writes:
>
>
> Now I'm completely lost. While opening the serial port outside the
function sounds like a good idea, I'm thinking that might not work unless I
am mistaken. The sensorRead function once it's called would then basically
own the serial port barring other traffic, yes? That won't work as the same
serial port that receives sensor data from the arduino sends propulsion and
nav signals to the arduino which, along with controlling/reading the
sensors, also controls the motors used for propulsion, hence only opening
the port when the data is called for. The sensorRead function works, heck
it's not even mine, it was written by one of the gurus here in response to a
question I posed months ago (either Alan or Eryksun IIRC) and does exactly
what it's supposed to do, except for the timing bit.
>
No doubt, the parsing the line into a dictionary part of the function works
(and, yes, it's obvious that it was written by someone with more Python
experience than yours - no offense here :) ).
What I'm trying to tell you is that the overall control flow of your program
seems to be wrong.
>
> Perhaps I'm looking for a simple solution where none exists but I rather
doubt it. I was thinking something along the lines of (psuedo code here)
> check incoming dict for length or number of elements
> if 8, keep
> else retry
>
This won't help, if you cannot implement the retry. You have lost a line of
data at this point and you won't be able to bring it back magically, so the
question is can you live with that?
> While I appreciate the above comments and any help that is offered, I
neither understand them as presented nor think they will fix the problem
with the limited understanding I do have. Again, it could be my lack of
knowledge is preventing me from seeing the light here, but it feels like
we're reinventing the wheel.
>
> I hope that didn't come across as rude, it truly was not intended to be such.
>
> regards, Richard
>
>
I think you're simply not understanding the mechanism of readline(). That
method will gather bytes from the serial port until it sees a newline, then
(and only then!) return the complete line to the caller. That means your
program will be blocked when you call readline() until a complete line has
been transmitted, i.e., your sensorRead function will "own the serial port"
as you call it anyway. If this is not what you want, you'll have to use
read() instead of readline() and manage buffering yourself. Again, with
readline() your script will be halted until a full line of sensor reads data
has been transmitted, independent of where and when you opened the connection.
Now about keeping the connection alive: As you realized (I guess that is
your sync issue), you cannot control when data gets sent. This means that
you need to keep listening continuously or you may miss the beginning of a
transmission. Again, readline() will make sure that you get everything up to
the end of a line, but if you happen to open the arduino connection in the
middle of the transmission of a line, readline() has no means of restoring
the beginning of the line and your input will be incomplete (this is exactly
what's going wrong when your current code fails).
So, keep the connection alive during your script!
Switching between sending and receiving is the task of your main program
(that's the # do something with the data part in my previous message, see
below again):
def sensorRead (arduino):
line = arduino.readline().strip()
line = line.lstrip('{').rstrip('}').strip()
# rest of your code
# your main program:
# open the connection
arduino = serial.Serial('/dev/ttyACM0', 9600)
sleep(1)
# keep operating
while True:
# parse a single line of sensor reads data and store it as a dict
reads = sensorRead(arduino)
# do something with the data, i.e. react to it by sending commands
Best,
Wolfgang
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] truncated dictionary return
Would something like
if len(dict) = 8
return d
else
continue
work?
On Sun, Dec 1, 2013 at 8:54 PM, richard kappler wrote:
> Now I'm completely lost. While opening the serial port outside the
> function sounds like a good idea, I'm thinking that might not work unless I
> am mistaken. The sensorRead function once it's called would then basically
> own the serial port barring other traffic, yes? That won't work as the same
> serial port that receives sensor data from the arduino sends propulsion and
> nav signals to the arduino which, along with controlling/reading the
> sensors, also controls the motors used for propulsion, hence only opening
> the port when the data is called for. The sensorRead function works, heck
> it's not even mine, it was written by one of the gurus here in response to
> a question I posed months ago (either Alan or Eryksun IIRC) and does
> exactly what it's supposed to do, except for the timing bit.
>
> Perhaps I'm looking for a simple solution where none exists but I rather
> doubt it. I was thinking something along the lines of (psuedo code here)
> check incoming dict for length or number of elements
> if 8, keep
> else retry
>
> While I appreciate the above comments and any help that is offered, I
> neither understand them as presented nor think they will fix the problem
> with the limited understanding I do have. Again, it could be my lack of
> knowledge is preventing me from seeing the light here, but it feels like
> we're reinventing the wheel.
>
> I hope that didn't come across as rude, it truly was not intended to be
> such.
>
> regards, Richard
>
>
> On Sun, Dec 1, 2013 at 3:06 PM, spir wrote:
>
>> On 12/01/2013 08:28 PM, richard kappler wrote:
>>
>>> I have a script that reads sensor values gathered by an Arduino board
>>> from
>>> serial as a dictionary, said values to later be used in the AI for Nav &
>>> Control. Here's the script:
>>>
>>> #!/usr/bin/python
>>>
>>> def sensorRead():
>>> import serial
>>> from time import sleep
>>>
>>> sensors = {}
>>> sensors = dict.fromkeys('Sonar1 Sonar2 Sonar3 Sonar4 Dewpoint
>>> Temperature Humidity Light'.split())
>>>
>>> arduino = serial.Serial('/dev/ttyACM0', 9600)
>>> sleep(1)
>>> line = arduino.readline().strip()
>>> line = line.lstrip('{').rstrip('}').strip()
>>>
>>> d = {}
>>> for item in line.split(','):
>>> item = item.strip()
>>> key, value = item.split(':')
>>> key = key.strip()
>>> value = value.strip()
>>> d[key]=int(value)
>>> return d
>>>
>>> I hope that comes through okay, I copied it from the text file so
>>> indentation and such should be fine, if not let me know.
>>>
>>> The script works great with one exception. I understand the problem, I'm
>>> just not sure how to address it. The problem is:
>>>
>>> The Arduino runs on a constant loop, it reads each sensor, sends the key
>>> and the value to the serial bus in format for python to read it as a
>>> dictionary, lather, rinse, repeat.
>>>
>>> Python querries the bus when told. Usually the python script gets the
>>> full
>>> dictionary (all 8 values with keys, brackets etc) but sometimes it
>>> doesn't.
>>> Sometimes it only gets the last few values, sometimes it gets nothing or
>>> misses a bracket and throws an error. This makes sense. They are not in
>>> sync.
>>>
>>> What I need to figure out how to do is have the python script wait until
>>> the next round of values as signified by the opening bracket "{" or check
>>> that it has all 8 values and if not retry or something.
>>>
>>> Would this be an if/else? try? exception?
>>>
>>> I've not yet delved into any of these in my quest to learn python except
>>> if/else and that doesn't feel right for this, so I'm at a loss as to how
>>> to
>>> proceed.
>>>
>>
>> * What is the point of the 'sensors' dict? (also, you don't need to
>> initialise it as an enmpty dict)
>> * If you know about regexps or another matching utility, use that to
>> decode the input line into (key,value) pairs. This is even easier if input
>> has a strict format. Would tell us?
>> * About wrong input (incomplete data), avoid try/except, except if ever
>> most cases are ok (it is very costly in case of exception). Anyway, you
>> need to decode input, so use that to check for errors/missing stuff.
>>
>> If you used a matching tool, its absence of result would directly tell
>> about wrong input (provided your pattern is correct! ;-) As of now, just
>> place checks in your decoding sequence. Possible checks places, at first
>> sight, marked below:
>>
>>
>> line = arduino.readline().strip()
>> line = line.lstrip('{').rstrip('}').strip()
>> # check line not empty
>> # (actually big enough for at least {} + one key:val entry)
>>
>> d = {}
>> # first get items in separate var and check how many:
>>
>> for item in line.split(','):
>> item = item.strip() # not needed, for you strip key/val
>> aga
Re: [Tutor] truncated dictionary return
Now I'm completely lost. While opening the serial port outside the function
sounds like a good idea, I'm thinking that might not work unless I am
mistaken. The sensorRead function once it's called would then basically own
the serial port barring other traffic, yes? That won't work as the same
serial port that receives sensor data from the arduino sends propulsion and
nav signals to the arduino which, along with controlling/reading the
sensors, also controls the motors used for propulsion, hence only opening
the port when the data is called for. The sensorRead function works, heck
it's not even mine, it was written by one of the gurus here in response to
a question I posed months ago (either Alan or Eryksun IIRC) and does
exactly what it's supposed to do, except for the timing bit.
Perhaps I'm looking for a simple solution where none exists but I rather
doubt it. I was thinking something along the lines of (psuedo code here)
check incoming dict for length or number of elements
if 8, keep
else retry
While I appreciate the above comments and any help that is offered, I
neither understand them as presented nor think they will fix the problem
with the limited understanding I do have. Again, it could be my lack of
knowledge is preventing me from seeing the light here, but it feels like
we're reinventing the wheel.
I hope that didn't come across as rude, it truly was not intended to be
such.
regards, Richard
On Sun, Dec 1, 2013 at 3:06 PM, spir wrote:
> On 12/01/2013 08:28 PM, richard kappler wrote:
>
>> I have a script that reads sensor values gathered by an Arduino board from
>> serial as a dictionary, said values to later be used in the AI for Nav &
>> Control. Here's the script:
>>
>> #!/usr/bin/python
>>
>> def sensorRead():
>> import serial
>> from time import sleep
>>
>> sensors = {}
>> sensors = dict.fromkeys('Sonar1 Sonar2 Sonar3 Sonar4 Dewpoint
>> Temperature Humidity Light'.split())
>>
>> arduino = serial.Serial('/dev/ttyACM0', 9600)
>> sleep(1)
>> line = arduino.readline().strip()
>> line = line.lstrip('{').rstrip('}').strip()
>>
>> d = {}
>> for item in line.split(','):
>> item = item.strip()
>> key, value = item.split(':')
>> key = key.strip()
>> value = value.strip()
>> d[key]=int(value)
>> return d
>>
>> I hope that comes through okay, I copied it from the text file so
>> indentation and such should be fine, if not let me know.
>>
>> The script works great with one exception. I understand the problem, I'm
>> just not sure how to address it. The problem is:
>>
>> The Arduino runs on a constant loop, it reads each sensor, sends the key
>> and the value to the serial bus in format for python to read it as a
>> dictionary, lather, rinse, repeat.
>>
>> Python querries the bus when told. Usually the python script gets the full
>> dictionary (all 8 values with keys, brackets etc) but sometimes it
>> doesn't.
>> Sometimes it only gets the last few values, sometimes it gets nothing or
>> misses a bracket and throws an error. This makes sense. They are not in
>> sync.
>>
>> What I need to figure out how to do is have the python script wait until
>> the next round of values as signified by the opening bracket "{" or check
>> that it has all 8 values and if not retry or something.
>>
>> Would this be an if/else? try? exception?
>>
>> I've not yet delved into any of these in my quest to learn python except
>> if/else and that doesn't feel right for this, so I'm at a loss as to how
>> to
>> proceed.
>>
>
> * What is the point of the 'sensors' dict? (also, you don't need to
> initialise it as an enmpty dict)
> * If you know about regexps or another matching utility, use that to
> decode the input line into (key,value) pairs. This is even easier if input
> has a strict format. Would tell us?
> * About wrong input (incomplete data), avoid try/except, except if ever
> most cases are ok (it is very costly in case of exception). Anyway, you
> need to decode input, so use that to check for errors/missing stuff.
>
> If you used a matching tool, its absence of result would directly tell
> about wrong input (provided your pattern is correct! ;-) As of now, just
> place checks in your decoding sequence. Possible checks places, at first
> sight, marked below:
>
>
> line = arduino.readline().strip()
> line = line.lstrip('{').rstrip('}').strip()
> # check line not empty
> # (actually big enough for at least {} + one key:val entry)
>
> d = {}
> # first get items in separate var and check how many:
>
> for item in line.split(','):
> item = item.strip() # not needed, for you strip key/val
> again later
> # first get tuple from split() and check its size is 2
>
> key, value = item.split(':')
> key = key.strip()
> value = value.strip()# not needed, for int() itself strips
> # maybe one try/except here for checking conversion to
Re: [Tutor] truncated dictionary return
On 12/01/2013 08:28 PM, richard kappler wrote:
I have a script that reads sensor values gathered by an Arduino board from
serial as a dictionary, said values to later be used in the AI for Nav &
Control. Here's the script:
#!/usr/bin/python
def sensorRead():
import serial
from time import sleep
sensors = {}
sensors = dict.fromkeys('Sonar1 Sonar2 Sonar3 Sonar4 Dewpoint
Temperature Humidity Light'.split())
arduino = serial.Serial('/dev/ttyACM0', 9600)
sleep(1)
line = arduino.readline().strip()
line = line.lstrip('{').rstrip('}').strip()
d = {}
for item in line.split(','):
item = item.strip()
key, value = item.split(':')
key = key.strip()
value = value.strip()
d[key]=int(value)
return d
I hope that comes through okay, I copied it from the text file so
indentation and such should be fine, if not let me know.
The script works great with one exception. I understand the problem, I'm
just not sure how to address it. The problem is:
The Arduino runs on a constant loop, it reads each sensor, sends the key
and the value to the serial bus in format for python to read it as a
dictionary, lather, rinse, repeat.
Python querries the bus when told. Usually the python script gets the full
dictionary (all 8 values with keys, brackets etc) but sometimes it doesn't.
Sometimes it only gets the last few values, sometimes it gets nothing or
misses a bracket and throws an error. This makes sense. They are not in
sync.
What I need to figure out how to do is have the python script wait until
the next round of values as signified by the opening bracket "{" or check
that it has all 8 values and if not retry or something.
Would this be an if/else? try? exception?
I've not yet delved into any of these in my quest to learn python except
if/else and that doesn't feel right for this, so I'm at a loss as to how to
proceed.
* What is the point of the 'sensors' dict? (also, you don't need to initialise
it as an enmpty dict)
* If you know about regexps or another matching utility, use that to decode the
input line into (key,value) pairs. This is even easier if input has a strict
format. Would tell us?
* About wrong input (incomplete data), avoid try/except, except if ever most
cases are ok (it is very costly in case of exception). Anyway, you need to
decode input, so use that to check for errors/missing stuff.
If you used a matching tool, its absence of result would directly tell about
wrong input (provided your pattern is correct! ;-) As of now, just place checks
in your decoding sequence. Possible checks places, at first sight, marked below:
line = arduino.readline().strip()
line = line.lstrip('{').rstrip('}').strip()
# check line not empty
# (actually big enough for at least {} + one key:val entry)
d = {}
# first get items in separate var and check how many:
for item in line.split(','):
item = item.strip() # not needed, for you strip key/val again
later
# first get tuple from split() and check its size is 2
key, value = item.split(':')
key = key.strip()
value = value.strip()# not needed, for int() itself strips
# maybe one try/except here for checking conversion to int:
# (else, you need to check yourself it is an int numeral)
d[key]=int(value)
Each check failure is a sign of wrong input. What do you need to do, then?
Abandon the whole round?
Denis
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] truncated dictionary return
"Wolfgang Maier" wrote in message
news:loom.20131201t230651-...@post.gmane.org...
> richard kappler gmail.com> writes:
>
>>
>> I have a script that reads sensor values gathered by an Arduino board
>> from
> serial as a dictionary, said values to later be used in the AI for Nav &
> Control. Here's the script:
>> #!/usr/bin/python
>>
>>
>> def sensorRead():
>> import serial
>> from time import sleep
>>
>> sensors = {}
>> sensors = dict.fromkeys('Sonar1 Sonar2 Sonar3 Sonar4 Dewpoint
> Temperature Humidity Light'.split())
>>
>> arduino = serial.Serial('/dev/ttyACM0', 9600)
>> sleep(1)
>> line = arduino.readline().strip()
>> line = line.lstrip('{').rstrip('}').strip()
>>
>> d = {}
>> for item in line.split(','):
>> item = item.strip()
>> key, value = item.split(':')
>> key = key.strip()
>>
>> value = value.strip()
>> d[key]=int(value)
>> return d
>>
>> The script works great with one exception. I understand the problem, I'm
> just not sure how to address it. The problem is:
>>
>> The Arduino runs on a constant loop, it reads each sensor, sends the key
> and the value to the serial bus in format for python to read it as a
> dictionary, lather, rinse, repeat.
>>
>> Python querries the bus when told. Usually the python script gets the
>> full
> dictionary (all 8 values with keys, brackets etc) but sometimes it
> doesn't.
> Sometimes it only gets the last few values, sometimes it gets nothing or
> misses a bracket and throws an error. This makes sense. They are not in
> sync.
>>
>> What I need to figure out how to do is have the python script wait until
> the next round of values as signified by the opening bracket "{" or check
> that it has all 8 values and if not retry or something.
>>
> There should be no sync issue here. The readline method should read from
> the
> serial port until it reaches an EOL character, then return the whole line
> (i.e., your sleep(1) should be removed since readline() already waits for
> input).
> From what you're describing, the real issue seems to be on the side of the
> sender. Are you sure, it terminates each line with \n as it should? Where
> is
> that code coming from?
> Best,
> Wolfgang
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
I also suspect sleep doesn't work. Two better options would be:
1. read/loop until line terminator,
2. Use serial signals, i.e., RTS/DTS if possible.
Paul Simon
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] truncated dictionary return
Hi again,
think I spotted the problem now:
you’re setting up your connection everytime you enter the function (with the
serial.Serial call), but I guess you’re calling that function repeatedly to
retrieve lines. That’s wrong and means you could loose data that was sent
while you weren’t listening.
You’ll have to open the connection once outside the sensorRead() function,
then pass it the arduino object like this:
def sensorRead (arduino):
line = arduino.readline().strip()
line = line.lstrip('{').rstrip('}').strip()
# rest of your code
# your main program:
arduino = serial.Serial('/dev/ttyACM0', 9600)
sleep(1)
while True:
reads = sensorRead(arduino)
# do something with the data
Hope that helps,
Wolfgang
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] truncated dictionary return
richard kappler gmail.com> writes:
>
> I have a script that reads sensor values gathered by an Arduino board from
serial as a dictionary, said values to later be used in the AI for Nav &
Control. Here's the script:
> #!/usr/bin/python
>
>
> def sensorRead():
> import serial
> from time import sleep
>
> sensors = {}
> sensors = dict.fromkeys('Sonar1 Sonar2 Sonar3 Sonar4 Dewpoint
Temperature Humidity Light'.split())
>
> arduino = serial.Serial('/dev/ttyACM0', 9600)
> sleep(1)
> line = arduino.readline().strip()
> line = line.lstrip('{').rstrip('}').strip()
>
> d = {}
> for item in line.split(','):
> item = item.strip()
> key, value = item.split(':')
> key = key.strip()
>
> value = value.strip()
> d[key]=int(value)
> return d
>
> The script works great with one exception. I understand the problem, I'm
just not sure how to address it. The problem is:
>
> The Arduino runs on a constant loop, it reads each sensor, sends the key
and the value to the serial bus in format for python to read it as a
dictionary, lather, rinse, repeat.
>
> Python querries the bus when told. Usually the python script gets the full
dictionary (all 8 values with keys, brackets etc) but sometimes it doesn't.
Sometimes it only gets the last few values, sometimes it gets nothing or
misses a bracket and throws an error. This makes sense. They are not in sync.
>
> What I need to figure out how to do is have the python script wait until
the next round of values as signified by the opening bracket "{" or check
that it has all 8 values and if not retry or something.
>
There should be no sync issue here. The readline method should read from the
serial port until it reaches an EOL character, then return the whole line
(i.e., your sleep(1) should be removed since readline() already waits for
input).
From what you're describing, the real issue seems to be on the side of the
sender. Are you sure, it terminates each line with \n as it should? Where is
that code coming from?
Best,
Wolfgang
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] truncated dictionary return
I have a script that reads sensor values gathered by an Arduino board from
serial as a dictionary, said values to later be used in the AI for Nav &
Control. Here's the script:
#!/usr/bin/python
def sensorRead():
import serial
from time import sleep
sensors = {}
sensors = dict.fromkeys('Sonar1 Sonar2 Sonar3 Sonar4 Dewpoint
Temperature Humidity Light'.split())
arduino = serial.Serial('/dev/ttyACM0', 9600)
sleep(1)
line = arduino.readline().strip()
line = line.lstrip('{').rstrip('}').strip()
d = {}
for item in line.split(','):
item = item.strip()
key, value = item.split(':')
key = key.strip()
value = value.strip()
d[key]=int(value)
return d
I hope that comes through okay, I copied it from the text file so
indentation and such should be fine, if not let me know.
The script works great with one exception. I understand the problem, I'm
just not sure how to address it. The problem is:
The Arduino runs on a constant loop, it reads each sensor, sends the key
and the value to the serial bus in format for python to read it as a
dictionary, lather, rinse, repeat.
Python querries the bus when told. Usually the python script gets the full
dictionary (all 8 values with keys, brackets etc) but sometimes it doesn't.
Sometimes it only gets the last few values, sometimes it gets nothing or
misses a bracket and throws an error. This makes sense. They are not in
sync.
What I need to figure out how to do is have the python script wait until
the next round of values as signified by the opening bracket "{" or check
that it has all 8 values and if not retry or something.
Would this be an if/else? try? exception?
I've not yet delved into any of these in my quest to learn python except
if/else and that doesn't feel right for this, so I'm at a loss as to how to
proceed.
regards, Richard
--
*Mater tua criceta fuit, et pater tuo redoluit bacarum sambucus*
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
