Re: [Tutor] data storage question
> Based on both replies I got, JSON is what I will use.
>
> I do need the keys in the dictionary to be numerals, specifically they are
> integers.
>
> I believe after I load a stored pt_table, I can use this script to convert
> the keys back to integers.
>
> pt_table = dict((int(key), value) for key, value in pt_table.items())
>
> Please correct me if there is something wrong with that or if there's
> something else I should now about converting the keys to ints after reading
> the stored data.
Hi Colby,
Yes, this looks ok to me. I think we can also express it as a
dictionary comprehension.
(https://docs.python.org/3/tutorial/datastructures.html#dictionaries)
pt_table = {int(key): value for key, value in pt_table.items()}
I think the dictionary comprehension approach is slightly more
idiomatic, but what you've got looks ok too.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] data storage question
> Date: Tue, 2 Aug 2016 12:14:04 +1000
> From: st...@pearwood.info
> To: tutor@python.org
> Subject: Re: [Tutor] data storage question
>
> On Mon, Aug 01, 2016 at 04:47:32PM -0400, Colby Christensen wrote:
>
>> I'm a novice programmer. I have a decent understanding of algorithms
>> but I don't have a lot of computer science/software engineering
>> experience. As a way to help me learn, I've begun a coordinate
>> geometry program similar to the COGO program developed years ago at
>> MIT. Currently, I store the points in a dictionary in the format
>> point_number : [North, East]. I eventually will add a Z component to
>> the points and possibly a description after I get enough of the
>> horizontal geometry worked through. I would like to be able to save
>> the point table so that I can have multiple projects.
>
> For external storage, you have lots of options. Two very common or
> popular suitable standards are JSON or PList.
>
> Suppose you have a point table:
>
> pt_table = {25: [30.1, 42.5, 2.8, 'The Shire'],
> 37: [17.2, 67.2, 11.6, 'Mt Doom'],
> 84: [124.0, 93.8, 65.2, 'Rivendell'],
> }
>
> I can save that table out to a JSON file, then read it back in, like
> this:
>
> py> import json
> py> with open('/tmp/data.json', 'w') as f: # save to table to disk
> ... json.dump(pt_table, f)
> ...
> py> with open('/tmp/data.json', 'r') as f: # read it back in
> ... new_table = json.load(f)
> ...
> py> print new_table
> {u'25': [30.1, 42.5, 2.8, u'The Shire'], u'37': [17.2, 67.2, 11.6, u'Mt
> Doom'], u'84': [124.0, 93.8, 65.2, u'Rivendell']}
>
>
> You'll see that the JSON format has made two changes to the point table:
>
> (1) All strings are Unicode strings instead of "byte strings" (sometimes
> called "ASCII strings").
>
> (2) The keys were numbers (25, 37, 84) but have now been turned into
> Unicode strings too.
Based on both replies I got, JSON is what I will use.
I do need the keys in the dictionary to be numerals, specifically they are
integers.
I believe after I load a stored pt_table, I can use this script to convert the
keys back to integers.
pt_table = dict((int(key), value) for key, value in pt_table.items())
Please correct me if there is something wrong with that or if there's something
else I should now about converting the keys to ints after reading the stored
data.
Thanks for your input!
>
> We can advise you how to deal with these changes. Nevertheless, JSON is
> probably the most common standard used today, and you can see how easy
> the writing and reading of the data is.
>
> Here is an alternative: Plists. Like JSON, Plist requires the keys to be
> strings, but unlike JSON, it won't convert them for you. So you have to
> use strings in the first place, or write a quick converter:
>
> py> pt_table = dict((str(key), value) for key, value in
> pt_table.items())
> py> print pt_table
> {'25': [30.1, 42.5, 2.8, 'The Shire'], '37': [17.2, 67.2, 11.6, 'Mt
> Doom'], '84': [124.0, 93.8, 65.2, 'Rivendell']}
>
>
> Notice that now the keys (which were numbers) are now strings. Now we
> can write to a plist, and read it back:
>
> py> plistlib.writePlist(pt_table, '/tmp/data.plist')
> py> new_table = plistlib.readPlist('/tmp/data.plist')
> py> new_table == pt_table
> True
>
>
> Again, if you need to work with numeric keys, there are ways to work
> around that.
>
> If anything is unclear, please feel free to ask questions on the mailing
> list, and somebody will try to answer them.
>
>
> --
> Steve
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] data storage question
One other thing: you might hear another popular approach is to use the "pickle" module. I don't think it'd be appropriate for your situation because it would be overkill for the problem you're describing. JSON is safer: if you have to choose between JSON and pickle, use JSON. --- More curmudgeon-y details: I'm not a fan of pickle because it does way too much for its own good. It's tied to Python, as opposed to being a language-agnostic file format, so no other tools besides Python know how to parse it well. Furthermore, it has eval() deeply buried within it. That is, it is too powerful for most programmers to use safely. There are a few links you can read if you're interested as to why pickle is probably not what you want. https://blog.nelhage.com/2011/03/exploiting-pickle/ https://lincolnloop.com/blog/playing-pickle-security/ http://www.benfrederickson.com/dont-pickle-your-data/ Just wanted to mention this because it used to be the case where 'pickle' would be a proposed solution to the kind of problem you're encountering. With experience, we can say with more certainty that 'pickle' is a sour pick. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] data storage question
I agree with Steven; JSON is probably one of the most popular formats for saving structured data externally, and it's probably the lightweight approach to use in this situation. By the way, it looks like you're already dealing with a certain file format in your program. In fact, it looks like a simple interpreter, if I understand the program's design. Nice! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] data storage question
On Mon, Aug 01, 2016 at 04:47:32PM -0400, Colby Christensen wrote:
> I'm a novice programmer. I have a decent understanding of algorithms
> but I don't have a lot of computer science/software engineering
> experience. As a way to help me learn, I've begun a coordinate
> geometry program similar to the COGO program developed years ago at
> MIT. Currently, I store the points in a dictionary in the format
> point_number : [North, East]. I eventually will add a Z component to
> the points and possibly a description after I get enough of the
> horizontal geometry worked through. I would like to be able to save
> the point table so that I can have multiple projects.
For external storage, you have lots of options. Two very common or
popular suitable standards are JSON or PList.
Suppose you have a point table:
pt_table = {25: [30.1, 42.5, 2.8, 'The Shire'],
37: [17.2, 67.2, 11.6, 'Mt Doom'],
84: [124.0, 93.8, 65.2, 'Rivendell'],
}
I can save that table out to a JSON file, then read it back in, like
this:
py> import json
py> with open('/tmp/data.json', 'w') as f: # save to table to disk
... json.dump(pt_table, f)
...
py> with open('/tmp/data.json', 'r') as f: # read it back in
... new_table = json.load(f)
...
py> print new_table
{u'25': [30.1, 42.5, 2.8, u'The Shire'], u'37': [17.2, 67.2, 11.6, u'Mt
Doom'], u'84': [124.0, 93.8, 65.2, u'Rivendell']}
You'll see that the JSON format has made two changes to the point table:
(1) All strings are Unicode strings instead of "byte strings" (sometimes
called "ASCII strings").
(2) The keys were numbers (25, 37, 84) but have now been turned into
Unicode strings too.
We can advise you how to deal with these changes. Nevertheless, JSON is
probably the most common standard used today, and you can see how easy
the writing and reading of the data is.
Here is an alternative: Plists. Like JSON, Plist requires the keys to be
strings, but unlike JSON, it won't convert them for you. So you have to
use strings in the first place, or write a quick converter:
py> pt_table = dict((str(key), value) for key, value in
pt_table.items())
py> print pt_table
{'25': [30.1, 42.5, 2.8, 'The Shire'], '37': [17.2, 67.2, 11.6, 'Mt
Doom'], '84': [124.0, 93.8, 65.2, 'Rivendell']}
Notice that now the keys (which were numbers) are now strings. Now we
can write to a plist, and read it back:
py> plistlib.writePlist(pt_table, '/tmp/data.plist')
py> new_table = plistlib.readPlist('/tmp/data.plist')
py> new_table == pt_table
True
Again, if you need to work with numeric keys, there are ways to work
around that.
If anything is unclear, please feel free to ask questions on the mailing
list, and somebody will try to answer them.
--
Steve
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] data storage question
I'm a novice programmer. I have a decent understanding of algorithms but I
don't have a lot of computer science/software engineering experience. As a way
to help me learn, I've begun a coordinate geometry program similar to the COGO
program developed years ago at MIT. Currently, I store the points in a
dictionary in the format point_number : [North, East]. I eventually will add a
Z component to the points and possibly a description after I get enough of the
horizontal geometry worked through. I would like to be able to save the point
table so that I can have multiple projects. My main python script is below. It
reads in a text file that contains commands to create/calculate the location of
the various points. My question is what is a good method to store the point
data outside of python? I eventually will also add the capability to define a
baseline that consists of multiple points with some attributes to define
relative locations along the baseline (stationing).
I use enthought canopy express with python 2.7.11 on windows 10 64 bit.
""" Coordinate geometry program """
import sys
from store_point import store_point
from clear_points import clear_points
from dump_points import dump_points
from redefine import redefine
from dist import dist
from locate_azimuth import locate_azimuth
from locate_bearing import locate_bearing
from locate_line import locate_line
from line_line_int import line_line_int
from parallel_line import parallel_line
from arc_line_pts import arc_line_pts
from divide_line import divide_line
try:
infile = open(raw_input("Enter input file name; name.txt:"),'r')
except:
print "Invalid filename"
exit()
pt_table = {}
cmd_table = {"5":store_point, "7":clear_points, "8":dump_points,
"9":redefine, "10":dist, "11":locate_azimuth,
"12":locate_bearing,
"15":locate_line, "18":parallel_line,
"19":line_line_int, "22":arc_line_pts, "40":divide_line}
count = 0
for line in infile:
#print line
args = line.split()
cmd = args.pop(0)
if cmd in cmd_table:
func = cmd_table[cmd]
func(pt_table, *args)
infile.close()
Thanks,
Colby
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
