Re: Storing a big amount of path names

2016-02-13 Thread mkondrashin
In my application I have used two approaches: 1. To store paths as a tree (as directories for a tree. 2. For long list of similar paths, to store difference of strings. Though this was c++/obj-c project, I can share a diff code with you if you drip me a line (mkondrashin & gmail , com) -- https

Re: Storing a big amount of path names

2016-02-12 Thread Matt Wheeler
On 12 Feb 2016 21:37, "Mark Lawrence" wrote: > Hopefully not as that would be a waste, it should be made more obvious by using a red hot poker to engrave it onto every newbies' forehead. Even then some simply wouldn't take a blind bit of notice. Yes sorry about that, I think our aim was a little

Re: Storing a big amount of path names

2016-02-12 Thread Ben Finney
Chris Angelico writes: > I actually had that built behind my house, at one point. Sadly, the > letters sank until they were partly embedded into the ground, and > what's left says, in the local language, "Go stick your head in a > PHP", so it's lit up only for special celebrations. Douglas Adams

Re: Storing a big amount of path names

2016-02-12 Thread Mark Lawrence
On 12/02/2016 17:05, Rob Gaddi wrote: Chris Angelico wrote: Start by coding things in the simple and obvious way, and then fix problems only when you see them. Is that statement available in 10 foot letters etched into stone? Hopefully not as that would be a waste, it should be made more o

Re: Storing a big amount of path names

2016-02-12 Thread Chris Angelico
On Sat, Feb 13, 2016 at 4:05 AM, Rob Gaddi wrote: > Chris Angelico wrote: > >> Start by coding things in the >> simple and obvious way, and then fix problems only when you see them. > > Is that statement available in 10 foot letters etched into stone? I actually had that built behind my house, at

Re: Storing a big amount of path names

2016-02-12 Thread Rob Gaddi
Chris Angelico wrote: > Start by coding things in the > simple and obvious way, and then fix problems only when you see them. Is that statement available in 10 foot letters etched into stone? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out o

Re: Storing a big amount of path names

2016-02-11 Thread srinivas devaki
On Feb 12, 2016 6:05 AM, "Paulo da Silva" wrote: > > Hi! > > What is the best (shortest memory usage) way to store lots of pathnames > in memory where: > > 1. Path names are pathname=(dirname,filename) > 2. There many different dirnames but much less than pathnames > 3. dirnames have in general ma

Re: Storing a big amount of path names

2016-02-11 Thread Steven D'Aprano
On Fri, 12 Feb 2016 04:02 pm, Chris Angelico wrote: > On Fri, Feb 12, 2016 at 3:45 PM, Paulo da Silva > wrote: >>> Correct. Two equal strings, passed to sys.intern(), will come back as >>> identical strings, which means they use the same memory. You can have >>> a million references to the same s

Re: Storing a big amount of path names

2016-02-11 Thread Paulo da Silva
Às 05:02 de 12-02-2016, Chris Angelico escreveu: > On Fri, Feb 12, 2016 at 3:45 PM, Paulo da Silva > wrote: ... >> I think a dict, as MRAB suggested, is needed. >> At the end of the store process I may delete the dict. > > I'm not 100% sure of what's going on here, but my suspicion is that a > s

Re: Storing a big amount of path names

2016-02-11 Thread Chris Angelico
On Fri, Feb 12, 2016 at 3:45 PM, Paulo da Silva wrote: >> Correct. Two equal strings, passed to sys.intern(), will come back as >> identical strings, which means they use the same memory. You can have >> a million references to the same string and it takes up no additional >> memory. > I have bein

Re: Storing a big amount of path names

2016-02-11 Thread Paulo da Silva
Às 04:23 de 12-02-2016, Chris Angelico escreveu: > On Fri, Feb 12, 2016 at 3:15 PM, Paulo da Silva > wrote: >> Às 03:49 de 12-02-2016, Chris Angelico escreveu: >>> On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: Apart from all of the other answers that have been given: >> ... >>> >>> Simpl

Re: Storing a big amount of path names

2016-02-11 Thread Chris Angelico
On Fri, Feb 12, 2016 at 3:15 PM, Paulo da Silva wrote: > Às 03:49 de 12-02-2016, Chris Angelico escreveu: >> On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: >>> Apart from all of the other answers that have been given: >>> > ... >> >> Simpler to let the language do that for you: >> > import sys

Re: Storing a big amount of path names

2016-02-11 Thread Paulo da Silva
Às 03:49 de 12-02-2016, Chris Angelico escreveu: > On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: >> Apart from all of the other answers that have been given: >> ... > > Simpler to let the language do that for you: > import sys p1 = sys.intern('foo/bar') p2 = sys.intern('foo/bar') >

Re: Storing a big amount of path names

2016-02-11 Thread Chris Angelico
On Fri, Feb 12, 2016 at 2:13 PM, MRAB wrote: > Apart from all of the other answers that have been given: > p1 = 'foo/bar' p2 = 'foo/bar' id(p1), id(p2) > (982008930176, 982008930120) d = {} id(d.setdefault(p1, p1)) > 982008930176 id(d.setdefault(p2, p2)) > 98200893017

Re: Storing a big amount of path names

2016-02-11 Thread MRAB
On 2016-02-12 00:31, Paulo da Silva wrote: Hi! What is the best (shortest memory usage) way to store lots of pathnames in memory where: 1. Path names are pathname=(dirname,filename) 2. There many different dirnames but much less than pathnames 3. dirnames have in general many chars The idea is

Re: Storing a big amount of path names

2016-02-11 Thread Rob Gaddi
Tim Chase wrote: > On 2016-02-12 00:31, Paulo da Silva wrote: >> What is the best (shortest memory usage) way to store lots of >> pathnames in memory where: >> >> 1. Path names are pathname=(dirname,filename) >> 2. There many different dirnames but much less than pathnames >> 3. dirnames have in

Re: Storing a big amount of path names

2016-02-11 Thread Tim Chase
On 2016-02-12 00:31, Paulo da Silva wrote: > What is the best (shortest memory usage) way to store lots of > pathnames in memory where: > > 1. Path names are pathname=(dirname,filename) > 2. There many different dirnames but much less than pathnames > 3. dirnames have in general many chars > > Th

Re: Storing a big amount of path names

2016-02-11 Thread Ben Finney
Paulo da Silva writes: > What is the best (shortest memory usage) way to store lots of > pathnames in memory I challenge the premise. Why is “shortest memory usage” your criterion for “best”, here? How have you determined that factors like “easily understandable when reading”, or “using standar

Re: Storing a big amount of path names

2016-02-11 Thread Chris Angelico
On Fri, Feb 12, 2016 at 11:31 AM, Paulo da Silva wrote: > What is the best (shortest memory usage) way to store lots of pathnames > in memory where: > > 1. Path names are pathname=(dirname,filename) > 2. There many different dirnames but much less than pathnames > 3. dirnames have in general many