Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-07 Thread Mark Lawrence 

On 07/07/2012 09:34, Alan Gauld wrote:

On 07/07/12 04:35, Steven D'Aprano wrote:


I find it ironic that you are suggesting removing the debugger to make
it easier to debug the code :)


But the pdb.trace() calls are not needed for debugging. The
interactive debugger will work just fine without them.
They are for very specialised use. (ie I've never found
a need for them! ;-)


I'd either add some more print statements or I'd run it inside a
debugger.


Which is what pdb is :)


Yes but it doesn't need  the trace() calls to use it and
winpdb (or even IDLE) is much easier to use!


Nevertheless, if the Original Poster is not actually using the debugger,
he should remove the debugger calls.


And that really was the point I was making.



I don't use a debbuger much but when I do winpdb is superb.

--
Cheers.

Mark Lawrence.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-07 Thread Alan Gauld 

On 07/07/12 04:35, Steven D'Aprano wrote:


I find it ironic that you are suggesting removing the debugger to make
it easier to debug the code :)


But the pdb.trace() calls are not needed for debugging. The
interactive debugger will work just fine without them.
They are for very specialised use. (ie I've never found
a need for them! ;-)


I'd either add some more print statements or I'd run it inside a
debugger.


Which is what pdb is :)


Yes but it doesn't need  the trace() calls to use it and
winpdb (or even IDLE) is much easier to use!


Nevertheless, if the Original Poster is not actually using the debugger,
he should remove the debugger calls.


And that really was the point I was making.

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Steven D'Aprano 

Prasad, Ramit wrote:

I believe that the usage of 'in ' converts it into a set (or 
set-like) object so probably that is the same as set(list(set())).


No, certainly not. That would be terribly inefficient, since it means first 
iterating over blah entirely to convert it into a set, and then iterating over 
the set. That does twice as much work as just iterating over blah once.


Besides, sets cannot contain duplicates. If you convert a list to a set, you 
lose any duplicate values in the set.



What 'for x in ' does is first try to use the iterator protocol, and if 
that doesn't work, it tries the sequence protocol, and if that doesn't work it 
raises TypeError.


The iterator protocol looks something like this, implemented in fast C code:


try:
it = iter(blah)  # build an iterator object from blah
except TypeError:
fall back on Sequence Protocol
else:
try:
while True:  # loop forever
x = next(it)  # ask the iterator object for the next value
process x
except StopIteration:
for loop is now done


The advantage here is that building an iterator object need not walk over the 
entire data structure in advance. Iterators should walk over it lazily, only 
as needed, rather than all at once like converting into a set requires.



The Sequence Protocol is the older version, going *way* back to the Python 1.x 
series, again implemented in fast C code. It looks something like this:


i = 0
while True:  # loop forever
try:
x = blah[i]  # get the first item, the second, third, fourth, ...
process x
i += 1
except IndexError:
for loop is now done




--
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Steven D'Aprano 

Ali Torkamani wrote:

I could resolve it by defining a small function:

def getValue(mydict,keys):
A=[];
for i in keys:
A=A+[mydict[i]]
return A



That will be terribly, painfully inefficient for large amounts of data. Do you 
understand Big O notation? The above is O(N**2), compared to O(N) for a list 
comprehension. That means that the amount of work it does increases like the 
square of the number of keys:


- if you double the number of keys, the list comp will do twice as much
  work, but getValue will do four times as much work;

- if you increase the number of keys by 100, the list comp has to do
  100 times more work, but getValue has to do ten thousand times more!


You really want to avoid O(N**2) algorithms.

http://www.joelonsoftware.com/articles/fog000319.html

You can fix that and make it O(N) by using list append instead:

def getValue(mydict, keys):
A=[];
for key in keys:
A.append(mydict[key])
return A

which is a literally translation of the list comprehension
A = [mydict[key] for key in keys]



P.S. please improve your naming conventions. If you have a list of keys, you 
should not use "i" for each key. "i" should only be used for iterating over 
lists of integers.




--
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Steven D'Aprano 

Alan Gauld wrote:

On 06/07/12 22:16, Ali Torkamani wrote:



have the pdb.set_trace()'s for debugging, to see what's going on inside.


So what does it show?

If you aren't using it get rid of it. The less distractions there are 
the easier it is to debug things.


I find it ironic that you are suggesting removing the debugger to make it 
easier to debug the code :)


(Not that I entirely disagree -- I find that 9 times out of 10 it's easier to 
debug code just using print statements and the interactive interpreter than 
with the debugger.)




ho would you debug that?


I'd either add some more print statements or I'd run it inside a 
debugger.


Which is what pdb is :)

Nevertheless, if the Original Poster is not actually using the debugger, he 
should remove the debugger calls.



--
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Steven D'Aprano 

Ali Torkamani wrote:

Dear Tutors,
I'm trying to write a dictionary into a csv file, in the following code
FlatData is the global dictionary whose keys are datetime objects, and the
values are list of dictionaries. sCommonFeatures are key's that exist in
all of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
in some of sub-dictionaries.

The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
(line 10 below), the program stucks!

I appreciate any help.



Start by spending some time preparing some sample code we can actually run. 
The code you supply has inconsistent indentation, which means we have to 
*guess* what the indentation is supposed to be. Even if we can guess, we have 
to guess what your data is, which is impossible.


You should try to prepare a minimal working example that anyone can run and 
that demonstrates the problem. Take care to keep your maximum line length 
under 79 characters, so that mail clients won't word-wrap it. See here for 
more information:


http://sscce.org/


for prog in FD:
D=[prog[key1] for key1 in sCommonFeatures]
print(prog)


So FD is a list of dictionaries? What does FD mean? You should have more 
descriptive names.


And prog is not actually a prog(ram), but a dictionary? You *really* should 
have more descriptive names.


Assuming that all the data you are using are standard, built-in lists and 
dicts, I can't see anything that could be causing this problem. Are you using 
any custom subclasses? If so, perhaps there is a bug in one of your 
__getitem__ methods which enters an infinite loop.


How many values are in sCommonFeatures? If there are tens of millions, perhaps 
you are running out of memory and your operating system is trying to page data 
in and out of virtual memory. That can take hours or even days(!), depending 
on how little physical memory you have and how big the list gets.


The symptom of that is that your entire computer basically becomes unresponsive.

What happens if you cut your input data all the way back to a trivially small 
set of values, say, two keys only in sCommonFeatures, prog only having exactly 
two values, exactly one prog in FD? Does the code still get stuck?


If you add another print statement just before the list comprehension, you can 
see whether or not it actually is the list comp that is getting stuck.


What happens if you take out *all* the pdb calls? Not just in your code 
sample, but everywhere in the entire program. Maybe something in the debugger 
is interfering with the list comp.




--
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Alan Gauld 

On 06/07/12 22:16, Ali Torkamani wrote:

Actually printing some thing after that line does not show any thing,


OK, But it could be any of the lines above the print.
Why are you so sure its that line? You are probably correct but I'd like 
to know why you are so sure? What have you done to isolate the problem?



have the pdb.set_trace()'s for debugging, to see what's going on inside.


So what does it show?

If you aren't using it get rid of it. The less distractions there are 
the easier it is to debug things.



ho would you debug that?


I'd either add some more print statements or I'd run it inside a 
debugger. Winpdb is a good example. I'd set a break point at the loop 
then single step through the loop the first couple of times, then set 
one or more breakpoints inside the loop and run to those.

But my first choice would be the print statements.

One just before the loop, one just inside the loop showing the loop 
variable value so I know how often I've gone round.


HTH,

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Prasad, Ramit 
> > Also what are you using the pdb traces for?
> > Do they show any output?
> Actually printing some thing after that line does not show any thing, I have 
> the pdb.set_trace()'s for debugging, to see what's going on inside. ho would 
> you debug that?

When debugging, I usually just add more print or logging statements.

 print('Calculating D for prog {0}'.format(prog))
 D=[prog[key1] for key1 in sCommonFeatures]
 print('D calculated to be {0}'.format(D))

This lets me see the inputs and output of a block of code.
And for longer algorithms I would put more print statements
to track what is occurring. For example, if you are in a loop, 
it can be useful to use enumerate() so you have a loop counter
and can see which iteration the problem is occurring on. 


But I have not used pdb very much.

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Mark Lawrence 
Please stop top posting, you have been asked at least once before, no 
further comments, TIA.


On 06/07/2012 22:16, Ali Torkamani wrote:

Actually printing some thing after that line does not show any thing, I
have the pdb.set_trace()'s for debugging, to see what's going on inside. ho
would you debug that?

On Fri, Jul 6, 2012 at 3:57 PM, Alan Gauld wrote:


On 06/07/12 18:21, Ali Torkamani wrote:


I'm using Python 2.7.
By stuck I mean, does not pass that specific line (but no errors).



How do you know that is the line it is stuck on?


def WriteOneDayToCSV(**sCommonFeatures,**sOtherFeatures,date):
 FD=FlatData[date]

A=['UniqeDate','Year','Month',**'Day']+list(sCommonFeatures)+**
list(sOtherFeatures)
 pdb.set_trace()
 with 
open(str(date.year)+"_"+str(**date.month)+"_"+str(date.day)+**".csv",'wb')
as myfile:
 fout = csv.writer(myfile, delimiter=',', quotechar="'")

 fout.writerow(A)
 for prog in FD:
 D=[prog[key1] for key1 in sCommonFeatures]

How do you know its not one of the lines above?

Also what are you using the pdb traces for?
Do they show any output?

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/




__**_
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/**mailman/listinfo/tutor





___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor




--
Cheers.

Mark Lawrence.



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Ali Torkamani 
Actually printing some thing after that line does not show any thing, I
have the pdb.set_trace()'s for debugging, to see what's going on inside. ho
would you debug that?

On Fri, Jul 6, 2012 at 3:57 PM, Alan Gauld wrote:

> On 06/07/12 18:21, Ali Torkamani wrote:
>
>> I'm using Python 2.7.
>> By stuck I mean, does not pass that specific line (but no errors).
>>
>>
> How do you know that is the line it is stuck on?
>
>
> def WriteOneDayToCSV(**sCommonFeatures,**sOtherFeatures,date):
> FD=FlatData[date]
>
> A=['UniqeDate','Year','Month',**'Day']+list(sCommonFeatures)+**
> list(sOtherFeatures)
> pdb.set_trace()
> with 
> open(str(date.year)+"_"+str(**date.month)+"_"+str(date.day)+**".csv",'wb')
> as myfile:
> fout = csv.writer(myfile, delimiter=',', quotechar="'")
>
> fout.writerow(A)
> for prog in FD:
> D=[prog[key1] for key1 in sCommonFeatures]
>
> How do you know its not one of the lines above?
>
> Also what are you using the pdb traces for?
> Do they show any output?
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
>
>
>
>
> __**_
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Alan Gauld 

On 06/07/12 18:21, Ali Torkamani wrote:

I'm using Python 2.7.
By stuck I mean, does not pass that specific line (but no errors).



How do you know that is the line it is stuck on?

def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
FD=FlatData[date]

A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures)
pdb.set_trace()
with 
open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') 
as myfile:

fout = csv.writer(myfile, delimiter=',', quotechar="'")

fout.writerow(A)
for prog in FD:
D=[prog[key1] for key1 in sCommonFeatures]

How do you know its not one of the lines above?

Also what are you using the pdb traces for?
Do they show any output?

--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/



___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Ali Torkamani 
I'm using Python 2.7.
By stuck I mean, does not pass that specific line (but no errors).

Like I said FD is a list of dictionaries. and sCommonFeatures are a subset
of key's that have value in all of the dictionaries.



On Fri, Jul 6, 2012 at 1:04 PM, Prasad, Ramit wrote:

> > > I could resolve it by defining a small function:
> > >
> > > def getValue(mydict,keys):
> > > A=[];
> > > for i in keys:
> > > A=A+[mydict[i]]
> > > return A
> > >
> > > and then calling it: D=getValue(prog,sCommonFeatures);
> > > (instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)
> > >
> > > but I'm still surprised why the latter one didn't work!
> >
> > It would be more efficient to do the following
> >
> > def getValue(mydict, keys):
> > A=[]
> > for i in keys:
> > A.append( mydict[i] )
> > return A
>
> Less efficiently, but maybe more useful if you need to check
> against sOtherFeatures you can do the following.
>
> D = [value for key, value in prog.items() if key in sOtherFeatures
>   or key in sCommonFeatures] # Python 3.x syntax
>
>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Prasad, Ramit 
> > I could resolve it by defining a small function:
> >
> > def getValue(mydict,keys):
> > A=[];
> > for i in keys:
> > A=A+[mydict[i]]
> > return A
> >
> > and then calling it: D=getValue(prog,sCommonFeatures);
> > (instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)
> >
> > but I'm still surprised why the latter one didn't work!
> 
> It would be more efficient to do the following
> 
> def getValue(mydict, keys):
> A=[]
> for i in keys:
> A.append( mydict[i] )
> return A

Less efficiently, but maybe more useful if you need to check
against sOtherFeatures you can do the following.

D = [value for key, value in prog.items() if key in sOtherFeatures 
  or key in sCommonFeatures] # Python 3.x syntax


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Prasad, Ramit 
Please do not top post.

> 
> BTW I changed it to:
> D=[prog[key1] for key1 in list(sCommonFeatures)]
> 
> because sCommonFeatures is actually a set, but it still has the smae problem
> On Fri, Jul 6, 2012 at 12:27 PM, Ali Torkamani  wrote:
> Thanks, I checked,  FD is not empty,
> Thanks for reminding about the flush and close, but I think some thing is
> wrong with:
> 
> D=[prog[key1] for key1 in sCommonFeatures]
> In the debug mode it works fine from the command line, but in the for loop it
> gets stuck.

I believe that the usage of 'in ' converts it into a set (or 
set-like) object so probably that is the same as set(list(set())).

Again what do you mean by "stuck"? Does it error or do you just wait? 
Can you provide a small sample script (including sample sCommonFeatures and FD) 
?
What version of Python / operating system?

> I could resolve it by defining a small function:
> 
> def getValue(mydict,keys):
> A=[];
> for i in keys:
> A=A+[mydict[i]]
> return A
> 
> and then calling it: D=getValue(prog,sCommonFeatures);
> (instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)
> 
> but I'm still surprised why the latter one didn't work!

It would be more efficient to do the following

def getValue(mydict, keys):
A=[]
for i in keys:
A.append( mydict[i] )
return A


Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Ali Torkamani 
I could resolve it by defining a small function:

def getValue(mydict,keys):
A=[];
for i in keys:
A=A+[mydict[i]]
return A

and then calling it: D=getValue(prog,sCommonFeatures);
(instead of D=[prog[key1] for key1 in list(sCommonFeatures)];)

but I'm still surprised why the latter one didn't work!


On Fri, Jul 6, 2012 at 12:28 PM, Ali Torkamani  wrote:
>
> BTW I changed it to:
> D=[prog[key1] for key1 in list(sCommonFeatures)]
>
> because sCommonFeatures is actually a set, but it still has the smae
problem
>
>
> On Fri, Jul 6, 2012 at 12:27 PM, Ali Torkamani 
wrote:
>>
>> Thanks, I checked,  FD is not empty,
>> Thanks for reminding about the flush and close, but I think some thing
is wrong with:
>>
>> D=[prog[key1] for key1 in sCommonFeatures]
>>
>> In the debug mode it works fine from the command line, but in the for
loop it gets stuck.
>>
>> Ali
>>
>>
>> On Fri, Jul 6, 2012 at 12:22 PM, Prasad, Ramit 
wrote:
>>>
>>> From: Ali Torkamani
>>> > I'm trying to write a dictionary into a csv file, in the following
code
>>> > FlatData is the global dictionary whose keys are datetime objects,
and the
>>> > values are list of dictionaries. sCommonFeatures are key's that exist
in all
>>> > of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
in some
>>> > of sub-dictionaries.
>>> >
>>> > The problem is that in line : D=[prog[key1] for key1 in
sCommonFeatures]
>>> > (line 10 below), the program stucks!
>>>
>>> Gets stuck how? I see nothing in that line that looks invalid. It may
just
>>> take awhile depending on the amount of keys it has to go through. Maybe
>>> you just need to wait longer?
>>>
>>> >
>>> > I appreciate any help.
>>> >
>>> > def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
>>> > FD=FlatData[date]
>>> >
>>> >
A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
>>> > )
>>> > pdb.set_trace()
>>> > with
>>> >
open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
>>> > myfile:
>>> > fout = csv.writer(myfile, delimiter=',', quotechar="'")
>>> >
>>> > fout.writerow(A)
>>> > for prog in FD:
>>>
>>> Try inserting a print here to make sure FD is not an empty list.
>>>
>>> > D=[prog[key1] for key1 in sCommonFeatures]
>>> > print(prog)
>>> > pdb.set_trace()
>>> > diff=sOtherFeatures.difference(prog.keys())
>>> > prog.update({x:'NONE' for x in diff})
>>> >
>>> > C=[prog[key2] for key2 in sOtherFeatures]
>>> > A=["%4d%02d%02d"
>>> > %(date.year,date.month,date.day),date.year,date.month,date.day]
>>> > fout.writerow(A+D+C)
>>> > pdb.set_trace()
>>> > myfile.flush()
>>> > myfile.close()
>>>
>>> There is no need to flush/close a file that is opened using "with".
>>>
>>> Ramit
>>>
>>>
>>> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
>>> 712 Main Street | Houston, TX 77002
>>> work phone: 713 - 216 - 5423
>>>
>>> --
>>>
>>> ___
>>> Tutor maillist  -  Tutor@python.org
>>> To unsubscribe or change subscription options:
>>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Ali Torkamani 
BTW I changed it to:
D=[prog[key1] for key1 in list(sCommonFeatures)]

because sCommonFeatures is actually a set, but it still has the smae problem

On Fri, Jul 6, 2012 at 12:27 PM, Ali Torkamani  wrote:

> Thanks, I checked,  FD is not empty,
> Thanks for reminding about the flush and close, but I think some thing is
> wrong with:
>
> D=[prog[key1] for key1 in sCommonFeatures]
>
> In the debug mode it works fine from the command line, but in the for loop
> it gets stuck.
>
> Ali
>
>
> On Fri, Jul 6, 2012 at 12:22 PM, Prasad, Ramit 
> wrote:
>
>> From: Ali Torkamani
>> > I'm trying to write a dictionary into a csv file, in the following code
>> > FlatData is the global dictionary whose keys are datetime objects, and
>> the
>> > values are list of dictionaries. sCommonFeatures are key's that exist
>> in all
>> > of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
>> in some
>> > of sub-dictionaries.
>> >
>> > The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
>> > (line 10 below), the program stucks!
>>
>> Gets stuck how? I see nothing in that line that looks invalid. It may just
>> take awhile depending on the amount of keys it has to go through. Maybe
>> you just need to wait longer?
>>
>> >
>> > I appreciate any help.
>> >
>> > def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
>> > FD=FlatData[date]
>> >
>> >
>> A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
>> > )
>> > pdb.set_trace()
>> > with
>> > open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb')
>> as
>> > myfile:
>> > fout = csv.writer(myfile, delimiter=',', quotechar="'")
>> >
>> > fout.writerow(A)
>> > for prog in FD:
>>
>> Try inserting a print here to make sure FD is not an empty list.
>>
>> > D=[prog[key1] for key1 in sCommonFeatures]
>> > print(prog)
>> > pdb.set_trace()
>> > diff=sOtherFeatures.difference(prog.keys())
>> > prog.update({x:'NONE' for x in diff})
>> >
>> > C=[prog[key2] for key2 in sOtherFeatures]
>> > A=["%4d%02d%02d"
>> > %(date.year,date.month,date.day),date.year,date.month,date.day]
>> > fout.writerow(A+D+C)
>> > pdb.set_trace()
>> > myfile.flush()
>> > myfile.close()
>>
>> There is no need to flush/close a file that is opened using "with".
>>
>> Ramit
>>
>>
>> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
>> 712 Main Street | Houston, TX 77002
>> work phone: 713 - 216 - 5423
>>
>> --
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> http://mail.python.org/mailman/listinfo/tutor
>>
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Ali Torkamani 
Thanks, I checked,  FD is not empty,
Thanks for reminding about the flush and close, but I think some thing is
wrong with:
D=[prog[key1] for key1 in sCommonFeatures]

In the debug mode it works fine from the command line, but in the for loop
it gets stuck.

Ali

On Fri, Jul 6, 2012 at 12:22 PM, Prasad, Ramit wrote:

> From: Ali Torkamani
> > I'm trying to write a dictionary into a csv file, in the following code
> > FlatData is the global dictionary whose keys are datetime objects, and
> the
> > values are list of dictionaries. sCommonFeatures are key's that exist in
> all
> > of such 'sub'-dictionaries, and sOtherFeatures are the key's that are in
> some
> > of sub-dictionaries.
> >
> > The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
> > (line 10 below), the program stucks!
>
> Gets stuck how? I see nothing in that line that looks invalid. It may just
> take awhile depending on the amount of keys it has to go through. Maybe
> you just need to wait longer?
>
> >
> > I appreciate any help.
> >
> > def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
> > FD=FlatData[date]
> >
> >
> A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
> > )
> > pdb.set_trace()
> > with
> > open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
> > myfile:
> > fout = csv.writer(myfile, delimiter=',', quotechar="'")
> >
> > fout.writerow(A)
> > for prog in FD:
>
> Try inserting a print here to make sure FD is not an empty list.
>
> > D=[prog[key1] for key1 in sCommonFeatures]
> > print(prog)
> > pdb.set_trace()
> > diff=sOtherFeatures.difference(prog.keys())
> > prog.update({x:'NONE' for x in diff})
> >
> > C=[prog[key2] for key2 in sOtherFeatures]
> > A=["%4d%02d%02d"
> > %(date.year,date.month,date.day),date.year,date.month,date.day]
> > fout.writerow(A+D+C)
> > pdb.set_trace()
> > myfile.flush()
> > myfile.close()
>
> There is no need to flush/close a file that is opened using "with".
>
> Ramit
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
> --
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Prasad, Ramit 
From: Ali Torkamani
> I'm trying to write a dictionary into a csv file, in the following code
> FlatData is the global dictionary whose keys are datetime objects, and the
> values are list of dictionaries. sCommonFeatures are key's that exist in all
> of such 'sub'-dictionaries, and sOtherFeatures are the key's that are in some
> of sub-dictionaries.
> 
> The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
> (line 10 below), the program stucks!

Gets stuck how? I see nothing in that line that looks invalid. It may just
take awhile depending on the amount of keys it has to go through. Maybe
you just need to wait longer?

> 
> I appreciate any help.
> 
> def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
> FD=FlatData[date]
> 
> A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures
> )
> pdb.set_trace()
> with
> open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
> myfile:
> fout = csv.writer(myfile, delimiter=',', quotechar="'")
> 
> fout.writerow(A)
> for prog in FD:

Try inserting a print here to make sure FD is not an empty list. 

> D=[prog[key1] for key1 in sCommonFeatures]
> print(prog)
> pdb.set_trace()
> diff=sOtherFeatures.difference(prog.keys())
> prog.update({x:'NONE' for x in diff})
> 
> C=[prog[key2] for key2 in sOtherFeatures]
> A=["%4d%02d%02d"
> %(date.year,date.month,date.day),date.year,date.month,date.day]
> fout.writerow(A+D+C)
> pdb.set_trace()
> myfile.flush()
> myfile.close()

There is no need to flush/close a file that is opened using "with".

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Program gets stuck after a creating a list from dictinary items!

2012-07-06 Thread Ali Torkamani 
Dear Tutors,
I'm trying to write a dictionary into a csv file, in the following code
FlatData is the global dictionary whose keys are datetime objects, and the
values are list of dictionaries. sCommonFeatures are key's that exist in
all of such 'sub'-dictionaries, and sOtherFeatures are the key's that are
in some of sub-dictionaries.

The problem is that in line : D=[prog[key1] for key1 in sCommonFeatures]
(line 10 below), the program stucks!

I appreciate any help.

def WriteOneDayToCSV(sCommonFeatures,sOtherFeatures,date):
FD=FlatData[date]

A=['UniqeDate','Year','Month','Day']+list(sCommonFeatures)+list(sOtherFeatures)
pdb.set_trace()
with
open(str(date.year)+"_"+str(date.month)+"_"+str(date.day)+".csv",'wb') as
myfile:
fout = csv.writer(myfile, delimiter=',', quotechar="'")

fout.writerow(A)
for prog in FD:
D=[prog[key1] for key1 in sCommonFeatures]
print(prog)
pdb.set_trace()
diff=sOtherFeatures.difference(prog.keys())
prog.update({x:'NONE' for x in diff})

C=[prog[key2] for key2 in sOtherFeatures]
A=["%4d%02d%02d"
%(date.year,date.month,date.day),date.year,date.month,date.day]
fout.writerow(A+D+C)
pdb.set_trace()
myfile.flush()
myfile.close()
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor