Re: How can I count word frequency in a web site?
On Sunday, November 29, 2015 at 9:51:46 PM UTC-5, Laura Creighton wrote: > In a message of Sun, 29 Nov 2015 21:31:49 -0500, Cem Karan writes: > >You might want to look into Beautiful Soup > >(https://pypi.python.org/pypi/beautifulsoup4), which is an HTML > >screen-scraping tool. I've never used it, but I've heard good things about > >it. > > > >Good luck, > >Cem Karan > > http://codereview.stackexchange.com/questions/73887/finding-the-occurrences-of-all-words-in-movie-scripts > > scrapes a site of movie scripts and then spits out the 10 most common > words. I suspect the OP could modify this script to suit his or her needs. > > Laura Thanks Laura! -- https://mail.python.org/mailman/listinfo/python-list
Re: How can I count word frequency in a web site?
On Sunday, November 29, 2015 at 7:49:40 PM UTC-5, ryguy7272 wrote: > I'm trying to figure out how to count words in a web site. Here is a sample > of the link I want to scrape data from and count specific words. > http://finance.yahoo.com/q/h?s=STRP+Headlines > > I only want to count certain words, like 'fraud', 'lawsuit', etc. I want to > have a way to control for specific words. I have a couple Python scripts > that do this for a text file, but not for a web site. I can post that, if > that's helpful. This works great! Thanks for sharing!! -- https://mail.python.org/mailman/listinfo/python-list
Re: How can I count word frequency in a web site?
On Sunday, November 29, 2015 at 9:32:22 PM UTC-5, Cem Karan wrote:
> You might want to look into Beautiful Soup
> (https://pypi.python.org/pypi/beautifulsoup4), which is an HTML
> screen-scraping tool. I've never used it, but I've heard good things about
> it.
>
> Good luck,
> Cem Karan
>
> On Nov 29, 2015, at 7:49 PM, ryguy7272 wrote:
>
> > I'm trying to figure out how to count words in a web site. Here is a
> > sample of the link I want to scrape data from and count specific words.
> > http://finance.yahoo.com/q/h?s=STRP+Headlines
> >
> > I only want to count certain words, like 'fraud', 'lawsuit', etc. I want
> > to have a way to control for specific words. I have a couple Python
> > scripts that do this for a text file, but not for a web site. I can post
> > that, if that's helpful.
> >
> > --
> > https://mail.python.org/mailman/listinfo/python-list
Ok, this small script will grab everything from the link.
import requests
from bs4 import BeautifulSoup
r = requests.get("http://finance.yahoo.com/q/h?s=STRP+Headlines";)
soup = BeautifulSoup(r.content)
htmltext = soup.prettify()
print htmltext
Now, how can I count specific words like 'fraud' and 'lawsuit'?
--
https://mail.python.org/mailman/listinfo/python-list
How can I count word frequency in a web site?
I'm trying to figure out how to count words in a web site. Here is a sample of the link I want to scrape data from and count specific words. http://finance.yahoo.com/q/h?s=STRP+Headlines I only want to count certain words, like 'fraud', 'lawsuit', etc. I want to have a way to control for specific words. I have a couple Python scripts that do this for a text file, but not for a web site. I can post that, if that's helpful. -- https://mail.python.org/mailman/listinfo/python-list
Re: Does Python allow variables to be passed into function for dynamic screen scraping?
On Saturday, November 28, 2015 at 8:59:04 PM UTC-5, Steven D'Aprano wrote: > On Sun, 29 Nov 2015 09:03 am, ryguy7272 wrote: > > > I'm looking at this URL. > > https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names > > Don't screen-scrape Wikipedia. Just don't. They have an official API for > downloading content, use it. There's even a Python library for downloading > from Wikipedia and other Mediawiki sites: > > https://www.mediawiki.org/wiki/Manual:Pywikibot > > Wikimedia does a fantastic job, for free, and automated screen-scraping > hurts their ability to provide that service. It is rude and anti-social. > Please don't do it. > > > > -- > Steven Thanks Steven. Do you know of a good tutorial for learning about Wikipedia APIs? I'm not sure where to get started on this topic. I did some Google searches, but didn't come up with a lot of useful info...not much actually... -- https://mail.python.org/mailman/listinfo/python-list
Re: Does Python allow variables to be passed into function for dynamic screen scraping?
On Saturday, November 28, 2015 at 5:28:55 PM UTC-5, Laura Creighton wrote:
> In a message of Sat, 28 Nov 2015 14:03:10 -0800, ryguy7272 writes:
> >I'm looking at this URL.
> >https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
> >
> >If I hit F12 I can see tags such as these:
> > > >And so on and so forth.
> >
> >I'm wondering if someone can share a script, or a function, that will allow
> >me to pass in variables and download (or simply print) the results. I saw a
> >sample online that I thought would work, and I made a few modifications but
> >now I keep getting a message that says: ValueError: All objects passed were
> >None
> >
> >Here's the script that I'm playing around with.
> >
> >import requests
> >import pandas as pd
> >from bs4 import BeautifulSoup
> >
> >#Get the relevant webpage set the data up for parsing
> >url = "https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
> >r = requests.get(url)
> >soup=BeautifulSoup(r.content,"lxml")
> >
> >#set up a function to parse the "soup" for each category of information and
> >put it in a DataFrame
> >def get_match_info(soup,tag,class_name):
> >info_array=[]
> >for info in soup.find_all('%s'%tag,attrs={'class':'%s'%class_name}):
> >return pd.DataFrame(info_array)
> >
> >#for each category pass the above function the relevant information i.e. tag
> >names
> >tag1 = get_match_info(soup,"td","title")
> >tag2 = get_match_info(soup,"td","class")
> >
> >#Concatenate the DataFrames to present a final table of all the above info
> >match_info = pd.concat([tag1,tag2],ignore_index=False,axis=1)
> >
> >print match_info
> >
> >I'd greatly appreciate any help with this.
>
> Post your error traceback. If you are getting Value Errors about None,
> then probably something you expect to return a match, isn't. But without
> the actual error, we cannot help much.
>
> Laura
Ok. How do I post the error traceback? I'm using Spyder Python 2.7.
--
https://mail.python.org/mailman/listinfo/python-list
Does Python allow variables to be passed into function for dynamic screen scraping?
I'm looking at this URL.
https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
If I hit F12 I can see tags such as these:
https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
r = requests.get(url)
soup=BeautifulSoup(r.content,"lxml")
#set up a function to parse the "soup" for each category of information and put
it in a DataFrame
def get_match_info(soup,tag,class_name):
info_array=[]
for info in soup.find_all('%s'%tag,attrs={'class':'%s'%class_name}):
return pd.DataFrame(info_array)
#for each category pass the above function the relevant information i.e. tag
names
tag1 = get_match_info(soup,"td","title")
tag2 = get_match_info(soup,"td","class")
#Concatenate the DataFrames to present a final table of all the above info
match_info = pd.concat([tag1,tag2],ignore_index=False,axis=1)
print match_info
I'd greatly appreciate any help with this.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Screen scraper to get all 'a title' elements
On Wednesday, November 25, 2015 at 6:34:00 PM UTC-5, Grobu wrote:
> On 25/11/15 23:48, ryguy7272 wrote:
> >> re.findall( r'\]+title="(.+?)"', html )
> [ ... ]
> > Thanks!! Is that regex? Can you explain exactly what it is doing?
> > Also, it seems to pick up a lot more than just the list I wanted, but
> > that's ok, I can see why it does that.
> >
> > Can you just please explain what it's doing???
> >
>
> Yes it's a regular expression. Because RegEx's use the backslash as an
> escape character, it is advisable to use the "raw string" prefix (r
> before single/double/triple quote. To illustrate it with an example :
> >>> print "1\n2"
> 1
> 2
> >>> print r"1\n2"
> 1\n2
> As the backslash escape character is "neutralized" by the raw string,
> you can use the usual RegEx syntax at leisure :
>
> \]+title="(.+?)"
>
> \<was a mistake on my part, a single < is perfectly enough
> [^>] is a class definition, and the caret (^) character indicates
> negation. Thus it means : any character other than >
> + incidates repetition : one or more of the previous element
> . will match just anything
> .+" is a _greedy_ pattern that would match anything until it encountered
> a double quote
>
> The problem with a greedy pattern is that it doesn't stop at the first
> match. To illustrate :
> >>> a = re.search( r'".+"', 'title="this is a test" class="test"' )
> >>> a.group()
> '"this is a test" class="test"'
>
> It matches the first quote up to the last one.
> On the other hand, you can use the "?" modifier to specify a non-greedy
> pattern :
>
> >>> b = re.search( r'".+?"', 'title="this is a test" class="test"' )
> '"this is a test"'
>
> It matches the first quote and stops looking for further matches after
> the second quote.
>
> Finally, the parentheses are used to indicate a capture group :
> >>> a = re.search( r'"this (is) a (.+?)"', 'title="this is a test"
> class="test"' )
> >>> a.groups()
> ('is', 'test')
>
>
> You can find detailed explanations about Python regular expressions at
> this page : https://docs.python.org/2/howto/regex.html
>
> HTH,
>
> -Grobu-
Wow! Awesome! I bookmarked that link!
Thanks for everything!!!
--
https://mail.python.org/mailman/listinfo/python-list
Re: Screen scraper to get all 'a title' elements
On Wednesday, November 25, 2015 at 5:30:14 PM UTC-5, Grobu wrote:
> Hi
>
> It seems that links on that Wikipedia page follow the structure :
>
>
> You could extract a list of link titles with something like :
> re.findall( r'\]+title="(.+?)"', html )
>
> HTH,
>
> -Grobu-
>
>
> On 25/11/15 21:55, MRAB wrote:
> > On 2015-11-25 20:42, ryguy7272 wrote:
> >> Hello experts. I'm looking at this url:
> >> https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
> >>
> >> I'm trying to figure out how to list all 'a title' elements. For
> >> instance, I see the following:
> >> >> href="/wiki/Accident,_Maryland">Accident
> >> >> href="/w/index.php?title=Ala-Lemu&action=edit&redlink=1">Ala-Lemu
> >> Alert
> >> Apocalypse
> >> Peaks
> >>
> >> So, I tried putting a script together to get 'title'. Here's my attempt.
> >>
> >> import requests
> >> import sys
> >> from bs4 import BeautifulSoup
> >>
> >> url = "https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
> >> source_code = requests.get(url)
> >> plain_text = source_code.text
> >> soup = BeautifulSoup(plain_text)
> >> for link in soup.findAll('title'):
> >> print(link)
> >>
> >> All that does is get the title of the page. I tried to get the links
> >> from that url, with this script.
> >>
> > A 'title' element has the form "". What you should be looking
> > for are 'a' elements, those of the form "".
> >
> >> import urllib2
> >> import re
> >>
> >> #connect to a URL
> >> website =
> >> urllib2.urlopen('https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names')
> >>
> >>
> >> #read html code
> >> html = website.read()
> >>
> >> #use re.findall to get all the links
> >> links = re.findall('"((http|ftp)s?://.*?)"', html)
> >>
> >> print links
> >>
> >> That doesn't work wither. Basically, I'd like to see this.
> >>
> >> Accident
> >> Ala-Lemu
> >> Alert
> >> Apocalypse Peaks
> >> Athol
> >> Å
> >> Barbecue
> >> Båstad
> >> Bastardstown
> >> Batman
> >> Bathmen (Battem), Netherlands
> >> ...
> >> Worms
> >> Yell
> >> Zigzag
> >> Zzyzx
> >>
> >> How can I do that?
> >> Thanks all!!
Thanks!! Is that regex? Can you explain exactly what it is doing?
Also, it seems to pick up a lot more than just the list I wanted, but that's
ok, I can see why it does that.
Can you just please explain what it's doing???
--
https://mail.python.org/mailman/listinfo/python-list
Re: Screen scraper to get all 'a title' elements
On Wednesday, November 25, 2015 at 3:42:21 PM UTC-5, ryguy7272 wrote:
> Hello experts. I'm looking at this url:
> https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
>
> I'm trying to figure out how to list all 'a title' elements. For instance, I
> see the following:
> Accident
> href="/w/index.php?title=Ala-Lemu&action=edit&redlink=1">Ala-Lemu
> Alert
> Apocalypse Peaks
>
> So, I tried putting a script together to get 'title'. Here's my attempt.
>
> import requests
> import sys
> from bs4 import BeautifulSoup
>
> url = "https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
> source_code = requests.get(url)
> plain_text = source_code.text
> soup = BeautifulSoup(plain_text)
> for link in soup.findAll('title'):
> print(link)
>
> All that does is get the title of the page. I tried to get the links from
> that url, with this script.
>
> import urllib2
> import re
>
> #connect to a URL
> website =
> urllib2.urlopen('https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names')
>
> #read html code
> html = website.read()
>
> #use re.findall to get all the links
> links = re.findall('"((http|ftp)s?://.*?)"', html)
>
> print links
>
> That doesn't work wither. Basically, I'd like to see this.
>
> Accident
> Ala-Lemu
> Alert
> Apocalypse Peaks
> Athol
> Å
> Barbecue
> Båstad
> Bastardstown
> Batman
> Bathmen (Battem), Netherlands
> ...
> Worms
> Yell
> Zigzag
> Zzyzx
>
> How can I do that?
> Thanks all!!
Ok, I guess that makes sense. So, I just tried the script below, and got
nothing...
import requests
from bs4 import BeautifulSoup
r = requests.get("https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";)
soup = BeautifulSoup(r.content)
print soup.find_all("a",{"title"})
--
https://mail.python.org/mailman/listinfo/python-list
Screen scraper to get all 'a title' elements
Hello experts. I'm looking at this url:
https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
I'm trying to figure out how to list all 'a title' elements. For instance, I
see the following:
Accident
Ala-Lemu
Alert
Apocalypse Peaks
So, I tried putting a script together to get 'title'. Here's my attempt.
import requests
import sys
from bs4 import BeautifulSoup
url = "https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
source_code = requests.get(url)
plain_text = source_code.text
soup = BeautifulSoup(plain_text)
for link in soup.findAll('title'):
print(link)
All that does is get the title of the page. I tried to get the links from that
url, with this script.
import urllib2
import re
#connect to a URL
website =
urllib2.urlopen('https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names')
#read html code
html = website.read()
#use re.findall to get all the links
links = re.findall('"((http|ftp)s?://.*?)"', html)
print links
That doesn't work wither. Basically, I'd like to see this.
Accident
Ala-Lemu
Alert
Apocalypse Peaks
Athol
Å
Barbecue
Båstad
Bastardstown
Batman
Bathmen (Battem), Netherlands
...
Worms
Yell
Zigzag
Zzyzx
How can I do that?
Thanks all!!
--
https://mail.python.org/mailman/listinfo/python-list
Re: How can I export data from a website and write the contents to a text file?
On Wednesday, November 18, 2015 at 12:41:19 PM UTC-5, ryguy7272 wrote:
> On Wednesday, November 18, 2015 at 12:21:47 PM UTC-5, Denis McMahon wrote:
> > On Wed, 18 Nov 2015 08:37:47 -0800, ryguy7272 wrote:
> >
> > > I'm trying the script below...
> >
> > The problem isn't that you're over-writing the lines (although it may
> > seem that way to you), the problem is that you're overwriting the whole
> > file every time you write a link to it. This is because you open and
> > close the file for every link you write, and you do so in file mode "wb"
> > which restarts writing at the first byte of the file every time.
> >
> > You only need to open and close the text file once, instead of for every
> > link you output. Try moving the lines to open and close the file outside
> > the outer for loop to change the loop from:
> >
> > for item in soup.find_all(class_='lister-list'):
> > for link in item.find_all('a'):
> > # open file
> > # write link to file
> > # close file
> >
> > to:
> >
> > # open file
> > for item in soup.find_all(class_='lister-list'):
> > for link in item.find_all('a'):
> > # write link to file
> > # close file
> >
> > Alternatively, use the with form:
> >
> > with open("blah","wb") as text_file:
> > for item in soup.find_all(class_='lister-list'):
> > for link in item.find_all('a'):
> > # write link to file
> >
> > --
> > Denis McMahon,
>
>
> Yes, I just figured it out. Thanks.
>
> It doesn't seem like the '\n' is doing anything useful. All the text is
> jumbled together. When I open the file in Excel, or Notepad++, it is easy to
> read. However, when I open it in as a regular text file, everything is
> jumbled together. Is there an easy way to fix this?
I finally got it working. It's like this:
"\r\n"
Thanks everyone!!
--
https://mail.python.org/mailman/listinfo/python-list
Re: How can I export data from a website and write the contents to a text file?
On Wednesday, November 18, 2015 at 12:21:47 PM UTC-5, Denis McMahon wrote:
> On Wed, 18 Nov 2015 08:37:47 -0800, ryguy7272 wrote:
>
> > I'm trying the script below...
>
> The problem isn't that you're over-writing the lines (although it may
> seem that way to you), the problem is that you're overwriting the whole
> file every time you write a link to it. This is because you open and
> close the file for every link you write, and you do so in file mode "wb"
> which restarts writing at the first byte of the file every time.
>
> You only need to open and close the text file once, instead of for every
> link you output. Try moving the lines to open and close the file outside
> the outer for loop to change the loop from:
>
> for item in soup.find_all(class_='lister-list'):
> for link in item.find_all('a'):
> # open file
> # write link to file
> # close file
>
> to:
>
> # open file
> for item in soup.find_all(class_='lister-list'):
> for link in item.find_all('a'):
> # write link to file
> # close file
>
> Alternatively, use the with form:
>
> with open("blah","wb") as text_file:
> for item in soup.find_all(class_='lister-list'):
> for link in item.find_all('a'):
> # write link to file
>
> --
> Denis McMahon,
Yes, I just figured it out. Thanks.
It doesn't seem like the '\n' is doing anything useful. All the text is
jumbled together. When I open the file in Excel, or Notepad++, it is easy to
read. However, when I open it in as a regular text file, everything is jumbled
together. Is there an easy way to fix this?
--
https://mail.python.org/mailman/listinfo/python-list
Re: How can I export data from a website and write the contents to a text file?
On Wednesday, November 18, 2015 at 12:04:16 PM UTC-5, ryguy7272 wrote:
> On Wednesday, November 18, 2015 at 11:58:17 AM UTC-5, Chris Angelico wrote:
> > On Thu, Nov 19, 2015 at 3:37 AM, ryguy7272 <> wrote:
> > > text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt",
> > > "wb")
> > > z = str(link)
> > > text_file.write(z + "\n")
> > > text_file.write("\n")
> > > text_file.close()
> >
> > You're opening the file every time you go through the loop,
> > overwriting each time. Instead, open the file once, then start the
> > loop, and then close it at the end. You can use a 'with' statement to
> > do the closing for you, or you can do it the way you are here.
> >
> > ChrisA
>
>
>
> Thanks. What would the code look like? I tried the code below, and got the
> same results.
>
>
> for item in soup.find_all(class_='lister-list'):
> for link in item.find_all('a'):
> #print(link)
> z = str(link)
> text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb")
> text_file.write(z + "\n")
> text_file.close()
Oh, I see, it's like this:
text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb")
var_file.close()
soup = BeautifulSoup(var_html)
for item in soup.find_all(class_='lister-list'):
for link in item.find_all('a'):
#print(link)
z = str(link)
text_file.write(z + "\n")
text_file.close()
However, it's not organized very well, and it's hard to read. I thought the
'\n' would create a new line after one line was written. Now, it seems like
everything is jumbled together. Kind of weird. Am I missing something?
--
https://mail.python.org/mailman/listinfo/python-list
Re: How can I export data from a website and write the contents to a text file?
On Wednesday, November 18, 2015 at 11:58:17 AM UTC-5, Chris Angelico wrote:
> On Thu, Nov 19, 2015 at 3:37 AM, ryguy7272 wrote:
> > text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb")
> > z = str(link)
> > text_file.write(z + "\n")
> > text_file.write("\n")
> > text_file.close()
>
> You're opening the file every time you go through the loop,
> overwriting each time. Instead, open the file once, then start the
> loop, and then close it at the end. You can use a 'with' statement to
> do the closing for you, or you can do it the way you are here.
>
> ChrisA
Thanks. What would the code look like? I tried the code below, and got the
same results.
for item in soup.find_all(class_='lister-list'):
for link in item.find_all('a'):
#print(link)
z = str(link)
text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb")
text_file.write(z + "\n")
text_file.close()
--
https://mail.python.org/mailman/listinfo/python-list
How can I export data from a website and write the contents to a text file?
I'm trying the script below, and it simple writes the last line to a text file.
I want to add a '\n' after each line is written, so I don't overwrite all the
lines.
from bs4 import BeautifulSoup
import urllib2
var_file = urllib2.urlopen("http://www.imdb.com/chart/top";)
var_html = var_file.read()
var_file.close()
soup = BeautifulSoup(var_html)
for item in soup.find_all(class_='lister-list'):
for link in item.find_all('a'):
print(link)
text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb")
z = str(link)
text_file.write(z + "\n")
text_file.write("\n")
text_file.close()
Can someone please help me get this working?
Thanks!!
--
https://mail.python.org/mailman/listinfo/python-list
Re: Automation of Windows app?
On Tuesday, March 31, 2015 at 9:14:38 PM UTC-4, alex23 wrote: > On 23/03/2015 1:43 PM, Michael Torrie wrote: > > As near as I can tell the standard go-to utility for this is a program > > called AutoIt. https://www.autoitscript.com/site/autoit/ > > > > Nothing to do with Python, and its scripting language is maybe not that > > appealing to many, but it does the job, and does it pretty well. > > Actually, it's not *entirely* unrelated, as there's a Python wrapper for it: > > https://pypi.python.org/pypi/PyAutoIt/0.3 > > It's 2.7 only, though, so if 3.x is required, it's also possible to > create your own wrapper using win32com: > > http://stackoverflow.com/questions/151846/get-other-running-processes-window-sizes-in-python/155587#155587 I know this is an old post, but anyway, can't you just use Windows Scheduler? http://windows.microsoft.com/en-us/windows/schedule-task#1TC=windows-7 -- https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV file
On Wednesday, June 17, 2015 at 11:00:24 AM UTC-4, kbtyo wrote:
> I would like to have this JSON object written out to a CSV file so that the
> keys are header fields (for each of the columns) and the values are values
> that are associated with each header field. Is there a best practice for
> working with this? Ideally I would like to recursively iterate through the
> key value pairs. Thank you in advance. I am using Python 3.4 on Windows. My
> editor is Sublime 2.
>
> {
> "CF": {
> "A": "5",
> "FEC": "1/1/0001 12:00:00 AM",
> "TE": null,
> "Locator": null,
> "Message": "Transfer Fee",
> "AT": null,
> "FT": null,
> "FR": "True",
> "FY": null,
> "FR": null,
> "FG": "0",
> "Comment": null,
> "FUD": null,
> "cID": null,
> "GEO": null,
> "ISO": null,
> "TRID": null,
> "XTY": "931083",
> "ANM": null,
> "NM": null
> },
> "CF": "Fee",
> "ID": "2"
> }
Please see this link.
https://pypi.python.org/pypi/xmlutils
I'm not sure if that will help you. I just found it toady. Also, I'm pretty
new to Python. Anyway, hopefully, it gets you going in the right direction...
--
https://mail.python.org/mailman/listinfo/python-list
Re: JSON Object to CSV Question
On Thursday, June 18, 2015 at 2:59:11 AM UTC-4, kbtyo wrote:
> Good Evening Everyone:
>
>
> I would like to have this JSON object written out to a CSV file so that the
> keys are header fields (for each of the columns) and the values are values
> that are associated with each header field. Is there a best practice for
> working with this? Ideally I would like to recursively iterate through the
> key value pairs. Thank you in advance. I am using Python 3.4 on Windows. My
> editor is Sublime 2.
>
>
>
> Here is the JSON object:
>
>
> {
> "Fee": {
> "A": "5",
> "FEC": "1/1/0001 12:00:00 AM",
> "TE": null,
> "Locator": null,
> "Message": "Transfer Fee",
> "AT": null,
> "FT": null,
> "FR": "True",
> "FY": null,
> "FR": null,
> "FG": "0",
> "Comment": null,
> "FUD": null,
> "cID": null,
> "GEO": null,
> "ISO": null,
> "TRID": null,
> "XTY": "931083",
> "ANM": null,
> "NM": null
> },
> "CF": "Fee",
> "ID": "2"
> }
>
> The value, "Fee" associated with the key, "CF" should not be included as a
> column header (only as a value of the key "CF").
>
>
>
> Other than the former, the keys should be headers and the corresponding
> tuples - the field values.
>
> In essence, my goal is to the following:
>
> You get a dictionary object (the "outer" dictionary)You get the data from the
> "CF" key (fixed name?) which is a string ("Fee" in your example)You use that
> value as a key to obtain another value from the same "outer" dictionary,
> which should be a another dictionary (the "inner" dictionary)You make a CSV
> file with:
> a header that contains "CF" plus all keys in the "inner" dictionary that have
> an associated valuethe value from key "CF" in the "outer" dictionary plus all
> non-null values in the "inner" dictionary.
> I have done the following:
>
>
> import csv
> import json
> import sys
>
> def hook(obj):
> return obj
>
> def flatten(obj):
> for k, v in obj:
> if isinstance(v, list):
> yield from flatten(v)
> else:
> yield k, v
>
> if __name__ == "__main__":
> with open("data.json") as f:
> data = json.load(f, object_pairs_hook=hook)
>
> pairs = list(flatten(data))
>
> writer = csv.writer(sys.stdout)
> writer.writerow([k for k, v in pairs])
> writer.writerow([v for k, v in pairs])
>
>
>
> The output is as follows:
>
> $ python3 json2csv.py
> A,FEC,TE,Locator,Message,AT,FT,FR,FY,FR,FG,Comment,FUD,cID,GEO,ISO,TRID,XTY,ANM,NM,CF,ID
>
> 5,1/1/0001 12:00:00 AM,,,Transfer Fee,,,True,,,0,,,931083,,,Fee,2
>
>
> I do not want to have duplicate column names.
>
>
>
> Any advice on other best practices that I may utilize?
I am literally just learning Python now...total newbie...don't take this as the
be-all-end-all...but maybe this will work for you:
https://pypi.python.org/pypi/xmlutils
Read the entire page.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Can I download XML data from the web and save, in as CSV or TXT delimitation?
On Wednesday, August 19, 2015 at 1:14:44 PM UTC-4, Petite Abeille wrote: > > On Aug 19, 2015, at 7:01 PM, Denis McMahon wrote: > > > > Downloading xml from the web is easy > > > > writing csv or txt is easy > > > > The tricky bit is converting the xml you have into the csv or text data > > you want. > > > > curl | xml2 | 2csv > > http://www.ofb.net/~egnor/xml2/ref WOW!!! This is so powerful!!! https://pypi.python.org/pypi/xmlutils You're right, Denis, it is a bit tricky. I definitely need to play with this a bit to get it to do what I really want it to do, but anyway, I think I can take it from here. Thanks everyone!! -- https://mail.python.org/mailman/listinfo/python-list
Re: Can I download XML data from the web and save, in as CSV or TXT delimitation?
On Wednesday, August 19, 2015 at 8:21:50 AM UTC-4, Laura Creighton wrote:
> In a message of Wed, 19 Aug 2015 04:57:44 -0700, ryguy7272 writes:
> >I'm trying to get R to download the data from here:
> >
> >http://www.usda.gov/oce/commodity/wasde/report_format/latest-July-2015-New-Format.xml
> >
> >
> ># install and load the necessary package
> >install.packages("XML")
> >library(XML)
> ># Save the URL of the xml file in a variable
> >
> >xml.url <-
> >"http://www.usda.gov/oce/commodity/wasde/report_format/latest-July-2015-New-Format.xml";
> ># Use the xmlTreePares-function to parse xml file directly from the web
> >
> >xmlfile <- xmlTreeParse(xml.url)
> ># the xml file is now saved as an object you can easily work with in R:
> >class(xmlfile)
> >
> >
> ># Use the xmlRoot-function to access the top node
> >xmltop = xmlRoot(xmlfile)
> ># have a look at the XML-code of the first subnodes:
> >print(xmltop)[1:3]
> >
> >
> >
> >Everything seems fine up to that point. The next line seems to NOT parse
> >the data as I thought it would.
> ># To extract the XML-values from the document, use xmlSApply:
> >datacat <- xmlSApply(xmltop, function(x) xmlSApply(x, xmlValue))
> >
> >
> >
> >I did some research on this, and it seemed to work in other examples of xml
> >data. I guess this data set is different...or I just don't understand this
> >well enough to know what's really going on...
> >
> >Basically, I want to get this:
> >
> >xmltop
> >
> >
> >Into a data table. How can I do that?
> >
> >Thanks.
>
> This is a mailing list about the Python programming language, not R
> xmlSApply is something R uses. The R mailing lists are here:
> https://www.r-project.org/mail.html
>
> When you talk to them, tell them exactly what you were expecting as
> a result, what you got instead, and what error messages were generated.
> Also let them know what verison of R you are using and what operating
> system you are running on. This will make it a lot easier for them
> to help you.
>
> Good luck,
>
> Laura Creighton
Well, yes, I was originally trying to do it it R, but I couldn't get it
working, so I thought I'd try to do it in Python. That was a sample R script.
Can I do essentially the same thing in Python? Can I read the XML from the web?
http://www.usda.gov/oce/commodity/wasde/report_format/latest-July-2015-New-Format.xml
Parse it, or clean it, or whatever, and save it as a CSV or TXT? Is that
possible?
Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
Can I download XML data from the web and save, in as CSV or TXT delimitation?
I'm trying to get R to download the data from here:
http://www.usda.gov/oce/commodity/wasde/report_format/latest-July-2015-New-Format.xml
# install and load the necessary package
install.packages("XML")
library(XML)
# Save the URL of the xml file in a variable
xml.url <-
"http://www.usda.gov/oce/commodity/wasde/report_format/latest-July-2015-New-Format.xml";
# Use the xmlTreePares-function to parse xml file directly from the web
xmlfile <- xmlTreeParse(xml.url)
# the xml file is now saved as an object you can easily work with in R:
class(xmlfile)
# Use the xmlRoot-function to access the top node
xmltop = xmlRoot(xmlfile)
# have a look at the XML-code of the first subnodes:
print(xmltop)[1:3]
Everything seems fine up to that point. The next line seems to NOT parse the
data as I thought it would.
# To extract the XML-values from the document, use xmlSApply:
datacat <- xmlSApply(xmltop, function(x) xmlSApply(x, xmlValue))
I did some research on this, and it seemed to work in other examples of xml
data. I guess this data set is different...or I just don't understand this well
enough to know what's really going on...
Basically, I want to get this:
xmltop
Into a data table. How can I do that?
Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to Calculate NPV?
On Wednesday, July 29, 2015 at 9:59:10 AM UTC-4, ryguy7272 wrote: > I am using Spyder Python 2.7. I'm running this sample code. > > import scipy as sp > cashflows=[50,40,20,10,50] > npv=sp.npv(0.1,cashflows) > round(npv,2) > > > Now, I'm trying to get the NPV, and I don't see any obvious way to get it. > The author of the book that I'm reading gets 144.56. I think that's wrong, > but I don't know for sure, as Python won't do any calculation at all. It's > easy to enter code and run it, but I can't tell how to get Python to actually > DO the calculation. > > Any idea what I'm doing wrong? PERFECT!! SO SIMPLE!! I don't know why the author didn't do that in the book. Thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: How to Calculate NPV?
On Wednesday, July 29, 2015 at 10:21:35 AM UTC-4, ryguy7272 wrote: > On Wednesday, July 29, 2015 at 9:59:10 AM UTC-4, ryguy7272 wrote: > > I am using Spyder Python 2.7. I'm running this sample code. > > > > import scipy as sp > > cashflows=[50,40,20,10,50] > > npv=sp.npv(0.1,cashflows) > > round(npv,2) > > > > > > Now, I'm trying to get the NPV, and I don't see any obvious way to get it. > > The author of the book that I'm reading gets 144.56. I think that's wrong, > > but I don't know for sure, as Python won't do any calculation at all. It's > > easy to enter code and run it, but I can't tell how to get Python to > > actually DO the calculation. > > > > Any idea what I'm doing wrong? > > PERFECT!! SO SIMPLE!! > I don't know why the author didn't do that in the book. > > Thanks! One last thing, for Excel users, leave out the initial CF. Do the NPV on the other CFs, and then add in the initial CF at the end of the NPV function. It's almost like a PV + 1stCF. I don't know why Excel does it like that... -- https://mail.python.org/mailman/listinfo/python-list
How to Calculate NPV?
I am using Spyder Python 2.7. I'm running this sample code. import scipy as sp cashflows=[50,40,20,10,50] npv=sp.npv(0.1,cashflows) round(npv,2) Now, I'm trying to get the NPV, and I don't see any obvious way to get it. The author of the book that I'm reading gets 144.56. I think that's wrong, but I don't know for sure, as Python won't do any calculation at all. It's easy to enter code and run it, but I can't tell how to get Python to actually DO the calculation. Any idea what I'm doing wrong? -- https://mail.python.org/mailman/listinfo/python-list
Re: Run Python Script; Nothing Happens
On Wednesday, July 29, 2015 at 9:19:19 AM UTC-4, ryguy7272 wrote:
> I am using Spyder Python 2.7. I'm running this sample code.
> import numpy as np
> import numpy.random as npr
> import matplotlib.pyplot as plt
> S0 = 100
> r = 0.05
> sigma = 0.25
> T = 30 / 365.
> I = 1
> ST = S0 * np.exp((r - 0.5 * sigma ** 2) * T + sigma * np.sqrt(T) *
> npr.standard_normal(I))
> R_gbm = np.sort(ST - S0)
> plt.hist(R_gbm, bins=50)
> plt.xlabel('absolute return')
> plt.ylabel('frequency')
> plt.grid(True)
>
> I found it in a book, and I'm trying to run various samples of code, in an
> effort to learn Python. So, I click the debug button, and this is what I get.
> > c:\users\rshuell001\untitled12.py(1)()
> -> import numpy as np
> (Pdb)
>
> It seems like it doesn't really do anything. So, I click the exit debug
> button and then click the run button and nothing happens. I get nothing at
> all. In the book, the author got a graph. I get nothing. I think, and I
> could be totally wrong, Python is sending something to a Console, but I can't
> tell where it goes. I opened every Console I could find, and I still see
> nothing happening whatsoever.
>
> Any idea what's wrong here?
YOU ARE RIGHT!! THAT WORKED PERFECT!!
I wonder why the author of the book didn't put that in there.
Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
Run Python Script; Nothing Happens
I am using Spyder Python 2.7. I'm running this sample code.
import numpy as np
import numpy.random as npr
import matplotlib.pyplot as plt
S0 = 100
r = 0.05
sigma = 0.25
T = 30 / 365.
I = 1
ST = S0 * np.exp((r - 0.5 * sigma ** 2) * T + sigma * np.sqrt(T) *
npr.standard_normal(I))
R_gbm = np.sort(ST - S0)
plt.hist(R_gbm, bins=50)
plt.xlabel('absolute return')
plt.ylabel('frequency')
plt.grid(True)
I found it in a book, and I'm trying to run various samples of code, in an
effort to learn Python. So, I click the debug button, and this is what I get.
> c:\users\rshuell001\untitled12.py(1)()
-> import numpy as np
(Pdb)
It seems like it doesn't really do anything. So, I click the exit debug button
and then click the run button and nothing happens. I get nothing at all. In
the book, the author got a graph. I get nothing. I think, and I could be
totally wrong, Python is sending something to a Console, but I can't tell where
it goes. I opened every Console I could find, and I still see nothing
happening whatsoever.
Any idea what's wrong here?
--
https://mail.python.org/mailman/listinfo/python-list
Error: valueError: ordinal must be >= 1
Hello experts. I'm working in Python > Anaconda > Spyder.
I'm reading a book called 'Python for Finance' and I'm trying to run this
sample code:
import numpy as np
import pandas as pd
import pandas.io.data as web
sp500 = web.DataReader('^GSPC', data_source='yahoo', start='1/1/2000',
end='4/14/2014')
sp500.info()
sp500['Close'].plot(grid=True, figsize=(8, 5))
sp500['42d'] = np.round(pd.rolling_mean(sp500['Close'], window=42), 2)
sp500['252d'] = np.round(pd.rolling_mean(sp500['Close'], window=252), 2)
sp500[['Close', '42d', '252d']].tail()
sp500[['Close', '42d', '252d']].plot(grid=True, figsize=(8, 5))
sp500['42-252'] = sp500['42d'] - sp500['252d']
sp500['42-252'].tail()
sp500['42-252'].head()
It seems to work perfectly find when I see the results in the book, but all I'm
getting is this . . .
*** ValueError: ordinal must be >= 1
(Pdb)
Does anyone have any idea what I'm doing wrong?
Thanks, all.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Is there a way to install ALL Python packages?
On Monday, July 20, 2015 at 10:57:47 PM UTC-4, ryguy7272 wrote: > I'd like to install ALL Python packages on my machine. Even if it takes up > 4-5GB, or more, I'd like to get everything, and then use it when I need it. > Now, I'd like to import packages, like numpy and pandas, but nothing will > install. I figure, if I can just install everything, I can simply use it > when I need it, and if I don't need it, then I just won't use it. > > I know R offers this as an option. I figure Python must allow it too. > > Any idea how to grab everything? > > Thanks all. Thanks for the tip. I just downloaded and installed Anaconda. I just successfully ran my first Python script. So, so happy now. Thanks again!! -- https://mail.python.org/mailman/listinfo/python-list
Re: Is there a way to install ALL Python packages?
On Monday, July 20, 2015 at 10:57:47 PM UTC-4, ryguy7272 wrote: > I'd like to install ALL Python packages on my machine. Even if it takes up > 4-5GB, or more, I'd like to get everything, and then use it when I need it. > Now, I'd like to import packages, like numpy and pandas, but nothing will > install. I figure, if I can just install everything, I can simply use it > when I need it, and if I don't need it, then I just won't use it. > > I know R offers this as an option. I figure Python must allow it too. > > Any idea how to grab everything? > > Thanks all. Ok, this makes sense. Thanks for the insight everyone!! -- https://mail.python.org/mailman/listinfo/python-list
Re: Can I copy/paste Python code?
On Monday, July 20, 2015 at 10:50:09 PM UTC-4, ryguy7272 wrote: > I'm trying to copy some Python code from a PDF book that I'm reading. I want > to test out the code, and I can copy it, but when I paste it into the Shell, > everything is all screwed up because of the indentation. Every time I paste > in any kind of code, it seems like everything is immediately left-justified, > and then nothing works. > > Any idea how to make this work easily? Without re-typing hundreds of lines > of code... > > Thanks to all. Thanks for all the help everyone!! -- https://mail.python.org/mailman/listinfo/python-list
Is there a way to install ALL Python packages?
I'd like to install ALL Python packages on my machine. Even if it takes up 4-5GB, or more, I'd like to get everything, and then use it when I need it. Now, I'd like to import packages, like numpy and pandas, but nothing will install. I figure, if I can just install everything, I can simply use it when I need it, and if I don't need it, then I just won't use it. I know R offers this as an option. I figure Python must allow it too. Any idea how to grab everything? Thanks all. -- https://mail.python.org/mailman/listinfo/python-list
Re: Can't Install Pandas
On Sunday, July 19, 2015 at 11:05:46 PM UTC-4, ryguy7272 wrote: > Hello experts. I odwnloaded Pandas, and put it here. > C:\Python34\Scripts\pandas-0.16.2 > > Then, I ran this in what most people call the c-prompt, but I call it the > 'Python 3.4.3 Shell' > "C:\Python34\Scripts\pandas-0.16.2>" "pip install 'setup.py'" > > It seems like everything ran fine, so I try this. > import pandas as pd > > Then I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named 'pandas' > > Any idea what I'm doing wrong? > > > I tried to follow the instructions here. > https://pip.pypa.io/en/latest/installing.html > > > That doesn't work either. > python get-pip.py > SyntaxError: invalid syntax > > I won't even ask the most obvious question, because I guess it's impossible > to do. Rather, can someone please help me to get this working? > > Thanks. Ok. Back to the basics. Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Can I copy/paste Python code?
I'm trying to copy some Python code from a PDF book that I'm reading. I want to test out the code, and I can copy it, but when I paste it into the Shell, everything is all screwed up because of the indentation. Every time I paste in any kind of code, it seems like everything is immediately left-justified, and then nothing works. Any idea how to make this work easily? Without re-typing hundreds of lines of code... Thanks to all. -- https://mail.python.org/mailman/listinfo/python-list
Re: Can't Install Pandas
On Sunday, July 19, 2015 at 11:05:46 PM UTC-4, ryguy7272 wrote: > Hello experts. I odwnloaded Pandas, and put it here. > C:\Python34\Scripts\pandas-0.16.2 > > Then, I ran this in what most people call the c-prompt, but I call it the > 'Python 3.4.3 Shell' > "C:\Python34\Scripts\pandas-0.16.2>" "pip install 'setup.py'" > > It seems like everything ran fine, so I try this. > import pandas as pd > > Then I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named 'pandas' > > Any idea what I'm doing wrong? > > > I tried to follow the instructions here. > https://pip.pypa.io/en/latest/installing.html > > > That doesn't work either. > python get-pip.py > SyntaxError: invalid syntax > > I won't even ask the most obvious question, because I guess it's impossible > to do. Rather, can someone please help me to get this working? > > Thanks. Well, I got everything to install on the cmd window, but I still can't run anything in the Shell. Here's two simple examples. import numpy as np Traceback (most recent call last): File "", line 1, in import numpy as np ImportError: No module named 'numpy' >>> import pandas as pd Traceback (most recent call last): File "", line 1, in import pandas as pd ImportError: No module named 'pandas' According to what I saw in the cmd window, these two things were installed. Nevertheless, they don't run in the Shell. I guess I spend 95% of my time trying to import stuff, and about 5% of my time doing something interesting or useful with Python. -- https://mail.python.org/mailman/listinfo/python-list
Can't Install Pandas
Hello experts. I odwnloaded Pandas, and put it here. C:\Python34\Scripts\pandas-0.16.2 Then, I ran this in what most people call the c-prompt, but I call it the 'Python 3.4.3 Shell' "C:\Python34\Scripts\pandas-0.16.2>" "pip install 'setup.py'" It seems like everything ran fine, so I try this. import pandas as pd Then I get this error. Traceback (most recent call last): File "", line 1, in import pandas as pd ImportError: No module named 'pandas' Any idea what I'm doing wrong? I tried to follow the instructions here. https://pip.pypa.io/en/latest/installing.html That doesn't work either. python get-pip.py SyntaxError: invalid syntax I won't even ask the most obvious question, because I guess it's impossible to do. Rather, can someone please help me to get this working? Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Trying to import numpy
I'm trying to use numpy. I get this error: >>> import numpy as np Traceback (most recent call last): File "", line 1, in import numpy as np ImportError: No module named numpy I followed the instructions here. https://pip.pypa.io/en/latest/installing.html In the c-prompt, I get this error. C:\>python get-pip.py python: can't open file 'get-pip.py': [Errno 2] No such file or directory In python 2.7, I get this error. >>> python get-pip.py SyntaxError: invalid syntax I would say 100% of my errors come from importing python modules. If this every worked, I could do some real work. Instead, I spend 100% of my time trying to make thing that don't work, work. I've already added ';C:\Python27' to the Path under Variable Name. Of course, this makes no difference whatsoever. Ugh. Any thoughts? Anyone? Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Re: very weird pandas behavior
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: > I downloaded pandas and put it in my python directory, then, at the C-prompt, > I ran this: > "pip install pandas" > > It looks like everything downloaded and installed fine. Great. > > Now, in Python Shell, I enter this: > import pandas as pd > > I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named pandas > > > Any idea what I'm doing wrong? Thanks Rustom. That is insightful advice, indeed. I will cherish your wisdom. To everyone else, I'm going back to VBA, VB, C#, Java, SQL, SSIS, R, & Matlab, simply because all of those work perfectly fine. I have countless ways to do everything in the world. For me, Python was just another way to do, what I already do now. I don't have time to screw around with all kind of nonsense that doesn't do anything, other than tell me 1+1=2. That pretty much the only thing that works in Python. To do anything more complex, seems impossible. Rather than make the impossible become possible, I'd rather focus on things that help me do stuff (like process 100,000 man-hours of work in less than 1 hour). Learning Python was both fun & frustrating. If you need to waste time, work with Python. If you need to do real work, use any on the following: VBA, VB, C#, Java, SQL, R, & Matlab. I just uninstalled Python and deleted 15 Python books that I found online. AH! I feel great That's all. -- https://mail.python.org/mailman/listinfo/python-list
Re: very weird pandas behavior
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: > I downloaded pandas and put it in my python directory, then, at the C-prompt, > I ran this: > "pip install pandas" > > It looks like everything downloaded and installed fine. Great. > > Now, in Python Shell, I enter this: > import pandas as pd > > I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named pandas > > > Any idea what I'm doing wrong? I just ran these two commands in the c-prompt: pip install --upgrade numpy pip install --upgrade pandas It seemed like everything was being downloaded and installed. Seems ok. Then I go back to the Python Shell and ran 'import numpy' & 'import pandas' and I still get the errors that I got before. So, I move on to this: pip uninstall numpy pip uninstall pandas It says, cannon uninstall, non installed. Sorry, but that's what drives me nuts. I install a few packages, and the messages that I get says the package is installed...then it says it's NOT installed...I don't know what to think... -- https://mail.python.org/mailman/listinfo/python-list
Re: very weird pandas behavior
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: > I downloaded pandas and put it in my python directory, then, at the C-prompt, > I ran this: > "pip install pandas" > > It looks like everything downloaded and installed fine. Great. > > Now, in Python Shell, I enter this: > import pandas as pd > > I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named pandas > > > Any idea what I'm doing wrong? Ok, if I enter this into the command prompt: 'pip show --files numpy' 'pip show --files pandas' I get nothing returned from either entry. That seems ot be a problem! -- https://mail.python.org/mailman/listinfo/python-list
Re: very weird pandas behavior
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: > I downloaded pandas and put it in my python directory, then, at the C-prompt, > I ran this: > "pip install pandas" > > It looks like everything downloaded and installed fine. Great. > > Now, in Python Shell, I enter this: > import pandas as pd > > I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named pandas > > > Any idea what I'm doing wrong? Thanks for being patient with me everyone. I just tried this: C:\Users\Ryan>pip --version pip 1.5.6 from C:\Python27\lib (python 2.7) C:\Users\Ryan>python --version Python 2.7 I just tried 'pip show --files numpy' in the command prompt; nothing happened. I'll read that link now. I know my eyes are not very good...my vision is quite poor...but I'm trying to read this stuff, and I'm trying to figure out what's going on here... -- https://mail.python.org/mailman/listinfo/python-list
Re: very weird pandas behavior
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: > I downloaded pandas and put it in my python directory, then, at the C-prompt, > I ran this: > "pip install pandas" > > It looks like everything downloaded and installed fine. Great. > > Now, in Python Shell, I enter this: > import pandas as pd > > I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named pandas > > > Any idea what I'm doing wrong? Sorry for being so dense here guys!! I do tons and tons of work with VBA, VB, C#, SQL, R, Matlab, and a little work with Pascal. I guess that's preventing me from understanding how Python works. Sorry, but I just don't understand this thing. I uninstalled Python27 and just reinstalled it. I open the cmd prompt, and entered two short commands: 'pip install numpy' & 'pip install pandas'. Then, I got to Python Shell and enter this: 'import numpy' & 'import pandas'. immediately I get errors. Traceback (most recent call last): File "", line 1, in import numpy ImportError: No module named numpy Traceback (most recent call last): File "", line 1, in import pandas ImportError: No module named pandas I guess if I can't do something simple, I can't do anything complex...or anything at all. Part of the problem is, I don't know why in 2014 we're entering commands in the C-prompt to run a Windows program. I thought all of that stuff was over in the very early 1990s. Also, I can't understand why Python can't download this from the Internet. If someone can help me figure this out, I'd be most appreciative. I don't want to give up, but after spending over 6 months learning Python, and reading 15 Python books cover to cover, I don't understand why a simple 'import pandas' can't even work!! That's about as basic as it gets. Ugh. -- https://mail.python.org/mailman/listinfo/python-list
Re: very weird pandas behavior
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: > I downloaded pandas and put it in my python directory, then, at the C-prompt, > I ran this: > "pip install pandas" > > It looks like everything downloaded and installed fine. Great. > > Now, in Python Shell, I enter this: > import pandas as pd > > I get this error. > Traceback (most recent call last): > File "", line 1, in > import pandas as pd > ImportError: No module named pandas > > > Any idea what I'm doing wrong? Thanks Steven. I just tried what you recommended, and got this. >>> import sys >>> print(sys.version) 2.7 (r27:82525, Jul 4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] >>> print(sys.path) ['C:\\Python27\\Lib\\idlelib', 'C:\\Python27\\lib\\site-packages\\requests-2.4.3-py2.7.egg', 'C:\\Python27\\lib\\site-packages\\html-1.16-py2.7.egg', 'C:\\Python34', 'C:\\WINDOWS\\SYSTEM32\\python27.zip', 'C:\\Python27\\DLLs', 'C:\\Python27\\lib', 'C:\\Python27\\lib\\plat-win', 'C:\\Python27\\lib\\lib-tk', 'C:\\Python27', 'C:\\Python27\\lib\\site-packages'] >>> I also got this. Microsoft Windows [Version 6.3.9600] (c) 2013 Microsoft Corporation. All rights reserved. C:\Users\Ryan>python Python 2.7 (r27:82525, Jul 4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win 32 Type "help", "copyright", "credits" or "license" for more information. >>> python27 Traceback (most recent call last): File "", line 1, in NameError: name 'python27' is not defined >>> python33 Traceback (most recent call last): File "", line 1, in NameError: name 'python33' is not defined >>> python34 Traceback (most recent call last): File "", line 1, in NameError: name 'python34' is not defined >>> Any idea what's wrong? -- https://mail.python.org/mailman/listinfo/python-list
very weird pandas behavior
I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: "pip install pandas" It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import pandas as pd I get this error. Traceback (most recent call last): File "", line 1, in import pandas as pd ImportError: No module named pandas Any idea what I'm doing wrong? -- https://mail.python.org/mailman/listinfo/python-list
Re: import graphics library; causes error
On Sunday, November 16, 2014 3:39:45 PM UTC-5, ryguy7272 wrote:
> These libraries drive me totally nuts. Sorry, just had to get it out there.
> Anyway, I open the cmd window, and typed this: 'easy_install python
> graphics'. So, it starts up and runs/downloads the appropriate library from
> the web. I get confirmation (in the cmd window) that it finishes, then I try
> to run this script.
>
>
> # futval_graph2.py
> from graphics import *
> def main():
> # Introduction
> print "This program plots the growth of a 10-year investment."
> # Get principal and interest rate
> principal = input("Enter the initial principal: ")
> apr = input("Enter the annualized interest rate: ")
> # Create a graphics window with labels on left edge
> win = GraphWin("Investment Growth Chart", 640, 480)
> win.setBackground("white")
> win.setCoords(-1.75,-200, 11.5, 10400)
> Text(Point(-1, 0), '0.0K').draw(win)
> Text(Point(-1, 2500), '2.5K').draw(win)
> Text(Point(-1, 5000), '5.0K').draw(win)
> Text(Point(-1, 7500), '7.5k').draw(win)
> Text(Point(-1, 1), '10.0K').draw(win)
> # Draw bar for initial principal
> bar = Rectangle(Point(0, 0), Point(1, principal))
> bar.setFill("green")
> bar.setWidth(2)
> bar.draw(win)
> # Draw a bar for each subsequent year
> for year in range(1, 11):
> principal = principal * (1 + apr)
> bar = Rectangle(Point(year, 0), Point(year+1, principal))
> bar.setFill("green")
> bar.setWidth(2)
> bar.draw(win)
> raw_input("Press to quit.")
>
>
> After I hit F5, I get this:
> Traceback (most recent call last):
> File "C:\Users\Ryan\Desktop\Graphics_Test.py", line 2, in
> from graphics import *
> ImportError: No module named graphics
In what directory?
Well, that's a damn good question. I thought, by defailt, everything was
downloaded to this folder:
'C:\Python27\Lib\site-packages'
In there, I have all kinds of things like:
'setuptools-6.1.dist-info', 'pip-1.5.6.dist-info', etc.
All kinds of other things too.
It seems there is always a copy, so I cut/paste the folders named 'setuptools'
& 'pip' (always taking off the versions and identifiers and the like...). Then
I cut/paste everything into this folder:
'C:\Python27\Lib'
Is that how it's done or not? Honestly, for the life of me, I don't know why a
human being would have do do any of this, including using the cmd window, to
install anything in 2014. I can code in 10 different languages, not including
Python. Python is by far the most backwards type of technology that I can
think of. Using it is completely counter-productive. I can't take it serious.
I have plenty of tools in my toolbox. I'll keep learning Python, and keep
reading books, and keep using it...but strictly for fun. I would never use
this as a foundation for a mission critical business application.
Thanks everyone!
--
https://mail.python.org/mailman/listinfo/python-list
import graphics library; causes error
These libraries drive me totally nuts. Sorry, just had to get it out there.
Anyway, I open the cmd window, and typed this: 'easy_install python graphics'.
So, it starts up and runs/downloads the appropriate library from the web. I
get confirmation (in the cmd window) that it finishes, then I try to run this
script.
# futval_graph2.py
from graphics import *
def main():
# Introduction
print "This program plots the growth of a 10-year investment."
# Get principal and interest rate
principal = input("Enter the initial principal: ")
apr = input("Enter the annualized interest rate: ")
# Create a graphics window with labels on left edge
win = GraphWin("Investment Growth Chart", 640, 480)
win.setBackground("white")
win.setCoords(-1.75,-200, 11.5, 10400)
Text(Point(-1, 0), '0.0K').draw(win)
Text(Point(-1, 2500), '2.5K').draw(win)
Text(Point(-1, 5000), '5.0K').draw(win)
Text(Point(-1, 7500), '7.5k').draw(win)
Text(Point(-1, 1), '10.0K').draw(win)
# Draw bar for initial principal
bar = Rectangle(Point(0, 0), Point(1, principal))
bar.setFill("green")
bar.setWidth(2)
bar.draw(win)
# Draw a bar for each subsequent year
for year in range(1, 11):
principal = principal * (1 + apr)
bar = Rectangle(Point(year, 0), Point(year+1, principal))
bar.setFill("green")
bar.setWidth(2)
bar.draw(win)
raw_input("Press to quit.")
After I hit F5, I get this:
Traceback (most recent call last):
File "C:\Users\Ryan\Desktop\Graphics_Test.py", line 2, in
from graphics import *
ImportError: No module named graphics
--
https://mail.python.org/mailman/listinfo/python-list
import math error
When I type 'import math', it seems like my Python recognizes this library.
Great. When I try to run the following script, I get an error, which suggests
(to me) the math library is not working correctly.
Script:
import math
def main():
print "This program finds the real solutions to a quadratic"
print
a, b, c = input("Please enter the coefficients (a, b, c): ")
discRoot = math.sqrt(b * b - 4 * a * c)
root1 = (-b + discRoot) / (2 * a)
root2 = (-b - discRoot) / (2 * a)
print
print "The solutions are:", root1, root2
main()
Error:
Traceback (most recent call last):
File "C:\Users\Ryan\Desktop\QuadraticEquation.py", line 11, in
main()
File "C:\Users\Ryan\Desktop\QuadraticEquation.py", line 5, in main
a, b, c = input("Please enter the coefficients (a, b, c): ")
File "", line 1
(1,1,2):
^
SyntaxError: unexpected EOF while parsing
The last line is line 11. It seems like I need this, but that's what's causing
the error. Does anyone here have any idea what is wrong here?
Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
Meetup in NYC?
Are there any Python professionals in or around NYC who can meetup for an hour or two to help me with a few things? I've been trying to run various Python scripts for a few months now, and I'm not having any success with this stuff at all. Basically, everything built into Python works perfectly fine for me. However, whenever I try to run something like numpy or beautifulsoup, or anything like that, I have all kinds of errors. If anyone could meet for an hour or two, in a Starbucks, and go through a few examples of Python code, it would be great!! I can take you out for a few beers afterwards. -- https://mail.python.org/mailman/listinfo/python-list
Re: No Error; No Output...Nothing
On Tuesday, October 21, 2014 5:44:33 PM UTC-4, ryguy7272 wrote:
> Hey everyone, I'm trying to run this code.
>
>
>
> import os
>
> import pickle
>
> #import urllib2
>
> from urllib.request import urlopen
>
> #import cookielib
>
> import http.cookiejar
>
> import re
>
> import time
>
> import numpy as np
>
> #import pylab as pl
>
>
>
> # symbol - USDEUR=X - problem that the server sometimes returns 0.0
>
> def getSpotPrice(symbol):
>
> numberOfAttempts = 0
>
> while numberOfAttempts < 10:
>
> url =
> 'http://download.finance.yahoo.com/d/quotes.csv?s='+symbol+'&f=l1&e=.cs'
>
> fxrate_pure = urllib2.urlopen(url).read()
>
> fxrate = fxrate_pure.strip()
>
> if fxrate != "0.00":
>
> return fxrate
>
> else:
>
> numberOfAttempts += 1
>
> time.sleep(1)
>
> raise Exception("Unable to obtain market data from Yahoo! ... wrong
> ticker???")
>
>
>
> # symbol = the yahoo ticker; the expected tickers of the components contain
> alphanumerical characters or dot or hyphen; if the yahoo format changes,
> nothing is returned
>
> def getConstituentsOfAnIndexFromYahoo(symbol):
>
> url = 'http://finance.yahoo.com/q/cp?s=%s' % symbol
>
> p = re.compile(' href=\"/q\?s=([A-Z0-9\.\-]*)\">')
>
> components = []
>
> pageIndex = 0
>
> finished = False
>
> while not finished:
>
> if pageIndex == 0:
>
> actualUrl = url
>
> else:
>
> actualUrl = url + "&c=" + str(pageIndex)
>
> pageResults = p.findall(urllib2.urlopen(actualUrl).read())
>
> if len(pageResults) == 0:
>
> finished = True
>
> else:
>
> components.extend(pageResults)
>
> pageIndex+=1
>
> return components
>
>
>
> # prices = data[:,6] or prices = data[:, title.index("Adj Close")],
> pl.num2date(data[:,1]) back dates
>
> # syntax
> http://ichart.yahoo.com/table.csv?s={Yahoo.Symbol.[isin]}&a={Von.M-1}&b={Von.T}&c={Von.J}&d={Bis.M}&e={Bis.T}&f={Bis.
> J}&g=d&y=0&z=jdsu&ignore=.csv
>
> def getNumpyHistoricalTimeseries(symbol,fromDate, toDate):
>
> f = urllib2.urlopen('http://ichart.yahoo.com/table.csv?a='+
> str(fromDate.month -1) +'&c='+ str(fromDate.year) +'&b=' + str(fromDate.day)
> + '&e='+ str(toDate.day) + '&d='+ str(toDate.month-1) +'&g=d&f=' +
> str(toDate.year) + '&s=' + symbol + '&ignore=.csv')
>
> header = f.readline().strip().split(",")
>
> #return np.loadtxt(f, dtype=np.float, delimiter=",", converters={0:
> pl.datestr2num})
>
>
>
> I commented out the import pylab as pl because I couldn't get the
> matplotlib.pylab import working. So, anyway, I hit F5, and it seems to run,
> but it doesn't really do anything. Isn't this either supposed to be
> downloading data from the web, or throwing an error so I can troubleshoot,
> and try to figure out what's going on? It's hard to troubleshoot, when you
> don't get any error. Does this work for others?
>
>
>
> Thanks.
OK. Thanks everyone!
--
https://mail.python.org/mailman/listinfo/python-list
No Error; No Output...Nothing
Hey everyone, I'm trying to run this code.
import os
import pickle
#import urllib2
from urllib.request import urlopen
#import cookielib
import http.cookiejar
import re
import time
import numpy as np
#import pylab as pl
# symbol - USDEUR=X - problem that the server sometimes returns 0.0
def getSpotPrice(symbol):
numberOfAttempts = 0
while numberOfAttempts < 10:
url =
'http://download.finance.yahoo.com/d/quotes.csv?s='+symbol+'&f=l1&e=.cs'
fxrate_pure = urllib2.urlopen(url).read()
fxrate = fxrate_pure.strip()
if fxrate != "0.00":
return fxrate
else:
numberOfAttempts += 1
time.sleep(1)
raise Exception("Unable to obtain market data from Yahoo! ... wrong
ticker???")
# symbol = the yahoo ticker; the expected tickers of the components contain
alphanumerical characters or dot or hyphen; if the yahoo format changes,
nothing is returned
def getConstituentsOfAnIndexFromYahoo(symbol):
url = 'http://finance.yahoo.com/q/cp?s=%s' % symbol
p = re.compile('')
components = []
pageIndex = 0
finished = False
while not finished:
if pageIndex == 0:
actualUrl = url
else:
actualUrl = url + "&c=" + str(pageIndex)
pageResults = p.findall(urllib2.urlopen(actualUrl).read())
if len(pageResults) == 0:
finished = True
else:
components.extend(pageResults)
pageIndex+=1
return components
# prices = data[:,6] or prices = data[:, title.index("Adj Close")],
pl.num2date(data[:,1]) back dates
# syntax
http://ichart.yahoo.com/table.csv?s={Yahoo.Symbol.[isin]}&a={Von.M-1}&b={Von.T}&c={Von.J}&d={Bis.M}&e={Bis.T}&f={Bis.
J}&g=d&y=0&z=jdsu&ignore=.csv
def getNumpyHistoricalTimeseries(symbol,fromDate, toDate):
f = urllib2.urlopen('http://ichart.yahoo.com/table.csv?a='+
str(fromDate.month -1) +'&c='+ str(fromDate.year) +'&b=' + str(fromDate.day) +
'&e='+ str(toDate.day) + '&d='+ str(toDate.month-1) +'&g=d&f=' +
str(toDate.year) + '&s=' + symbol + '&ignore=.csv')
header = f.readline().strip().split(",")
#return np.loadtxt(f, dtype=np.float, delimiter=",", converters={0:
pl.datestr2num})
I commented out the import pylab as pl because I couldn't get the
matplotlib.pylab import working. So, anyway, I hit F5, and it seems to run,
but it doesn't really do anything. Isn't this either supposed to be
downloading data from the web, or throwing an error so I can troubleshoot, and
try to figure out what's going on? It's hard to troubleshoot, when you don't
get any error. Does this work for others?
Thanks.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Quick Question About Setting Up Pytz
On Saturday, October 18, 2014 3:55:02 PM UTC-4, ryguy7272 wrote: > I downloaded PYTZ and put it here. > > C:\Python27\pytz > > > > Now, in the cmd window, I typed this: > > C:\Python27\pytz\setup.py > > > > A text file opens and nothing else happens. I thought it was supposed to > install the PYTZ library. > > > > What am I doing wrong? Thanks Mark. That makes sense. I moved the folder to the desktop and I just tried that: C:\Users\Ryan\Desktop\pytz>setup.py install So, when I run it, the setup.py text file opens. Nothing runs; nothing installs. This makes no sense whatsoever. If I tell a program to install something, it should install something. It should not pop open a text file. I'll probably give it until the end of the year, and start learning Chinese. There's other things I want to do with my time. I know 10 programming languages. I thought it would be fun to learn Python, but after 2 months, I still can't run a single line of code. This language makes no sense whatsoever, and it never does anything that you tell it to do. -- https://mail.python.org/mailman/listinfo/python-list
Question about PANDAS
I'm trying to install Pandas. I went to this link. https://pypi.python.org/pypi/pandas/0.14.1/#downloads I downloaded this: pandas-0.14.1.win32-py2.7.exe (md5) I have Python27 installed. So, I run the executable and re-run my Python script and I get the same error as before. Traceback (most recent call last): File "C:/Python27/stock_data.py", line 3, in import pandas as pd ImportError: No module named pandas I thought I just installed it! Isn't that what the executable is for? It seems like 100% of my errors are with uninstalled libraries. I don't understand why there are so, so, so many dependencies running Python. Also, I don't understand why something isn't installed, right after I just installed it. Can someone please explain the logic to me? Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Quick Question About Setting Up Pytz
I downloaded PYTZ and put it here. C:\Python27\pytz Now, in the cmd window, I typed this: C:\Python27\pytz\setup.py A text file opens and nothing else happens. I thought it was supposed to install the PYTZ library. What am I doing wrong? -- https://mail.python.org/mailman/listinfo/python-list
Re: Import Doesn't Import
On Wednesday, October 15, 2014 8:40:40 PM UTC-4, ryguy7272 wrote:
> So sorry everyone. I've posted here several times today. This is VERY
> frustrating.
>
>
>
> So, I'm reading this link.
>
> https://docs.python.org/2/howto/urllib2.html
>
>
>
>
>
> Fetching URLs
>
> The simplest way to use urllib2 is as follows:
>
> import urllib2
>
> response = urllib2.urlopen('http://python.org/')
>
> html = response.read()
>
>
>
>
>
> So, I fire up Python, and create a new file and name it and hit F5.
>
>
>
> All I have is thins in the file:
>
> import urllib2
>
> response = urllib2.urlopen('http://python.org/')
>
> html = response.read()
>
>
>
> It immediately produces this error:
>
> Traceback (most recent call last):
>
> File "C:/Python34/import_web_data.py", line 1, in
>
> import urllib2
>
> ImportError: No module named 'urllib2'
>
>
>
>
>
> ImportError: No module named 'urllib2'
>
> I'm telling Python to import because it doesn't exist and it throws an error.
> I don't get it; I just don't get it. If I'm working with R, I can import
> thousands of libraries with no errors whatsoever. With Python, I just get
> thousands of errors with nothing working whatsoever. I totally don't
> understand this language. Import means import. Right. WTF!
Either this is the most brilliant thing ever invented, or it's the biggest
piece of shit ever invented. I just can't tell. All I know for sure, is that
it doesn't do ANYTHING that I tell it to do.
If there is another way of running Python code, like using Visual Studio, or a
text file, or some such thing, let me know, and I'll do it. It seems like you
should use Python to run Python, but my Python doesn't do anything at all.
Yesterday I uninstalled Python 2.7, because that wouldn't do anything either.
I'm just about ready to uninstall 3.4.
--
https://mail.python.org/mailman/listinfo/python-list
Import Doesn't Import
So sorry everyone. I've posted here several times today. This is VERY
frustrating.
So, I'm reading this link.
https://docs.python.org/2/howto/urllib2.html
Fetching URLs
The simplest way to use urllib2 is as follows:
import urllib2
response = urllib2.urlopen('http://python.org/')
html = response.read()
So, I fire up Python, and create a new file and name it and hit F5.
All I have is thins in the file:
import urllib2
response = urllib2.urlopen('http://python.org/')
html = response.read()
It immediately produces this error:
Traceback (most recent call last):
File "C:/Python34/import_web_data.py", line 1, in
import urllib2
ImportError: No module named 'urllib2'
ImportError: No module named 'urllib2'
I'm telling Python to import because it doesn't exist and it throws an error.
I don't get it; I just don't get it. If I'm working with R, I can import
thousands of libraries with no errors whatsoever. With Python, I just get
thousands of errors with nothing working whatsoever. I totally don't
understand this language. Import means import. Right. WTF!
--
https://mail.python.org/mailman/listinfo/python-list
Question About Running Python code
I'm trying to run this script (using IDLE 3.4)
#!/usr/bin/env python
import urllib2
import pytz
import pandas as pd
from bs4 import BeautifulSoup
from datetime import datetime
from pandas.io.data import DataReader
SITE = "http://en.wikipedia.org/wiki/List_of_S%26P_500_companies";
START = datetime(1900, 1, 1, 0, 0, 0, 0, pytz.utc)
END = datetime.today().utcnow()
def scrape_list(site):
hdr = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(site, headers=hdr)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page)
table = soup.find('table', {'class': 'wikitable sortable'})
sector_tickers = dict()
for row in table.findAll('tr'):
col = row.findAll('td')
if len(col) > 0:
sector = str(col[3].string.strip()).lower().replace(' ', '_')
ticker = str(col[0].string.strip())
if sector not in sector_tickers:
sector_tickers[sector] = list()
sector_tickers[sector].append(ticker)
return sector_tickers
def download_ohlc(sector_tickers, start, end):
sector_ohlc = {}
for sector, tickers in sector_tickers.iteritems():
print 'Downloading data from Yahoo for %s sector' % sector
data = DataReader(tickers, 'yahoo', start, end)
for item in ['Open', 'High', 'Low']:
data[item] = data[item] * data['Adj Close'] / data['Close']
data.rename(items={'Open': 'open', 'High': 'high', 'Low': 'low',
'Adj Close': 'close', 'Volume': 'volume'},
inplace=True)
data.drop(['Close'], inplace=True)
sector_ohlc[sector] = data
print 'Finished downloading data'
return sector_ohlc
def store_HDF5(sector_ohlc, path):
with pd.get_store(path) as store:
for sector, ohlc in sector_ohlc.iteritems():
store[sector] = ohlc
def get_snp500():
sector_tickers = scrape_list(SITE)
sector_ohlc = download_ohlc(sector_tickers, START, END)
store_HDF5(sector_ohlc, 'snp500.h5')
if __name__ == '__main__':
get_snp500()
I got it from this link.
http://www.thealgoengineer.com/2014/download_sp500_data/
I'm just about going insane here. I've been doing all kinds of computer
programming for 11 years, and I know 10 languages. I'm trying to learn Python
now, but this makes no sense to me.
I would be most appreciative if someone could respond to a few questions.
The error that I get is this.
'invalid syntax'
The second single quote in this line is highlighted pink.
print 'Downloading data from Yahoo for %s sector' % sector
#1) That's very bizarre to mix single quotes and double quotes in a single
language. Does Python actually mix single quotes and double quotes?
#2) In the Python 3.4 Shell window, I turn the debugger on by clicking
'Debugger'. Then I run the file I just created; it's called 'stock.py'. I get
the error immediately, and I can't debug anything so I can't tell what's going
on. This is very frustrating. All the controls in the debugger are greyed
out. What's up with the debugger?
#3) My final question is this? How do I get this code running? It seems like
there is a problem with a single quote, which is just silly. I can't get the
debugger working, so I can't tell what's going on. The only thins I know, or I
think I know, is that the proper libraries seem to be installed, so that's one
thing that's working.
I'd really appreciate it if someone could please answer my questions and help
me get this straightened out, so I can have some fun with Python. So far, I've
spent 2 months reading 4 books, and trying all kinds of sample code...and
almost every single thing fails and I have no idea why.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to debug Python IDLE?
On Wednesday, October 15, 2014 3:10:05 PM UTC-4, ryguy7272 wrote: > I'm wondering how to debug code in IDLE Python 3.4. I found this. > > http://www.cs.uky.edu/~paulp/CS115F11/notes/debug/debug.html > > > > That looks pretty helpful, but mine is nothing like that. All my controls > are greyed out. The debugger does basically...nothing. All I get is > messages like this. > > > > [DEBUG ON] > > >>> > > [DEBUG OFF] > > >>> > > [DEBUG ON] > > >>> > > > > It doesn't debug anything at all. > > > > Any idea what's wrong? > > Thanks. Yes. exactly. So why is it throwing an error and how can I truly debug it? Normally, you can step through code line by line, but I don't see any way to do this using Python. Man, I've been in all kinds of development roles, for over 10 years, using all kinds of languages, and I've never worked with anything as difficult as Python. It seems so simple, but actually nothing works for me. I've read 4 books on Python and I've been working on small snippets of code, for 2 months now. Almost nothing at all has worked for me. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to debug Python IDLE?
On Wednesday, October 15, 2014 3:23:22 PM UTC-4, Terry Reedy wrote: > On 10/15/2014 3:09 PM, ryguy7272 wrote: > > > I'm wondering how to debug code in IDLE Python 3.4. I found this. > > > > Use 3.4.2, which has an important bugfix for debugger. > > > > > http://www.cs.uky.edu/~paulp/CS115F11/notes/debug/debug.html > > > > > > That looks pretty helpful, > > > > > but mine is nothing like that. All my controls are greyed out. > > > > That is normal when you first turn debugger on. You have to load a file > > in the editor and then run it (F5). On the site above, the file is > > volume.py. The the debugger is 'activated'. > > > > -- > > Terry Jan Reedy Oh, I didn't know that's how it works. ok. Makes sense. Something is still wrong though. I have a file named 'test_script.py'. Here's the code: def showMaxFactor(num): count = num / 2 while count > 1: if num % count == 0: print 'largest factor of %d is %d' % \ (num, count) break count -= 1 else: print num, "is prime" for eachNum in range(10, 21): showMaxFactor(eachNum) With the debugger ON, I hit F5 and get this. 'Expected an indent block' I wouldn't say that's debugging anything. I can't tell what line is throwing the error, or how to fix it. Any thoughts? -- https://mail.python.org/mailman/listinfo/python-list
How to debug Python IDLE?
I'm wondering how to debug code in IDLE Python 3.4. I found this. http://www.cs.uky.edu/~paulp/CS115F11/notes/debug/debug.html That looks pretty helpful, but mine is nothing like that. All my controls are greyed out. The debugger does basically...nothing. All I get is messages like this. [DEBUG ON] >>> [DEBUG OFF] >>> [DEBUG ON] >>> It doesn't debug anything at all. Any idea what's wrong? Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Is there an easy way to control indents in Python
I'm just learning Python. It seems like indents are EXTREMELY important. I guess, since there are no brackets, everything is controlled by indents. Well, I'm reading a couple books on Python now, and in almost all of the examples they don't have proper indents, so when I copy/paste the code (from the PDF to the IDE) the indents are totally screwed up. I'm thinking that there should be some control, or setting, for this. I hope. :) I have PyCharm 3.4 and Python 3.4. -- https://mail.python.org/mailman/listinfo/python-list
Re: I'm looking to start a team of developers, quants, and financial experts, to setup and manage an auto-trading-money-making-machine
On Tuesday, October 14, 2014 9:38:55 AM UTC-4, Johann Hibschman wrote:
> Marko Rauhamaa writes:
>
>
>
> > ryguy7272 :
>
> >
>
> >> I'm looking to start a team of developers, quants, and financial
>
> >> experts, to setup and manage an auto-trading-money-making-machine
>
> >
>
> > This has already been done: http://en.wikipedia.org/wiki/Sampo
>
>
>
> And mocked by MST3K ("sampo means flavor!"):
>
>
>
> https://www.youtube.com/watch?v=cdfUkrbNvwA
>
>
>
> -Johann (whose cousins are all Mattinens and Nikkanens)
Good stuff! Very funny!!
--
https://mail.python.org/mailman/listinfo/python-list
I'm looking to start a team of developers, quants, and financial experts, to setup and manage an auto-trading-money-making-machine
I'm looking to start a team of developers, quants, and financial experts, to setup and manage an auto-trading-money-making-machine #1) Setup a VM; must be a Windows machine (maybe Azure) #2) Load & configure the software (Matlab, Python, R, Excel, and SQL Server) #3) Develop and test code (Matlab or Pythonmost likely) #4) Hook into trading tool for order execution (system to be determined; depends on API) #5) Setup a corporate business structure (this is virtually done, we just need to flip a switch) #5) Shop around for clients (if we have a real money-making machine, this should be super-easy) Just so you know, I've done this kind of thing before, using Excel and VBA, as well as an execution program called Sterling Trader. It was quite profitable, and typically had less than 1 down day in a 22-trading-day month. The system was profitable about 95% of the time. However, I don't want to use Excel for this project; I want this to be a real system. I think we can use Matlab, or Python. If this is project turns out to be very profitable, I know we can raise capital very quickly and very easily. At the beginning, I believe it will take a fair amount of work, but if we put in the effort, we can have a system that constantly monitors the equity markets during trading hours, finds the best profit opportunities, according to the trading strategies that we deploy, and once it is setup and running, we really won't have to do a whole lot. Ideally, once you turn it on, the whole process will require almost no intervention. I know this can be done; many banks have groups that do exactly what I'm proposing here. The top hedge funds do this too. In conclusion, I think the trading strategies that we choose to employ should be easy to setup and test (a well-defined Google search will reveal countless trading strategies). I think getting the data should be pretty easy as well (we can load all kinds of indices into the tool). I think the hard part will be developing the automation processes; the tool will have to follow MANY rules and run all by itself. Finally, as neither Matlab nor Python are really execution tools, we'll have to connect to some type of system that allows us to send trade orders. This may, or may not, present somewhat of a challenge. As an alternative, if we decide we don't want to manage money for other people, we could setup the business as a subscription service, and charge users, let's say $25k/year or whatever, and let people run simulations in our hosted environment, and they can manage money themselves...we simply provide all kinds of analytic tools for them to do what they want to do. Hopefully everyone is close to New York City or Washington DC, or somewhere close, like Boston or Philadelphia. -- https://mail.python.org/mailman/listinfo/python-list
problem with pytz
I just tried running the code from here. http://www.thealgoengineer.com/2014/download_sp500_data/ I got this error. Traceback (most recent call last): File "C:/Python27/stock_data.py", line 4, in import pytz ImportError: No module named pytz So, I got here and download the files. https://pypi.python.org/pypi/pytz/#downloads I run this: python setup.py install I get this error. python: can't open file 'setup.py': [Errno 2] No such file or directory I tried this: easy_install --upgrade pytz Now, that seems like it downloaded and installed. So, I re-run the program, and I get the same error I had initially. Traceback (most recent call last): File "C:/Python27/stock_data.py", line 4, in import pytz ImportError: No module named pytz I don't get it. I go to the path 'C:\Python27\pytz' and the folder is right there! It's right there! It's a little frustrating because all my problems seem to be exactly the same, which is running scripts that should run easily...bu nothing actually runs. What else do I need to do to make this thing work? -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install and run a script?
On Sunday, October 12, 2014 4:18:19 PM UTC-4, ryguy7272 wrote: > I'm an absolute noob to Python, although I have been programming in several > other languages for over 10 years. > > > > I'm trying to install and run some scripts, and I'm not having much success. > > > > I followed the steps at this site. > > https://docs.python.org/2/install/ > > > > I have Python 2.7.6 Shell. > > > > If I type this: > > python setup.py install > > > > I get this: > > SyntaxError: invalid syntax > > > > > > If I try this: > > setup.py beautifulsoup > > > > I get this: > > SyntaxError: invalid syntax > > > > > > If I got to File > Open . . . . navigate to a folder and open the setup.py > file and hit F5 . . . . it looks like it runs, but I can't tell what it does > for sure, and then nothing works after that.For instance, I downloaded > something called 'Flask'. I put that folder in my Python27 folder, and typed > 'pip install Flask' and I keep getting errors. > > > > Any idea how to run a simple program??? Ah!!! I didn't know I needed to run it from the command prompt! Ok, not it makes sense, and everything works. Thanks to all! -- https://mail.python.org/mailman/listinfo/python-list
