Interesting... I don’t wish to waste Jprogramming bandwidth with non-J issues,
but perhaps you won’t mind my just asking whether there’s a similar get-round
for current, or at least close of play daily data?
I gave up trying to keep my J script using gethttp on yahoo prices working when
they pulled the plug a couple of years ago.
Thanks,
Mike
Sent from my iPad
> On 25 Nov 2019, at 04:19, Thomas McGuire <tmcguir...@gmail.com> wrote:
>
> OK so here is my final code cleaned up and now working due to the double
> quote issue (see second to last line of code):
>
> NB. Navigating yahoo.com to programmatically get historical stock prices
> NB.
> require 'web/gethttp'
> require 'regex'
>
> NB. use the linux date command to create a linux time stamp
> epochtime =: 3 : 0
> 2!:0 'date -jf ''%m/%d/%Y %H:%M:%S %p'' ''',y,' 05:00:00 PM'' ''+%s'''
> )
>
> NB. precision functions
> ppq =: 9 !: 10 NB. print-precision query
> pps =: 9 !: 11 NB. print-precision set
> NB. I set the precision to 16 to ensure full printing of the linux timestamps
>
> NB. Conversion of \u00xx escape sequences
> HEX=:16#.'0123456789abcdef'i.]
>
> xutf =: 3 : 0
> u: HEX tolower 2 }. y
> )
>
> crumbstr =: '"CrumbStore":{"crumb":"'
> NB. the crumb is on the page with the link to downloading the historical
> NB. data. If you call the correct first page you only need to search
> NB. for the above crumbstr there will be only one.
> getcrumb =: 3 : 0
> NB. find the start index and end index of the crumb
> sidx =. (#crumbstr)+({: I. crumbstr E. y)
> sstr =. (sidx + i. 30){y
> eidx =. {. I. '"' E. sstr
>
> NB. using rxapply convert all \u00xx unicode escape sequences
> crumb =. '(\\u[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F])' xutf rxapply
> (i.eidx){sstr
> )
>
> financeURL =: 'https://finance.yahoo.com/quote/' NB. AAPL/history?p=AAPL'
> histURL =: 'https://query1.finance.yahoo.com/v7/finance/download/'
> NB. the histURL needs to have a ticker symbol followed by:
> NB. ?period1=<unixts for p1>&period2=<unixts for
> p2&interval=1d&events=history&crumb=<crumbval>
> NB.
> NB. here is a full fledged quote request from the website itself for Apple
> Computer
> NB.
> https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1543024670&period2=1574560670&interval=1d&events=history&crumb=jZO816Y7CSK
>
> gethistorical=: 3 : 0
> 'symbol d1 d2' =. y
>
> NB. Create start URL for the start page with the crumb to get historical
> download
> NB. a BASH implementation uses the following format:
> NB. sURL =. financeURL,symbol,'/?p=',symbol
>
> NB. But the link to the download of historical prices is:
> sURL =. financeURL,symbol,'/history?p=',symbol
>
> NB. Get the response using gethttp. -c cookie.txt will open a cookie file
> res =. '-s -c cookie.txt' gethttp sURL
>
> crumb =. getcrumb res
>
> qstr =. '?period1=',(}:epochtime d1),'&period2=',(}:epochtime
> d2),'&interval=1d&events=history&crumb=',crumb
> URL=. histURL,symbol,qstr
>
> NB. turns out that to get a file download you need to double quote the URL
> NB. There is a built in function for that in J
> res2 =. '-s -b cookie.txt ' gethttp dquote URL
> res2
> )
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
