Thomas
To avoid the dependency on Linux, without changing your syntax, I
redefined epochtime as:
NB.create a time stamp [of type character]
NB. epochtime'1/1/2019'
NB.1546318800
epochtime =: 3 : 0
":18000+86400*62091-~todayno _1|.".;._1 '/',y NB.normalized yr mn dy
)
and I removed the two curtails in (using a fixed width font)
qstr =. '?period1=',(}:epochtime d1),'&period2=',(}:epochtime
d2),'&interval=1d&events=history&crumb=',crumb
.....................^^...........................^^
todayno is in the standard library.
Thanks!
~ Gilles
Le 2019-11-24 à 23:19, Thomas McGuire a écrit :
> OK so here is my final code cleaned up and now working due to the double
> quote issue (see second to last line of code):
>
> NB. Navigating yahoo.com to programmatically get historical stock prices
> NB.
> require 'web/gethttp'
> require 'regex'
>
> NB. use the linux date command to create a linux time stamp
> epochtime =: 3 : 0
> 2!:0 'date -jf ''%m/%d/%Y %H:%M:%S %p'' ''',y,' 05:00:00 PM'' ''+%s'''
> )
>
> NB. precision functions
> ppq =: 9 !: 10 NB. print-precision query
> pps =: 9 !: 11 NB. print-precision set
> NB. I set the precision to 16 to ensure full printing of the linux timestamps
>
> NB. Conversion of \u00xx escape sequences
> HEX=:16#.'0123456789abcdef'i.]
>
> xutf =: 3 : 0
> u: HEX tolower 2 }. y
> )
>
> crumbstr =: '"CrumbStore":{"crumb":"'
> NB. the crumb is on the page with the link to downloading the historical
> NB. data. If you call the correct first page you only need to search
> NB. for the above crumbstr there will be only one.
> getcrumb =: 3 : 0
> NB. find the start index and end index of the crumb
> sidx =. (#crumbstr)+({: I. crumbstr E. y)
> sstr =. (sidx + i. 30){y
> eidx =. {. I. '"' E. sstr
>
> NB. using rxapply convert all \u00xx unicode escape sequences
> crumb =. '(\\u[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F])' xutf rxapply
> (i.eidx){sstr
> )
>
> financeURL =: 'https://finance.yahoo.com/quote/' NB. AAPL/history?p=AAPL'
> histURL =: 'https://query1.finance.yahoo.com/v7/finance/download/'
> NB. the histURL needs to have a ticker symbol followed by:
> NB. ?period1=<unixts for p1>&period2=<unixts for
> p2&interval=1d&events=history&crumb=<crumbval>
> NB.
> NB. here is a full fledged quote request from the website itself for Apple
> Computer
> NB.
> https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1543024670&period2=1574560670&interval=1d&events=history&crumb=jZO816Y7CSK
>
> gethistorical=: 3 : 0
> 'symbol d1 d2' =. y
>
> NB. Create start URL for the start page with the crumb to get historical
> download
> NB. a BASH implementation uses the following format:
> NB. sURL =. financeURL,symbol,'/?p=',symbol
>
> NB. But the link to the download of historical prices is:
> sURL =. financeURL,symbol,'/history?p=',symbol
>
> NB. Get the response using gethttp. -c cookie.txt will open a cookie file
> res =. '-s -c cookie.txt' gethttp sURL
>
> crumb =. getcrumb res
>
> qstr =. '?period1=',(}:epochtime d1),'&period2=',(}:epochtime
> d2),'&interval=1d&events=history&crumb=',crumb
> URL=. histURL,symbol,qstr
>
> NB. turns out that to get a file download you need to double quote the URL
> NB. There is a built in function for that in J
> res2 =. '-s -b cookie.txt ' gethttp dquote URL
> res2
> )
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
