Thanks, Tom Sorry - I tried replying from my android phone. The msg doesn’t seem to have been sent, so I’m trying again. Apologies if a similar msg appears!
Anyway, I did have a go at getting your script working for “historical” data on London Stock Exchange quotes. Naturally, I prefer ddmmyyyy dates, being English, so I varied epochtime a bit, using “toDayNo” . It appeared to be necessary to multiply the result by 10 . It wasn’t clear to me when “yesterday’s “ historical data becomes available, in gmt or bst, naturally. Anyway, after some false starts, the script yielded some day-old quotes on London symbols, which is pretty useful; I’ll do a little digging into current prices, following your link, but won’t fret too much if I get lost. Thanks again, Mike Sent from my iPad On 30/11/2019 05:48, Thomas McGuire wrote: The realtime quote gets updated at a different address. That page looks like some sort of JSON interaction is taking place to give the realtime update in the stock price. So for apple stock the realtime page would be: https://finance.yahoo.com/quote/AAPL?p=AAPL&.tsrc=fin-srch <https://finance.yahoo.com/quote/AAPL?p=AAPL&.tsrc=fin-srch> I glanced at the source code for the page and good luck finding the JSON interaction. Some simple searches just jump to a mess of HTML. There is however a Crumbstore with a crumb value placed that can be accessed the same way my code for the Historical Prices does. So if you had some working code for Yahoo in the past you may be able to update the URLs and make a “crumb” access to get the data. Tom McGuire > On Nov 26, 2019, at 6:14 PM, 'Mike Day' via Programming > <programm...@jsoftware.com> wrote: > Might I ask once more whether there’s a get-around for capturing real-time > or, failing that, close of play quotes from Yahoo? > Thanks, > Mike > > Sent from my iPad > >> On 26 Nov 2019, at 22:11, Gilles Kirouac <g1...@myriade.ca> wrote: >> >> Thomas >> To avoid the dependency on Linux, without changing your syntax, I >> redefined epochtime as: >> >> NB.create a time stamp [of type character] >> NB. epochtime'1/1/2019' >> NB.1546318800 >> epochtime =: 3 : 0 >> ":18000+86400*62091-~todayno _1|.".;._1 '/',y NB.normalized yr mn dy >> ) >> >> and I removed the two curtails in (using a fixed width font) >> >> qstr =. '?period1=',(}:epochtime d1),'&period2=',(}:epochtime >> d2),'&interval=1d&events=history&crumb=',crumb >> .....................^^...........................^^ >> >> todayno is in the standard library. >> >> Thanks! >> >> ~ Gilles >>> Le 2019-11-24 à 23:19, Thomas McGuire a écrit : >>> OK so here is my final code cleaned up and now working due to the double >>> quote issue (see second to last line of code): >>> >>> NB. Navigating yahoo.com to programmatically get historical stock prices >>> NB. >>> require 'web/gethttp' >>> require 'regex' >>> >>> NB. use the linux date command to create a linux time stamp >>> epochtime =: 3 : 0 >>> 2!:0 'date -jf ''%m/%d/%Y %H:%M:%S %p'' ''',y,' 05:00:00 PM'' ''+%s''' >>> ) >>> >>> NB. precision functions >>> ppq =: 9 !: 10 NB. print-precision query >>> pps =: 9 !: 11 NB. print-precision set >>> NB. I set the precision to 16 to ensure full printing of the linux >>> timestamps >>> >>> NB. Conversion of \u00xx escape sequences >>> HEX=:16#.'0123456789abcdef'i.] >>> >>> xutf =: 3 : 0 >>> u: HEX tolower 2 }. y >>> ) >>> >>> crumbstr =: '"CrumbStore":{"crumb":"' >>> NB. the crumb is on the page with the link to downloading the historical >>> NB. data. If you call the correct first page you only need to search >>> NB. for the above crumbstr there will be only one. >>> getcrumb =: 3 : 0 >>> NB. find the start index and end index of the crumb >>> sidx =. (#crumbstr)+({: I. crumbstr E. y) >>> sstr =. (sidx + i. 30){y >>> eidx =. {. I. '"' E. sstr >>> >>> NB. using rxapply convert all \u00xx unicode escape sequences >>> crumb =. '(\\u[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F])' xutf rxapply >>> (i.eidx){sstr >>> ) >>> >>> financeURL =: 'https://finance.yahoo.com/quote/' NB. AAPL/history?p=AAPL' >>> histURL =: 'https://query1.finance.yahoo.com/v7/finance/download/' >>> NB. the histURL needs to have a ticker symbol followed by: >>> NB. ?period1=<unixts for p1>&period2=<unixts for >>> p2&interval=1d&events=history&crumb=<crumbval> >>> NB. >>> NB. here is a full fledged quote request from the website itself for Apple >>> Computer >>> NB. >>> https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1543024670&period2=1574560670&interval=1d&events=history&crumb=jZO816Y7CSK >>> >>> gethistorical=: 3 : 0 >>> 'symbol d1 d2' =. y >>> >>> NB. Create start URL for the start page with the crumb to get historical >>> download >>> NB. a BASH implementation uses the following format: >>> NB. sURL =. financeURL,symbol,'/?p=',symbol >>> >>> NB. But the link to the download of historical prices is: >>> sURL =. financeURL,symbol,'/history?p=',symbol >>> >>> NB. Get the response using gethttp. -c cookie.txt will open a cookie file >>> res =. '-s -c cookie.txt' gethttp sURL >>> >>> crumb =. getcrumb res >>> >>> qstr =. '?period1=',(}:epochtime d1),'&period2=',(}:epochtime >>> d2),'&interval=1d&events=history&crumb=',crumb >>> URL=. histURL,symbol,qstr >>> >>> NB. turns out that to get a file download you need to double quote the URL >>> NB. There is a built in function for that in J >>> res2 =. '-s -b cookie.txt ' gethttp dquote URL >>> res2 >>> ) >>> >>> ---------------------------------------------------------------------- >>> For information about J forums see http://www.jsoftware.com/forums.htm >> >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
