[Jchat] looking at recent stock market volatility with J
I recently invested in an S&P index fund - great timing, I know, so I figured I'd let J help calm my emotional response to the recent market volatility. The questions on my mind are: 1. Is this level of daily volatility extreme? -- restated as how common are 2% swings in daily close price (SOLVED) 2. Is it uncommon to have this much volatility over a two week period? -- restated as how common are streaks of 2% change in price across multiple weeks (SOLVED) 3. How long does it take to recover after a period over period change of -5% (SOLVED) I set out to answer these questions: Step 1: Let's get some data require 'web/gethttp' NB. beware linebreak csv=: gethttp ' http://real-chart.finance.yahoo.com/table.csv?s=%5EGSPC&d=8&e=1&f=2015&g=d&a=0&b=3&c=1950&ignore=.csv ' arr=: > ',' cut each LF cut csv NB. closing price is 4th column close =: ". every 4{"1 }. arr QUESTION 1 - Daily volatility pctChange =: }: -&1@% }. NB. 708 days greater than 2% change from previous close (+/ 0.02 < | pctChange close) 708 NB. 4% of the time since 1950 (+/ 0.02 < | pctChange close) % (# close) 0.042852 NB. 354 times since 2000 ]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) 354 NB. which is 9% of time... c%(#d) 0.0898477 so +/ 2% swings is not exceptional, relatively uncommon but nothing to worry about QUESTION 2 - Volatility over a two week period NB. calculate the period close over 5 sessions periodClose=: _5({.)\ close NB. count of consecutive periods with more than 2% volatility NB. http://www.jsoftware.com/pipermail/programming/2014-October/039985.html [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) 131 NB. 4% of the periods since 1950 had 3 or more weeks of 2% volatility c % (#d) 0.0396489 NB. since 2000 periodClose=: _5({.)\ ((1999 < ". 4&{. every dates) # close) NB. 50 times there have been 3 or more weeks of 2% volatility since 2000 [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) 50 NB. 6.3% of the time c % (#d) 0.0635324 QUESTION 3 - Given a decline of 10% in two weeks, how long does it take to recover? NB. close at the end of 10 sessions periodClose=: _10({.)\ close NB. 59 times there's been more than 5% decline over 10 sessions [ +/ _.05 > pctChange periodClose These are some of the dates: ,. (I. _.05 > pctChange periodClose) { (_10({.)\ dates) +--+ |2015-08-31| +--+ |2011-08-10| +--+ |2010-07-01| +--+ |2009-03-11| +--+ |2009-02-25| +--+ |2008-11-12| +--+ |2008-10-15| +--+ |2008-09-17| +--+ |2008-01-15| +--+ |2003-01-28| +--+ This is starting to look more concerning Now, to find how long it takes to recover: Here are 10 instances: 10 {. (I. _.05 > pctChange periodClose) 0 102 130 163 164 171 173 175 192 317 A crude attempt: NB. find # of 10 day periods between period before y and period closing above that close findRecovery =: 3 : 'y - >./ I. ((y+1) { periodClose) < ((i. y) { periodClose)' NB. number of weeks between the 5% decline over two weeks and the recovery 2 * findRecovery every 10 {. (I. _.05 > pctChange periodClose) _ 22 10 2 8 28 72 116 252 16 22 weeks to recover since the last time it was > 5% decline over two weeks Graphical (from yahoo charts) http://pasteboard.co/k1aGw1i.png I guess I'm not feeling as great now :), but at least it's not that uncommon and in each case it recovered Hope this helps anyone looking at today's market and/or wishing to play with J Happy to take any feedback on the approach as well. -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jchat] looking at recent stock market volatility with J
What is dates? ]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) |value error: dates Probably this... dates=: 0{"1 }. arr ]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) 354 I could muse at length about market volatility issues, but my short form would be that the efficient market hypothesis is almost certainly incorrect and that economists have had a tradition of putting too much emphasis on the wrong issues. Still, > (I. _.05 > pctChange periodClose) { (_10({.)\ dates) is interesting to look at I notice, for example, that 1987-10-23 is in there (and I think I remember hearing some traders from Morgen Stanley saying they cleaned up right around then, because the total valuation of the stock market overflowed 32 bits but they had been using integers in APL, which mean their implementation overflowed to floating point - which meant their stock trading programs just slowed down a bit while everyone else's crashed. And I think that that was right around when they said that that had happened...) I imagine that there's bits of history surrounding each of these episodes, and that each crash corresponds to someone "cleaning up". Thanks, -- Raul On Tue, Sep 1, 2015 at 3:53 PM, Joe Bogner wrote: > I recently invested in an S&P index fund - great timing, I know, so I > figured I'd let J help calm my emotional response to the recent market > volatility. > > > The questions on my mind are: > > 1. Is this level of daily volatility extreme? -- restated as how common are > 2% swings in daily close price (SOLVED) > > 2. Is it uncommon to have this much volatility over a two week period? -- > restated as how common are streaks of 2% change in price across multiple > weeks (SOLVED) > > 3. How long does it take to recover after a period over period change of > -5% (SOLVED) > > I set out to answer these questions: > > Step 1: Let's get some data > > require 'web/gethttp' > > NB. beware linebreak > csv=: gethttp ' > http://real-chart.finance.yahoo.com/table.csv?s=%5EGSPC&d=8&e=1&f=2015&g=d&a=0&b=3&c=1950&ignore=.csv > ' > > arr=: > ',' cut each LF cut csv > > NB. closing price is 4th column > close =: ". every 4{"1 }. arr > > > QUESTION 1 - Daily volatility > > pctChange =: }: -&1@% }. > > NB. 708 days greater than 2% change from previous close > (+/ 0.02 < | pctChange close) > 708 > > NB. 4% of the time since 1950 > (+/ 0.02 < | pctChange close) % (# close) > 0.042852 > > > NB. 354 times since 2000 > ]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) > 354 > > NB. which is 9% of time... > c%(#d) > 0.0898477 > > so +/ 2% swings is not exceptional, relatively uncommon but nothing to > worry about > > QUESTION 2 - Volatility over a two week period > > NB. calculate the period close over 5 sessions > periodClose=: _5({.)\ close > > NB. count of consecutive periods with more than 2% volatility > NB. http://www.jsoftware.com/pipermail/programming/2014-October/039985.html > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > 131 > > NB. 4% of the periods since 1950 had 3 or more weeks of 2% volatility >c % (#d) > 0.0396489 > > NB. since 2000 > periodClose=: _5({.)\ ((1999 < ". 4&{. every dates) # close) > > NB. 50 times there have been 3 or more weeks of 2% volatility since 2000 > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > 50 > > NB. 6.3% of the time >c % (#d) > 0.0635324 > > QUESTION 3 - Given a decline of 10% in two weeks, how long does it take to > recover? > > NB. close at the end of 10 sessions > periodClose=: _10({.)\ close > > NB. 59 times there's been more than 5% decline over 10 sessions > [ +/ _.05 > pctChange periodClose > > > These are some of the dates: > >,. (I. _.05 > pctChange periodClose) { (_10({.)\ dates) > +--+ > |2015-08-31| > +--+ > |2011-08-10| > +--+ > |2010-07-01| > +--+ > |2009-03-11| > +--+ > |2009-02-25| > +--+ > |2008-11-12| > +--+ > |2008-10-15| > +--+ > |2008-09-17| > +--+ > |2008-01-15| > +--+ > |2003-01-28| > +--+ > > This is starting to look more concerning > > Now, to find how long it takes to recover: > > Here are 10 instances: > > 10 {. (I. _.05 > pctChange periodClose) > 0 102 130 163 164 171 173 175 192 317 > > A crude attempt: > > NB. find # of 10 day periods between period before y and period closing > above that close > findRecovery =: 3 : 'y - >./ I. ((y+1) { periodClose) < ((i. y) { > periodClose)' > > > NB. number of weeks between the 5% decline over two weeks and the recovery > 2 * findRecovery every 10 {. (I. _.05 > pctChange periodClose) > _ 22 10 2 8 28 72 116 252 16 > > > 22 weeks to recover since the last time it was > 5% decline over two weeks > > Graphical (from yahoo charts) > http://pasteboard.co/k1aGw1i.png > > I guess I'm not feeling as great now :), but at least it's not that > uncommon and in each case it recovered > > Hope this helps anyone lo
Re: [Jchat] looking at recent stock market volatility with J
oops - thanks for pointing out my missing dates definition. You are correct On Tue, Sep 1, 2015 at 5:31 PM, Raul Miller wrote: > What is dates? > >]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) > |value error: dates > > Probably this... > >dates=: 0{"1 }. arr >]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) > 354 > > I could muse at length about market volatility issues, but my short > form would be that the efficient market hypothesis is almost certainly > incorrect and that economists have had a tradition of putting too much > emphasis on the wrong issues. > > Still, >> (I. _.05 > pctChange periodClose) { (_10({.)\ dates) > > is interesting to look at > > I notice, for example, that 1987-10-23 is in there (and I think I > remember hearing some traders from Morgen Stanley saying they cleaned > up right around then, because the total valuation of the stock market > overflowed 32 bits but they had been using integers in APL, which mean > their implementation overflowed to floating point - which meant their > stock trading programs just slowed down a bit while everyone else's > crashed. And I think that that was right around when they said that > that had happened...) > > I imagine that there's bits of history surrounding each of these > episodes, and that each crash corresponds to someone "cleaning up". > > Thanks, > > -- > Raul > > On Tue, Sep 1, 2015 at 3:53 PM, Joe Bogner wrote: > > I recently invested in an S&P index fund - great timing, I know, so I > > figured I'd let J help calm my emotional response to the recent market > > volatility. > > > > > > The questions on my mind are: > > > > 1. Is this level of daily volatility extreme? -- restated as how common > are > > 2% swings in daily close price (SOLVED) > > > > 2. Is it uncommon to have this much volatility over a two week period? -- > > restated as how common are streaks of 2% change in price across multiple > > weeks (SOLVED) > > > > 3. How long does it take to recover after a period over period change of > > -5% (SOLVED) > > > > I set out to answer these questions: > > > > Step 1: Let's get some data > > > > require 'web/gethttp' > > > > NB. beware linebreak > > csv=: gethttp ' > > > http://real-chart.finance.yahoo.com/table.csv?s=%5EGSPC&d=8&e=1&f=2015&g=d&a=0&b=3&c=1950&ignore=.csv > > ' > > > > arr=: > ',' cut each LF cut csv > > > > NB. closing price is 4th column > > close =: ". every 4{"1 }. arr > > > > > > QUESTION 1 - Daily volatility > > > > pctChange =: }: -&1@% }. > > > > NB. 708 days greater than 2% change from previous close > > (+/ 0.02 < | pctChange close) > > 708 > > > > NB. 4% of the time since 1950 > > (+/ 0.02 < | pctChange close) % (# close) > > 0.042852 > > > > > > NB. 354 times since 2000 > > ]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) > > 354 > > > > NB. which is 9% of time... > > c%(#d) > > 0.0898477 > > > > so +/ 2% swings is not exceptional, relatively uncommon but nothing to > > worry about > > > > QUESTION 2 - Volatility over a two week period > > > > NB. calculate the period close over 5 sessions > > periodClose=: _5({.)\ close > > > > NB. count of consecutive periods with more than 2% volatility > > NB. > http://www.jsoftware.com/pipermail/programming/2014-October/039985.html > > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > > 131 > > > > NB. 4% of the periods since 1950 had 3 or more weeks of 2% volatility > >c % (#d) > > 0.0396489 > > > > NB. since 2000 > > periodClose=: _5({.)\ ((1999 < ". 4&{. every dates) # close) > > > > NB. 50 times there have been 3 or more weeks of 2% volatility since 2000 > > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > > 50 > > > > NB. 6.3% of the time > >c % (#d) > > 0.0635324 > > > > QUESTION 3 - Given a decline of 10% in two weeks, how long does it take > to > > recover? > > > > NB. close at the end of 10 sessions > > periodClose=: _10({.)\ close > > > > NB. 59 times there's been more than 5% decline over 10 sessions > > [ +/ _.05 > pctChange periodClose > > > > > > These are some of the dates: > > > >,. (I. _.05 > pctChange periodClose) { (_10({.)\ dates) > > +--+ > > |2015-08-31| > > +--+ > > |2011-08-10| > > +--+ > > |2010-07-01| > > +--+ > > |2009-03-11| > > +--+ > > |2009-02-25| > > +--+ > > |2008-11-12| > > +--+ > > |2008-10-15| > > +--+ > > |2008-09-17| > > +--+ > > |2008-01-15| > > +--+ > > |2003-01-28| > > +--+ > > > > This is starting to look more concerning > > > > Now, to find how long it takes to recover: > > > > Here are 10 instances: > > > > 10 {. (I. _.05 > pctChange periodClose) > > 0 102 130 163 164 171 173 175 192 317 > > > > A crude attempt: > > > > NB. find # of 10 day periods between period before y and period closing > > above that close > > findRecovery =: 3 : 'y - >./ I. ((y+1) { periodClose) < ((i. y) { > > periodClose)'
Re: [Jchat] looking at recent stock market volatility with J
Nice example, thanks. I would encourage you to post this kind of example in the programming forum rather than chat. Even better would be to make a wiki Essay on it. On 1 September 2015 at 12:53, Joe Bogner wrote: > I recently invested in an S&P index fund - great timing, I know, so I > figured I'd let J help calm my emotional response to the recent market > volatility. > > > The questions on my mind are: > > 1. Is this level of daily volatility extreme? -- restated as how common are > 2% swings in daily close price (SOLVED) > > 2. Is it uncommon to have this much volatility over a two week period? -- > restated as how common are streaks of 2% change in price across multiple > weeks (SOLVED) > > 3. How long does it take to recover after a period over period change of > -5% (SOLVED) > > I set out to answer these questions: > > Step 1: Let's get some data > > require 'web/gethttp' > > NB. beware linebreak > csv=: gethttp ' > > http://real-chart.finance.yahoo.com/table.csv?s=%5EGSPC&d=8&e=1&f=2015&g=d&a=0&b=3&c=1950&ignore=.csv > ' > > arr=: > ',' cut each LF cut csv > > NB. closing price is 4th column > close =: ". every 4{"1 }. arr > > > QUESTION 1 - Daily volatility > > pctChange =: }: -&1@% }. > > NB. 708 days greater than 2% change from previous close > (+/ 0.02 < | pctChange close) > 708 > > NB. 4% of the time since 1950 > (+/ 0.02 < | pctChange close) % (# close) > 0.042852 > > > NB. 354 times since 2000 > ]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) > 354 > > NB. which is 9% of time... > c%(#d) > 0.0898477 > > so +/ 2% swings is not exceptional, relatively uncommon but nothing to > worry about > > QUESTION 2 - Volatility over a two week period > > NB. calculate the period close over 5 sessions > periodClose=: _5({.)\ close > > NB. count of consecutive periods with more than 2% volatility > NB. > http://www.jsoftware.com/pipermail/programming/2014-October/039985.html > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > 131 > > NB. 4% of the periods since 1950 had 3 or more weeks of 2% volatility >c % (#d) > 0.0396489 > > NB. since 2000 > periodClose=: _5({.)\ ((1999 < ". 4&{. every dates) # close) > > NB. 50 times there have been 3 or more weeks of 2% volatility since 2000 > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > 50 > > NB. 6.3% of the time >c % (#d) > 0.0635324 > > QUESTION 3 - Given a decline of 10% in two weeks, how long does it take to > recover? > > NB. close at the end of 10 sessions > periodClose=: _10({.)\ close > > NB. 59 times there's been more than 5% decline over 10 sessions > [ +/ _.05 > pctChange periodClose > > > These are some of the dates: > >,. (I. _.05 > pctChange periodClose) { (_10({.)\ dates) > +--+ > |2015-08-31| > +--+ > |2011-08-10| > +--+ > |2010-07-01| > +--+ > |2009-03-11| > +--+ > |2009-02-25| > +--+ > |2008-11-12| > +--+ > |2008-10-15| > +--+ > |2008-09-17| > +--+ > |2008-01-15| > +--+ > |2003-01-28| > +--+ > > This is starting to look more concerning > > Now, to find how long it takes to recover: > > Here are 10 instances: > > 10 {. (I. _.05 > pctChange periodClose) > 0 102 130 163 164 171 173 175 192 317 > > A crude attempt: > > NB. find # of 10 day periods between period before y and period closing > above that close > findRecovery =: 3 : 'y - >./ I. ((y+1) { periodClose) < ((i. y) { > periodClose)' > > > NB. number of weeks between the 5% decline over two weeks and the recovery > 2 * findRecovery every 10 {. (I. _.05 > pctChange periodClose) > _ 22 10 2 8 28 72 116 252 16 > > > 22 weeks to recover since the last time it was > 5% decline over two weeks > > Graphical (from yahoo charts) > http://pasteboard.co/k1aGw1i.png > > I guess I'm not feeling as great now :), but at least it's not that > uncommon and in each case it recovered > > Hope this helps anyone looking at today's market and/or wishing to play > with J > > Happy to take any feedback on the approach as well. > -- > For information about J forums see http://www.jsoftware.com/forums.htm > -- For information about J forums see http://www.jsoftware.com/forums.htm
Re: [Jchat] looking at recent stock market volatility with J
Thanks Chris. I will polish it up and add some plot examples and will post to the wiki On Sep 1, 2015 5:55 PM, "chris burke" wrote: > Nice example, thanks. > > I would encourage you to post this kind of example in the programming forum > rather than chat. Even better would be to make a wiki Essay on it. > > > On 1 September 2015 at 12:53, Joe Bogner wrote: > > > I recently invested in an S&P index fund - great timing, I know, so I > > figured I'd let J help calm my emotional response to the recent market > > volatility. > > > > > > The questions on my mind are: > > > > 1. Is this level of daily volatility extreme? -- restated as how common > are > > 2% swings in daily close price (SOLVED) > > > > 2. Is it uncommon to have this much volatility over a two week period? -- > > restated as how common are streaks of 2% change in price across multiple > > weeks (SOLVED) > > > > 3. How long does it take to recover after a period over period change of > > -5% (SOLVED) > > > > I set out to answer these questions: > > > > Step 1: Let's get some data > > > > require 'web/gethttp' > > > > NB. beware linebreak > > csv=: gethttp ' > > > > > http://real-chart.finance.yahoo.com/table.csv?s=%5EGSPC&d=8&e=1&f=2015&g=d&a=0&b=3&c=1950&ignore=.csv > > ' > > > > arr=: > ',' cut each LF cut csv > > > > NB. closing price is 4th column > > close =: ". every 4{"1 }. arr > > > > > > QUESTION 1 - Daily volatility > > > > pctChange =: }: -&1@% }. > > > > NB. 708 days greater than 2% change from previous close > > (+/ 0.02 < | pctChange close) > > 708 > > > > NB. 4% of the time since 1950 > > (+/ 0.02 < | pctChange close) % (# close) > > 0.042852 > > > > > > NB. 354 times since 2000 > > ]c=. +/ 0.02 < | pctChange d=. ((1999 < ". 4&{. every dates) # close) > > 354 > > > > NB. which is 9% of time... > > c%(#d) > > 0.0898477 > > > > so +/ 2% swings is not exceptional, relatively uncommon but nothing to > > worry about > > > > QUESTION 2 - Volatility over a two week period > > > > NB. calculate the period close over 5 sessions > > periodClose=: _5({.)\ close > > > > NB. count of consecutive periods with more than 2% volatility > > NB. > > http://www.jsoftware.com/pipermail/programming/2014-October/039985.html > > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > > 131 > > > > NB. 4% of the periods since 1950 had 3 or more weeks of 2% volatility > >c % (#d) > > 0.0396489 > > > > NB. since 2000 > > periodClose=: _5({.)\ ((1999 < ". 4&{. every dates) # close) > > > > NB. 50 times there have been 3 or more weeks of 2% volatility since 2000 > > [ c=.+/ 2 < ([*+)/\.&.|. d=. (0.02 < | pctChange periodClose) > > 50 > > > > NB. 6.3% of the time > >c % (#d) > > 0.0635324 > > > > QUESTION 3 - Given a decline of 10% in two weeks, how long does it take > to > > recover? > > > > NB. close at the end of 10 sessions > > periodClose=: _10({.)\ close > > > > NB. 59 times there's been more than 5% decline over 10 sessions > > [ +/ _.05 > pctChange periodClose > > > > > > These are some of the dates: > > > >,. (I. _.05 > pctChange periodClose) { (_10({.)\ dates) > > +--+ > > |2015-08-31| > > +--+ > > |2011-08-10| > > +--+ > > |2010-07-01| > > +--+ > > |2009-03-11| > > +--+ > > |2009-02-25| > > +--+ > > |2008-11-12| > > +--+ > > |2008-10-15| > > +--+ > > |2008-09-17| > > +--+ > > |2008-01-15| > > +--+ > > |2003-01-28| > > +--+ > > > > This is starting to look more concerning > > > > Now, to find how long it takes to recover: > > > > Here are 10 instances: > > > > 10 {. (I. _.05 > pctChange periodClose) > > 0 102 130 163 164 171 173 175 192 317 > > > > A crude attempt: > > > > NB. find # of 10 day periods between period before y and period closing > > above that close > > findRecovery =: 3 : 'y - >./ I. ((y+1) { periodClose) < ((i. y) { > > periodClose)' > > > > > > NB. number of weeks between the 5% decline over two weeks and the > recovery > > 2 * findRecovery every 10 {. (I. _.05 > pctChange periodClose) > > _ 22 10 2 8 28 72 116 252 16 > > > > > > 22 weeks to recover since the last time it was > 5% decline over two > weeks > > > > Graphical (from yahoo charts) > > http://pasteboard.co/k1aGw1i.png > > > > I guess I'm not feeling as great now :), but at least it's not that > > uncommon and in each case it recovered > > > > Hope this helps anyone looking at today's market and/or wishing to play > > with J > > > > Happy to take any feedback on the approach as well. > > -- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > -- > For information about J forums see http://www.jsoftware.com/forums.htm > -- For information about J forums see http://www.jsoftware.com/forums.htm