I'm back with another question! ;-) I know how to amend a list, but
amending a table presents challenges beyond my knowledge of J. I read
the several sources that deal very briefly with amending tables, but
they assume knowledge and familiarity which is beyond me at the
moment. Additionally, I found it difficult to translate the
simplicity and artificiality of the examples to my "real life"
situation.
Since attachments aren't permitted, I will include in this message
some test data and the code I've written for the interpolation of
"missing" stock market data:
##################################################
itest.csv
1952-05-22,261.78,264.02,261.54,263.33,1360000
1952-05-23,263.33,264.59,262.54,263.27,1150000
1952-05-24,263.27,263.47,262.84,263.23,300000
1952-05-25,0,0,0,0,0
1952-05-26,263.23,264.87,262.62,264.22,940000
1952-05-27,264.22,265.17,262.82,263.92,1040000
1952-05-28,263.92,264.54,262.25,262.78,1130000
1952-05-29,262.78,263.51,261.62,262.94,1100000
1952-05-30,0,0,0,0,0
1952-05-31,0,0,0,0,0
1952-06-01,0,0,0,0,0
1952-06-02,262.94,264.61,261.48,262.31,1190000
1952-06-03,262.31,262.82,260.83,262.09,940000
1952-06-04,262.09,264.20,261.91,263.67,1200000
1952-06-05,263.77,266.80,263.77,266.29,1410000
1952-06-06,266.29,268.95,266.21,268.03,1520000
1952-06-07,0,0,0,0,0
1952-06-08,0,0,0,0,0
1952-06-09,268.03,269.92,267.67,269.15,1270000
1952-06-10,269.15,269.15,266.76,267.67,1220000
1952-06-11,267.67,268.52,266.33,267.93,1190000
##################################################
itest2.ijs
NB. these are standard in all of my programs, whether used or not:
require 'files strings dates stdlib'
require 'tables\dsv' NB. per Ric Sherlock, 6 Jan 2019
require 'types\datetime' NB. per Raul Miller, 30 Mar 2020
require 'format\printf' NB. per Raul Miller, 30 Mar 2020
9!:11 (15) NB. set print precision to 15-digits
root=: '~user\data\stocks\' NB. data location
NB. syntax: itest2 'itest.csv'
itest2=: 3 : 0
filin=. y
fileA=. ',' readdsv (jpath root,filin) NB. based on Ric Sherlock, 6 Jan 2019
dates=. {."1 fileA
data=. |: }."1 fileA NB. transpose for usual J handling
smoutput data NB. to compare starting data with finished data
rows=. 5 NB. 0 to 4: open, hi, lo, close, volume
closerow=. 3 NB. close data is in row 3
NB. close data always exists--use it to find weekends, etc.
cols=. #(closerow{data) NB. number of columns (days) in data
col=. 1 NB. start at column 1 (2nd position)
while. cols > col do.
first=. col
if. 0 = ". > col{closerow{data do.
while. 0 = ". > col{closerow{data do.
last=. 1+col
col=. 1+col
end. NB. while
first=. first-1
diff=. last-first
NB. the following loops process single and multiple
NB. empty "zero" rows for interpolation purposes
start=. 1+first
stop=. last-1
numerator=. 1
while. start <: stop do.
for_row. i. rows do.
frdata=. ". >first{row{data
lrdata=. ". >last{row{data
rowdiff=. lrdata-frdata
coldiff=. rowdiff % diff
numdiff=. numerator*coldiff
interpoldata=. frdata+numdiff
if. 4 = row do. NB. volume
interpolated=. 0j0 ": interpoldata
else. NB . everything else
interpolated=. 0j2 ": interpoldata
end. NB. if
NB. smoutput 'col=';start;'row=';row;'interpolation=';interpolated
NB. the following line, of course, is illegal in J,
NB. but, conceptually, it's what I'm trying to do
NB. [ i.e., amend the boxed datum at start { row { data ]
NB.
NB. (row{data)=. (< interpolated) start } row{data
end. NB. for_row
numerator=. 1+numerator
start=. 1+start
end. NB. while
end. NB. if
col=. 1+col
end. NB. while
smoutput data NB. to compare starting data with finished data
NB. in the actual program, the data is transposed again, the
NB. dates list is reappended to the data table, and they are
NB. output to a file with a suffix to indicate it's a processed file;
NB. that file is then passed on to the next step in a larger program
smoutput 'DONE'
)
##################################################
Please, I am not seeking comments on my programming code--after a lot
of effort, it finally works correctly for me, and that's all that
counts. What I am asking your help with is how to code (one or more
lines of EXPLICIT code--not tacit) the amending of each interpolated
value into its correct place in the original table. Basically, my
code finds a weekday or holiday (which initially has a zero value as
an identifier), seeks how many zero days in sequence there are,
calculates all 5 daily interpolated values (one value at a time) for
each day in the sequence (one day at a time), and repeats the process
until the end of data is reached. I chose the particular test data
because it exhibits 1-day, 2-day, and 3-day sequences of days that the
stock market was closed within a short period of time. I have
inserted smoutput commands to visually compare the data file before
and after the entire process (to see if the amending worked
correctly).
I am "stuck" until my amending issue can be resolved, so I GREATLY
appreciate any help with this!
Harvey
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
