You may like to look at how we covered this topic at a meeting of NYCJUG last year: http://www.jsoftware.com/jwiki/NYCJUG/2012-12-11#Example_of_Free-Form_Text_Wrangling.
On Tue, Oct 8, 2013 at 12:28 PM, Dan Farmer <[email protected]> wrote: > Yeah, this thread has been a gold mine for me -- lots to study, to > really understand how they all work, but very useful. Also in > particular thank you Joe, I think you and I started learning J around > the same time and it's a bit of an inspiration to see how quickly > you're picking it up. I'll have to step up my game =) > > Thanks to everyone for all of the tips and tricks! > > -Dan > > On Tue, Oct 8, 2013 at 7:05 AM, Joe Bogner <[email protected]> wrote: > > I wrote: > >> Is there a better way to find indices of a subset within a greater list? > > This is my hacky solution > > > > Sorry, please disregard my question. I figured out a cleaner > implementation > > using the table adverb > > > > (; > L:0 I. each <"1 (find =/ list)) { list > > > > NB. 13 : '(; > L:0 I. each <"1 (x =/ y)) { y' > > in=:] {~ [: ; [: >L:0 [: I.&.> [: <"1 =/ > > > > ] find in list > > > > ┌─┬─┬─┬─┐ > > > > │a│a│b│q│ > > > > └─┴─┴─┴─┘ > > > > > > ] (<'a') in ('a';'b') > > > > ┌─┐ > > > > │a│ > > > > └─┘ > > > > > > ] ('a';'b') in ('a';'b';'c';'a') > > > > ┌─┬─┬─┐ > > > > │a│a│b│ > > > > └─┴─┴─┘ > > > > > > > > idx=:13 : '(; > L:0 I. each <"1 (x =/ y))' > > > > > > Which then lets us do things like the following to merge/join tables > > > > > > (find idx list) { list > > > > > > My previous hack of namesTenRows=.(; L:0 > (3 : '(<y) ([: I. =) names' ) > > each namesTen) { data > > > > Becomes: namesTenRows=. (namesTen idx names) { data > > > > > > Also quite a bit faster than the hacky solution > > > > > > ]1e5 * 1e5 (6!:2) '; > L:0 (3 : ''(<y) ([: I. =) list'' ) each find' > > > > 0.589178 > > > > > > ]1e5 * 1e5 (6!:2) '(find idx list)' > > > > 0.224204 > > > > > > Fun fun > > > > > > http://www.jsoftware.com/help/dictionary/d420.htm > > http://www.jsoftware.com/help/learning/18.htm > > > > I'd be happy to contribute to a wiki page on slicing/dicing tables. I > don't > > have access to modify the wiki at present. > > > > > > On Tue, Oct 8, 2013 at 9:20 AM, Joe Bogner <[email protected]> wrote: > > > >> One more -- matching to a list (joining tables) > >> > >> NB. Get the names of everyone from the frequency table with exactly 10 > >> matches > >> > >> ]namesTen=. > 0}"1 (10 ([: I. =) (> 1{"1 nameFrequency)) { > nameFrequency > >> > >> ┌─────┬─────┬─────┬─────┬─────┬──────┬─────┬──────┬────────┬─────┐ > >> > >> │Ruben│Paddy│Cecil│Chico│Edwin│Martin│Kelly│Junior│Jonathan│Duane│ > >> > >> └─────┴─────┴─────┴─────┴─────┴──────┴─────┴──────┴────────┴─────┘ > >> > >> > >> > >> namesTenRows=.(; L:0 > (3 : '(<y) ([: I. =) names' ) each namesTen) { > data > >> > >> > >> NB. Get the unique names in our subset to validate it matches the above > >> > >> ] ~. (colIdx <'nameFirst') {"1 namesTenRows > >> > >> ┌─────┬─────┬─────┬─────┬─────┬──────┬─────┬──────┬────────┬─────┐ > >> > >> │Ruben│Paddy│Cecil│Chico│Edwin│Martin│Kelly│Junior│Jonathan│Duane│ > >> > >> └─────┴─────┴─────┴─────┴─────┴──────┴─────┴──────┴────────┴─────┘ > >> > >> > >> > >> Is there a better way to find indices of a subset within a greater list? > >> This is my hacky solution > >> > >> > >> find=.('a';'b';'q') > >> > >> list=.('a';'b';'z';'c';'a';'q') > >> > >> > >> ] (; > L:0 (3 : '(<y) ([: I. =) list' ) each find) { list > >> > >> ┌─┬─┬─┬─┐ > >> > >> │a│a│b│q│ > >> > >> └─┴─┴─┴─┘ > >> > >> > >> find xxx list > >> > >> > >> > >> On Tue, Oct 8, 2013 at 7:14 AM, Ric Sherlock <[email protected]> wrote: > >> > >>> A relatively simple adverb like the one below can be useful if you a > >>> repeatedly trying to access different fields in the table: > >>> > >>> NB.*getFields a Returns fields (columns) from y specified in x > >>> > >>> NB. EG: 'fieldname' tablehdr getFields table > >>> > >>> NB. EG: ('Lastname';'Firstname') hdr getFields dat > >>> > >>> NB. y is: table of records > >>> > >>> NB. m is: list of boxed field names in y > >>> > >>> NB. x is: list of boxed field names to return > >>> > >>> getFields=: 1 : 0 > >>> > >>> : > >>> > >>> flds=. boxopen x > >>> > >>> idx=. m i. flds > >>> > >>> idx {"1 y > >>> > >>> ) > >>> > >>> > >>> require 'tables/csv' > >>> > >>> 'hdr dat'=: split readcsv 'c:/tmp/master.csv' > >>> > >>> > >>> 4{. dat hdr getFields~ 'nameFirst';'birthYear';'weight' > >>> > >>> > >>> ┌──────┬────┬───┐ > >>> > >>> │Hank │1934│180│ > >>> > >>> ├──────┼────┼───┤ > >>> > >>> │Tommie│1939│190│ > >>> > >>> ├──────┼────┼───┤ > >>> > >>> │Don │1954│190│ > >>> > >>> ├──────┼────┼───┤ > >>> > >>> │Andy │1972│184│ > >>> > >>> └──────┴────┴───┘ > >>> > >>> > >>> > >>> > >>> On Tue, Oct 8, 2013 at 11:39 PM, Joe Bogner <[email protected]> > wrote: > >>> > >>> > Here are a couple other examples: > >>> > > >>> > NB. Convert years to a flat list of numbers > >>> > > >>> > birthYears=. , L:0 (> ". each ((colIdx <'birthYear') {"1 data)) > >>> > > >>> > > >>> > NB. Find players born between 1930 and 1950 named Hank > >>> > > >>> > (I. (birthYears > 1930) * (birthYears < 1950) * (names = <'Hank')) { > >>> data > >>> > > >>> > > >>> > NB. Find players born between 1930 and 1950 named Hank or Tommie > >>> > > >>> > (I. (birthYears > 1930) * (birthYears < 1950) * ( (names = <'Hank') + > >>> > (names = <'Tommie'))) { data > >>> > > >>> > > >>> > > >>> > NB. http://www.jsoftware.com/jwiki/Essays/DataStructures > >>> > > >>> > diff=: -. [ > >>> > > >>> > intersect=: e. # [ > >>> > > >>> > union=: ~.@, > >>> > > >>> > > >>> > in=: ] ([: I. =)~ [: < [ NB. 4 : '(<x) ([: I. > =) > >>> y' > >>> > > >>> > inT=: 4 : 'x in (colIdx <y) {"1 data' > >>> > > >>> > > >>> > NB. Another style > >>> > > >>> > NB. Hank or Tommie - the union of indices > >>> > > >>> > (('Hank' inT 'nameFirst') union ('Tommie' inT 'nameFirst')) { data > >>> > > >>> > > >>> > NB. Hank or Tommie AND born in Cuba > >>> > > >>> > ((('Hank' inT 'nameFirst') union ('Tommie' inT 'nameFirst')) > intersect > >>> > ('Cuba' inT 'birthCountry')) { data > >>> > > >>> > > >>> > > >>> > On Tue, Oct 8, 2013 at 3:50 AM, R.E. Boss <[email protected]> > wrote: > >>> > > >>> > > So 'data' was plural? > >>> > > > >>> > > > >>> > > R.E. Boss > >>> > > > >>> > > (Add your info to > >>> http://www.jsoftware.com/jwiki/Community/Demographics) > >>> > > > >>> > > > >>> > > > -----Original Message----- > >>> > > > From: [email protected] [mailto: > programming- > >>> > > > [email protected]] On Behalf Of Henry Rich > >>> > > > Sent: dinsdag 8 oktober 2013 3:52 > >>> > > > To: [email protected] > >>> > > > Subject: Re: [Jprogramming] Slicing and dicing tables > >>> > > > > >>> > > > That is what he meant, but it's a minority opinion. > >>> > > > > >>> > > > Henry Rich > >>> > > > > >>> > > > On 10/7/2013 8:56 PM, Dan Bron wrote: > >>> > > > > You mean " 'data' is plural " . > >>> > > > > > >>> > > > > -Dan > >>> > > > > > >>> > > > > PS: http://en.m.wikipedia.org/wiki/Muphry's_law > >>> > > > > > >>> > > > > (Yes, that link is spelled correctly :) > >>> > > > > > >>> > > > > Please excuse typos; composed on a handheld device. > >>> > > > > > >>> > > > > On Oct 7, 2013, at 8:00 PM, Eldon Eller <[email protected]> > >>> wrote: > >>> > > > > > >>> > > > >> Data are plural. > >>> > > > >> > >>> > > > >> On 10/07/2013 02:04 PM, Dan Bron wrote: > >>> > > > >>> Ganesh Rapolu > >>> > > > >>>> Because the data is boxed, all comparisons must be boxed. > >>> > > > >>> This was very well put (one short sentence which both > clarifies > >>> the > >>> > > > >>> problem and justifies the solution). > >>> > > > >>> > >>> > > > >>> -Dan > >>> > > > >>> > >>> > > > >>> > >>> > > > >>> > >>> > > > >>> > >>> > -------------------------------------------------------------------- > >>> > > > >>> -- For information about J forums see > >>> > > > >>> http://www.jsoftware.com/forums.htm > >>> > > > >> > >>> > > > >> > >>> > --------------------------------------------------------------------- > >>> > > > >> - For information about J forums see > >>> > > > >> http://www.jsoftware.com/forums.htm > >>> > > > > > >>> > > ---------------------------------------------------------------------- > >>> > > > > For information about J forums see > >>> > > > http://www.jsoftware.com/forums.htm > >>> > > > > > >>> > > > > >>> ---------------------------------------------------------------------- > >>> > > > For information about J forums see > >>> http://www.jsoftware.com/forums.htm > >>> > > > >>> > > > ---------------------------------------------------------------------- > >>> > > For information about J forums see > >>> http://www.jsoftware.com/forums.htm > >>> > > > >>> > > ---------------------------------------------------------------------- > >>> > For information about J forums see > http://www.jsoftware.com/forums.htm > >>> > > >>> ---------------------------------------------------------------------- > >>> For information about J forums see http://www.jsoftware.com/forums.htm > >>> > >> > >> > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > -- Devon McCormick, CFA ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
