Answering Justin's questions.
01 question
02 answer
11. The data in your solution seems to be similar to the data in
relational databases, but the query space seems to be the same. Cyclic
relations seem to be ruled out, but this maybe isn't a problem. Am I wrong, and
is either stronger than the other?
12. The relations between two ordinal fractions are: equality (a=b),
subordination (a<b), superordination (a>b), compatibility (a<>b) and
incompatility (a><b). You are right, no cyclic relation. The browser omits the
records incompatible with the query ordinal fraction from the answer. (in
another browser I painted an answer (a) to a query (b) white if a=b, green if
a<b, red if a>b, yellow if a<>b, and invisible if a><b.)
21. Is it possible to provide a well-performing ordinal fraction
interface to multiple separate files (hopefully memory mapped) in an intuitive
way in J? For instance, if we are looking at a subset of
columns which does not intersect with the set of columns of a given table, then
we would like to skip reading the rows in that table seamlessly.
22. Two files are merged into one by prefixing the line numbers of the first
file by "1" and the line numbers of the second file by "2". The resulting
merged file is then another ordinal fraction file to replace the original two
files. Well-performing means fast. If the data base is big it is conveniently
stored as an indexed sorted file and the browser is made to skip lines before a
possible match.
31. I am not fully convinced that this is easier to manage than a
relational database.
32. How to fit the credo text into a relational data base? I don't think it is
at all feasible.
41. If we did need to define explicit tables (sets of columns), then
this could happen in a separate schema table. What are your thoughts?
42. No, there is no need for a separate schema table. A table (like this one)
with rows 10 20 30 40 50 60 and columns 01 02 has elements 11 12 13 14 15 16 21
22 23 24 25 26. There is but one file containing everything.
51. This doesn't fully solve the issue of coordinating the updating of all
resources and the dependencies between them for updating. I guess Make would
help.
52. What is the problem?
61. I wonder how Jd could contribute.
62. So do I!
Thank you very much!
I will comment on Hauke's contribution tomorrow. Now only this: I am very
impressed. Hauke's is the most qualified response I have ever received. Note
that a line number in the data base may be extended by zeroes. " AMEN" =
"0000000 AMEN". So "AMEN" is unavoidable in the answer to any query.
Thank you everyone!
Bo.
Den fredag den 8. januar 2021 10.36.26 CET skrev Justin Paston-Cooper
<paston.coo...@gmail.com>:
Quite an interesting paper. What were the reasons for its rejection?
The mathematics is quite simple and well defined, however it doesn't
go too much into the application aspect.
I don't have the time to actually explore this in depth right now, but:
1. The data in your solution seems to be similar to the data in
relational databases, but the query space seems to be the same. Cyclic
relations seem to be ruled out, but this maybe isn't a problem. Am I
wrong, and is either stronger than the other?
2. Is it possible to provide a well-performing ordinal fraction
interface to multiple separate files (hopefully memory mapped) in an
intuitive way in J? For instance, if we are looking at a subset of
columns which does not intersect with the set of columns of a given
table, then we would like to skip reading the rows in that table
seamlessly.
3. I am not fully convinced that this is easier to manage than a
relational database.
4. If we did need to define explicit tables (sets of columns), then
this could happen in a separate schema table. What are your thoughts?
5. This doesn't fully solve the issue of coordinating the updating of
all resources and the dependencies between them for updating. I guess
Make would help.
6. I wonder how Jd could contribute.
On Fri, 8 Jan 2021 at 00:08, 'Bo Jacoby' via Programming
<programm...@jsoftware.com> wrote:
>
> "I am looking for a way to better organise my research. If not
> spreadsheets, do you have some advice on how to coordinate all this
> separate data in one place?"
> I have used ordinal fractions for structuring data since 1980. ORDINAL
> FRACTIONS - the algebra of data
>
> |
> |
> |
> | | |
>
> |
>
> |
> |
> | |
> ORDINAL FRACTIONS - the algebra of data
>
> This paper was submitted to the 10th World Computer Congress, IFIP 1986
> conference, but rejected by the referee....
> |
>
> |
>
> |
>
>
>
> I wrote software for processing this kind of data in fortran, BASIC, and
> pascal, but not (yet) in J.
> A BASIC program for browsing the data base is this.
> 1 INPUT;C$: IF C$="" THEN END
> 2 OPEN"CREDO" FOR INPUT AS 1: PRINT":";
> 3 IF EOF(1) THEN CLOSE:PRINT:GOTO 1
> 4 LINE INPUT#1,A$: B$=C$
> 5 IF A$=""THEN A%=-1 ELSE A%=ASC(A$)-48:A$=MID$(A$,2)
> 6 IF B$=""THEN B%=-1 ELSE B%=ASC(B$)-48:B$=MID$(B$,2)
> 7 IF A%<0 THEN PRINT" ";A$;:GOTO 3
> 8 IF A%=0 OR B%=0 OR A%=B% THEN 5 ELSE 3
>
> The test data base for illustrating the possibilities is this.
> 1 CREDO
> 11 IN
> 111 UNUM
> 11 DEUM
> 112 PATREM
> 1121 OMNIPOTENTEM
> 113 FACTOREM
> 1131 CÆLI
> 1139 ET
> 1132 TERRÆ
> 11331 VISIBILIUM
> 1133 OMNIUM
> 11339 ET
> 11332 INVISIBILIUM
> 19 ET
> 12 IN
> 1211 UNUM
> 1211 DOMINUM
> 12 JESUM
> 1211 CHRISTUM
> 1212 FILIUM
> 1212 DEI
> 12121 UNIGENITUM
> 1219 ET
> 1213 EX
> 1213 PATRE
> 1213 NATUM
> 12131 ANTE
> 121311 OMNIA
> 12131 SÆCULA
> 1221 DEUM
> 12211 DE
> 12211 DEO
> 1222 LUMEN
> 12221 DE
> 12221 LUMINE
> 1223 DEUM
> 12231 VERUM
> 12232 DE
> 12232 DEO
> 122321 VERO
> 1231 GENITUM
> 12311 NON
> 12311 FACTUM
> 1232 CONSUBSTANTIALEM
> 1232 PATRI
> 12321 PER
> 12321 QUEM
> 12321 OMNIA
> 12321 FACTA
> 12321 SUNT
> 124 QUI
> 124101 PROPTER
> 124101 NOS
> 12410101 HOMINES
> 124109 ET
> 124102 PROPTER
> 12410201 NOSTRAM
> 124102 SALUTEM
> 12411 DESCENDIT
> 1241101 DE
> 1241101 CÆLIS
> 12419 ET
> 12412 INCARNATUS EST
> 1241201 DE
> 1241201 SPIRITU 124120101 SANCTO
> 1241202 EX
> 1241202 MARIA
> 124120201 VIRGINE
> 12419 ET
> 1241301 HOMO
> 12413 FACTUS EST
> 124211 CRUCIFIXUS
> 1242101 ETIAM
> 1242101 PRO
> 1242101 NOBIS
> 1242102 SUB
> 1242102 PONTIO
> 1242102 PILATO
> 124212 PASSUS
> 124219 ET
> 124213 SEPULTUS
> 12421 EST
> 12429 ET
> 12422 RESURREXIT
> 124221 TERTIA
> 124221 DIE
> 124222 SECUMDUM
> 124222 SCRIPTURAS
> 12429 ET
> 12423 ASCENDIT
> 124231 IN
> 124231 CÆLUM
> 12424 SEDET
> 124241 AD
> 124241 DEXTERAM
> 124241 PATRIS
> 12429 ET
> 124251 ITERUM
> 12425 VENTURUS EST
> 124252 CUM
> 124252 GLORIA
> 124253 JUDICARE
> 1242531 VIVOS
> 1242539 ET
> 1242532 MORTUOS
> 125 CUJUS
> 125 REGNI
> 125 NON ERIT
> 125 FINIS
> 19 ET
> 13 IN
> 13 SPIRITUM
> 131 SANCTUM
> 132 DOMINUM
> 139 ET
> 133 VIVIFICANTEM
> 134 QUI
> 134 EX
> 1341 PATRE
> 1342 FILIO
> 1349 QUE
> 134 PROCEDIT
> 135 QUI
> 135 CUM
> 13501 PATRE
> 13509 ET
> 13502 FILIO
> 13509 SIMUL
> 1351 ADORATUR
> 1359 ET
> 1352 GLORIFICATUR
> 136 QUI
> 136 LOCUTUS EST
> 1361 PER
> 1361 PROPHETAS
> 19 ET
> 141 UNAM
> 142 SANCTAM
> 143 CATHOLICAM
> 149 ET
> 144 APOSTOLICAM
> 14 ECCLESIAM
> 2 CONFITEOR
> 211 UNUM
> 21 BAPTISMA
> 212 IN
> 212 REMISSIONEM
> 2121 PECCATORUM
> 9 ET
> 3 EXPECTO
> 31 RESURRECTIONEM
> 311 MORTUORUM
> 39 ET
> 32 VITAM
> 3211 VENTURI
> 321 SÆCULI
> AMEN
>
> Some test runs of the program look like this.
> 13510: CREDO IN SPIRITUM QUI CUM PATRE ET FILIO SIMUL ADORATUR AMEN
> 13520: CREDO IN SPIRITUM QUI CUM PATRE ET FILIO SIMUL GLORIFICATUR AMEN
> 13501: CREDO IN SPIRITUM QUI CUM PATRE ADORATUR ET GLORIFICATUR AMEN
> 13502: CREDO IN SPIRITUM QUI CUM FILIO ADORATUR ET GLORIFICATUR AMEN
> 13511: CREDO IN SPIRITUM QUI CUM PATRE ADORATUR AMEN
> 13512: CREDO IN SPIRITUM QUI CUM FILIO ADORATUR AMEN
> 13521: CREDO IN SPIRITUM QUI CUM PATRE GLORIFICATUR AMEN
> 13522: CREDO IN SPIRITUM QUI CUM FILIO GLORIFICATUR AMEN
>
> I realize that this is not easy to understand, but I know that it is worth
> while.
> Good luck!
> Bo. Den torsdag den 7. januar 2021 21.35.12 CET skrev Justin Paston-Cooper
> <paston.coo...@gmail.com>:
>
> Thanks. I have been meaning to look at that.
>
> On Thu, 7 Jan 2021 at 23:33, Joe Bogner <joebog...@gmail.com> wrote:
> >
> > Jupyter notebooks may help you with organizing your research -
> > https://code.jsoftware.com/wiki/Guides/Jupyter
> >
> > This has been my preferred tool - far above Excel.
> >
> > On Thu, Jan 7, 2021 at 2:39 PM Justin Paston-Cooper
> > <paston.coo...@gmail.com>
> > wrote:
> >
> > > I am open to suggestions. Right now I'm researching a lot of related
> > > things concurrently. I'm storing some of the results in TSV files.
> > > Some of the scripts are Python, some are curl | jq | awk. Some of the
> > > results I am storing as variables in J scripts. I am constantly going
> > > back and forth between differing representations, differing
> > > environments, recalculating things needlessly, and so on.
> > >
> > > I am looking for a way to better organise my research. If not
> > > spreadsheets, do you have some advice on how to coordinate all this
> > > separate data in one place? A Make file could be a start, but this
> > > doesn't satisfy the requirement of having a nice editable GUI to
> > > arrange and display all the separate sources of data. Maybe wd would
> > > be a start in that direction. I haven't researched the alternatives.
> > >
> > > How do you organise your research?
> > >
> > > Application: Researching interactions between prices of a set of
> > > things in each of a set of places. There are many different analyses
> > > that can be made. I am finding it hard to keep track of all the angles
> > > I have looked at. These angles all reside in separate directories,
> > > which is not ideal. I have hand-written notes, but those need to be
> > > updated by hand.
> > >
> > > By the way, I wasn't envisioning doing any calculation in the
> > > spreadsheet. The idea of the spreadsheet was simply to coordinate
> > > communication and (re)calculation between various calculation
> > > processes, display the results, and allow the display of the results
> > > to be edited.
> > >
> > > Imagine an actor system with the spreadsheet being the coordinator.
> > >
> > > On Thu, 7 Jan 2021 at 20:23, Devon McCormick <devon...@gmail.com> wrote:
> > > >
> > > > It would be remiss of me not to mention that you really ought to
> > > > re-consider making a spreadsheet an integral part of your design, not
> > > > the
> > > > least due to the historically high rates of error that have been
> > > > measured
> > > > in spreadsheets - 1 to 5%:
> > > > https://arxiv.org/ftp/arxiv/papers/1602/1602.02601.pdf . It seems
> > > > incongruous to worry about the sixth decimal place in numbers with many
> > > > digits before the decimal point but ignoring error rates that dwarf this
> > > > imprecision.
> > > >
> > > > By way of comparison, in most code-bases where people measure errors, an
> > > > error rate of 10 bad lines per 1000 lines of code would be considered
> > > > unacceptably high.
> > > > ----------------------------------------------------------------------
> > > > For information about J forums see http://www.jsoftware.com/forums.htm
> > > ----------------------------------------------------------------------
> > > For information about J forums see http://www.jsoftware.com/forums.htm
> > >
> > ----------------------------------------------------------------------
> > For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
