[twdev] Proposing a plaintext parse & render mode for attributes & more

[Evan Balster]

Hey, all —

Currently TiddlyWiki implements plain-text wikification by applying 
WikiText parsing to generate a DOM, then extracting and concatenating any 
text-widgets in the DOM while discarding everything else.  While this 
functionality is very useful, it has some tricky drawbacks — wikification 
involves (often) unnecessary work processing irrelevant parse rules, and 
anything resembling an  or //wikitext markup will be stripped out, 
when this might be very undesirable (eg, when encountering JavaScript or 
C++ comments).

I'd like to propose an alternative mode where only those parse rules which 
are applicable to plaintext are considered — that is, widgets, 
transclusions (including lists), variables and a few other items.  That 
way, we can write things like this (a method stub generator):

<$list variable=class filter="[tag[Class]]">
<$list filter="[tag[Method][tag]]">
// {{!!summary}}
{{!!return_type}} {{!!name}}({{!!parameters}}) {
}




I'd like to propose, further, that this functionality be available for 
attributes.  This will eliminate several situations where it's necessary to 
use macros or $wikify, replacing these mechanisms with a more efficient, 
briefer, pre-parsable expression.  As noted by Jeremy, one of the most 
common user mistakes involves expecting wiki syntax inside quoted strings 
(or imported via macro/transclusion) to work in an attribute...  Things 
like title="my name is {{!!name}}".  I see very few reasons (aside from 
compatibility) why they shouldn't work as expected!

A conservative approach could mandate a double-equals syntax or an 
alternative quotation style for wikified attributes:

 

...Though if I'm to be honest I would prefer to see all attributes parsed 
by default with an alternative syntax for "direct" values.  Perhaps I'm too 
radical.  :)

I don't think wikification needs to change much to implement this — another 
parse mode, with a smaller ruleset.  The tricker part of the implementation 
is getting widgets to parse imported text according to the proper rules.  
To that end, I suggest that the namespace variable could be used to 
determine parsing behavior for imported text.  In the future this might 
also be leveraged to keep inappropriate elements out of  and  
namespaces...


With these changes, I believe TiddlyWiki's logic would become more powerful 
and, crucially, more consistent, helping newcomers to learn it with fewer 
"gotchas" and making it simpler for experts like me to build complicated 
constructs.  I also think it's an essential piece of the puzzle for 
evolving TiddlyWiki beyond macro dependency.

Interested to hear others' thoughts.

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/7e867e78-da41-466a-b9ab-754439100af2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Lists of numbers in core tiddlers

[Simon Huber]

Hey Evan,

oh man, it's really worth looking a bit deeper into what you made there - 
this is perfect,
thank you for pointing this out!

Am Donnerstag, 11. Januar 2018 18:21:09 UTC+1 schrieb Evan Balster:
>
> Hey, Simon —
>
> See my range operator:  
> http://evanbalster.com/tiddlywiki/formulas.html#range%20Operator
>
> eg.  [range[-100,100,.5]] produces -100, -99.5, -99 ... 99, 99.5, 100.
>
> It generates evenly-spaced numbers in an arbitrary range, and can deal 
> with whatever step-size you're interested in.  The numbers produced never 
> have more decimal points than the arguments and the range is always 
> inclusive.
>
> This is a more robust and high-performance implementation than can be 
> managed with TiddlyWiki's built-in mechanisms.  It's part of my math & 
> formulas plugin (which I'm designing as a "universal" computation plugin) 
> but it can be used separately.  See the attached JSON file.
>
>  
now this is gonna be my new allrounder

thanks, Evan!

all the best,
Simon

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/3cb3a691-0414-45c4-95b2-0fd661abaa8d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Lists of numbers in core tiddlers

[Evan Balster]

Hey, Simon —

See my range operator:  
http://evanbalster.com/tiddlywiki/formulas.html#range%20Operator

eg.  [range[-100,100,.5]] produces -100, -99.5, -99 ... 99, 99.5, 100.

It generates evenly-spaced numbers in an arbitrary range, and can deal with 
whatever step-size you're interested in.  The numbers produced never have 
more decimal points than the arguments and the range is always inclusive.

This is a more robust and high-performance implementation than can be 
managed with TiddlyWiki's built-in mechanisms.  It's part of my math & 
formulas plugin (which I'm designing as a "universal" computation plugin) 
but it can be used separately.  See the attached JSON file.

On Thursday, 11 January 2018 08:04:06 UTC-6, Simon Huber wrote:
>
> @PMario,
>
> Am Donnerstag, 11. Januar 2018 14:26:11 UTC+1 schrieb PMario:
>>
>> On Thursday, January 11, 2018 at 1:38:33 PM UTC+1, Simon Huber wrote:
>>>
>>> negative and floating numbers with a certain limitation in accuracy 
>>> would also be great
>>>
>>
>> IMO negative numbers can be created with [addprefix[-]]
>>
>
> you're right, that works for me 
>
>>
>> floating point will be a problem. eg: 0 -> 99.99 will create about 10,000 
>> numbers. So users will create endless loops, without being aware. 
>> But you could use nested INTs.
>>
>
> that'd be ok 
>
>>
>> I was thinking about a syntax like: [create[1..100#04]] ... which would 
>> mean:
>>
>> start: 1 .. including 1
>> stop: 100 .. including 100
>>
>> format: #04 means
>>  use leading zeros
>>  use 4 digits
>>
>> which would result in 0001 ... 0100 with every number in between
>>
>> format: 1..100#03 means 001 ... 100
>>
>> format: 1..100 means 1 ... 100
>>
>> format: 33..55#03 means 033 ... 055
>>
>> and so on. 
>>
>
> I like this syntax, it would offer pretty good flexibility 
>
>>
>> Just some thoughts. No promises.
>>
>
> thanks, I can work around this with my own lists but with what you 
> proposed I imagine that all kinds of things could be made easier / possible
>
>>
>> The problem now is, that "start" stop may be variables. ... like 
>> [create..#] ... which is ugly, complex and the parser 
>> can't handle it at the moment.
>>
>
> I'd be totally fine with this if the parser could handle it, I'm lacking 
> ideas and knowledge tbh to propose something myself at this point
>  
> Grüße,
> Simon
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/bfe67493-9753-47b5-aacb-88e1048b32ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


$__plugins_ebalster_formula_filters_range.js.tid
Description: Binary data

Re: [twdev] Re: Creating a test rig for TiddlyWiki optimizations

[Evan Balster]

Hey, Mario —

I get the idea you might be thinking about a broader set of test 
functionality that I have in mind.  Can you give me some background?

Right now the only parts I can visualize are this sync and DOM monitoring 
mechanisms.

On Thursday, 11 January 2018 06:00:52 UTC-6, PMario wrote:
>
> On Wednesday, January 10, 2018 at 11:47:07 PM UTC+1, Jeremy Ruston wrote:
>>
>> I think that Evan is describing optimisations for the current core 
>> architecture, and wants to explore introducing the tests to ensure that we 
>> retain backwards compatibility. So, I don't think we need a separate 
>> organisation 
>>
>
> I think we do need a new organisation, because we need an automated CI/CD 
> system, which isn't possible with the existing structure. 
>
> I do want to have the possibility to create automated editions, if a 
> contributor with write access pushes to a new feature branch, for review.
> So reviewers can browse a working copy of the PR and experiment with it. 
> No initial manual work needed for reviewers. 
> I expect to create one edition per test. That's the only way to see the 
> differences per change. 
>
> (nor do we need a new organisation for TWX; it's one thing to talk about 
>> it, but we're nowhere near ready to start coding).
>>
>
> My proposal isn't about TWX. For me TWX also contains the "What if 
> ?" 
> thread, that I started.
>  
>
>> As Mario says, the issue with testing a lot of the current core code is 
>> the use of anonymous functions, so perhaps a component of this work would 
>> be PRs to refactor the relevant functions into named, exportable functions 
>> for testability.
>>
>
> I want to have fast iterations, that automatically distribute edtions to 
> github pages, with minimal manual interaction. .. I did show a proof of 
> concep t in Feb. 2016, which wasn't 
> possible to be used, since the tiddlywiki github homepage still points to 
> classic.tiddlywiki.com ... 
>
> This demo has been 2 years ago! ... I really don't want to wait an other 2 
> years to get the stuff going. ... 
>
> The exact same mechanisms could be used to create community maintained 
> documentation ... BUT all of these improvements are not possible with the 
> existing build process. 
>
> With a new organisation we could experiment and iterate with a new 
> structure, that is desperately needed, from my point of view. 
>
> just my thoughts
> mario
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/4d34bc28-84d9-4484-ac68-5c47fcc1a71b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Lists of numbers in core tiddlers

[Simon Huber]

@PMario,

Am Donnerstag, 11. Januar 2018 14:26:11 UTC+1 schrieb PMario:
>
> On Thursday, January 11, 2018 at 1:38:33 PM UTC+1, Simon Huber wrote:
>>
>> negative and floating numbers with a certain limitation in accuracy would 
>> also be great
>>
>
> IMO negative numbers can be created with [addprefix[-]]
>

you're right, that works for me 

>
> floating point will be a problem. eg: 0 -> 99.99 will create about 10,000 
> numbers. So users will create endless loops, without being aware. 
> But you could use nested INTs.
>

that'd be ok 

>
> I was thinking about a syntax like: [create[1..100#04]] ... which would 
> mean:
>
> start: 1 .. including 1
> stop: 100 .. including 100
>
> format: #04 means
>  use leading zeros
>  use 4 digits
>
> which would result in 0001 ... 0100 with every number in between
>
> format: 1..100#03 means 001 ... 100
>
> format: 1..100 means 1 ... 100
>
> format: 33..55#03 means 033 ... 055
>
> and so on. 
>

I like this syntax, it would offer pretty good flexibility 

>
> Just some thoughts. No promises.
>

thanks, I can work around this with my own lists but with what you proposed 
I imagine that all kinds of things could be made easier / possible

>
> The problem now is, that "start" stop may be variables. ... like 
> [create..#] ... which is ugly, complex and the parser 
> can't handle it at the moment.
>

I'd be totally fine with this if the parser could handle it, I'm lacking 
ideas and knowledge tbh to propose something myself at this point
 
Grüße,
Simon

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/9a9a6519-383c-40e8-9699-799539a3170b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Lists of numbers in core tiddlers

[PMario]

On Thursday, January 11, 2018 at 1:38:33 PM UTC+1, Simon Huber wrote:
>
> negative and floating numbers with a certain limitation in accuracy would 
> also be great
>

IMO negative numbers can be created with [addprefix[-]] 

floating point will be a problem. eg: 0 -> 99.99 will create about 10,000 
numbers. So users will create endless loops, without being aware. 
But you could use nested INTs. 

I was thinking about a syntax like: [create[1..100#04]] ... which would 
mean:

start: 1 .. including 1
stop: 100 .. including 100

format: #04 means
 use leading zeros
 use 4 digits

which would result in 0001 ... 0100 with every number in between

format: 1..100#03 means 001 ... 100

format: 1..100 means 1 ... 100

format: 33..55#03 means 033 ... 055

and so on. 

Just some thoughts. No promises.

The problem now is, that "start" stop may be variables. ... like 
[create..#] ... which is ugly, complex and the parser 
can't handle it at the moment.
-m



-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/0fc29ee8-b46a-4205-9146-d4c1ea3dc8b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Lists of numbers in core tiddlers

[Simon Huber]

negative and floating numbers with a certain limitation in accuracy would 
also be great

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/cadf05d3-d82b-4c1d-8dd2-a8543ea1822c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Lists of numbers in core tiddlers

[Simon Huber]

Hello @PMario,

Am Donnerstag, 11. Januar 2018 13:13:33 UTC+1 schrieb PMario:
>
> Hi Simon,
>
> I think that's a very specific usecase and therefor plugin territory. ... 
> Can you describe your usage of this list a bit closer. ... So we may be 
> able, to dynamically create "virtual" lists, that don't stop at 
>

 is the max i could store in a field. virtual lists would be much 
better, that would be a nice feature, it would add a lot of flexibility I 
think

   - for listing like [listlimit[100]] when I want to do something 
   100 times
   - for getting the next (previous) value of a variable with 
   [next[numbertiddler]] if I need to count something that happens 
   various times
   - various other things with lists

it would be great if something like filter="[[1 - 100]]" could be done and 
also [next[1 - 100]]


> My first question was: "What if I need 1?" ... 
>

doesn't happen in my cases but you're absolutely right, it's limiting

>
> -m
>

thanks for your reply,
Simon 

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/369fdf00-e335-4c42-98a7-0a2bba89eb85%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Lists of numbers in core tiddlers

[PMario]

Hi Simon,

I think that's a very specific usecase and therefor plugin territory. ... 
Can you describe your usage of this list a bit closer. ... So we may be 
able, to dynamically create "virtual" lists, that don't stop at . 

My first question was: "What if I need 1?" ... 

-m



-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/e319331d-8f94-4a42-b51b-7b02bb66cd81%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [twdev] Re: Creating a test rig for TiddlyWiki optimizations

[PMario]

On Wednesday, January 10, 2018 at 11:47:07 PM UTC+1, Jeremy Ruston wrote:
>
> I think that Evan is describing optimisations for the current core 
> architecture, and wants to explore introducing the tests to ensure that we 
> retain backwards compatibility. So, I don't think we need a separate 
> organisation 
>

I think we do need a new organisation, because we need an automated CI/CD 
system, which isn't possible with the existing structure. 

I do want to have the possibility to create automated editions, if a 
contributor with write access pushes to a new feature branch, for review.
So reviewers can browse a working copy of the PR and experiment with it. No 
initial manual work needed for reviewers. 
I expect to create one edition per test. That's the only way to see the 
differences per change. 

(nor do we need a new organisation for TWX; it's one thing to talk about 
> it, but we're nowhere near ready to start coding).
>

My proposal isn't about TWX. For me TWX also contains the "What if 
?" 
thread, that I started.
 

> As Mario says, the issue with testing a lot of the current core code is 
> the use of anonymous functions, so perhaps a component of this work would 
> be PRs to refactor the relevant functions into named, exportable functions 
> for testability.
>

I want to have fast iterations, that automatically distribute edtions to 
github pages, with minimal manual interaction. .. I did show a proof of 
concep t in Feb. 2016, which wasn't 
possible to be used, since the tiddlywiki github homepage still points to 
classic.tiddlywiki.com ... 

This demo has been 2 years ago! ... I really don't want to wait an other 2 
years to get the stuff going. ... 

The exact same mechanisms could be used to create community maintained 
documentation ... BUT all of these improvements are not possible with the 
existing build process. 

With a new organisation we could experiment and iterate with a new 
structure, that is desperately needed, from my point of view. 

just my thoughts
mario

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/d6dd854e-5c61-4cd6-af48-5c1890ec5fca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Lists of numbers in core tiddlers

[Simon Huber]

Hello, I'd like to discuss if it wouldn't be very useful to have some 
tiddlers in the core that hold lists with numbers

I'm using a normal list with all integers from 1 -  quite often to get 
the next or previous value for variables and I don't want to use math 
plugins for such easy things
I'm using such a list 
on 
http://muritest.tiddlyspot.com/#%24%3A%2Fplugins%2FBTC%2Ftiddly-touch%2Flists%2Fnumbers
I guess there could be other math-related usecases when I think about 
logarithmic tables making multiplication an addition of logarithms... (like 
when mankind didn't have calculators but logarithmic tables)

what do you think about such lists in the tiddlywiki core?

all the best,
Simon

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/8a387e3f-41ca-4e47-beac-537eac140606%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.