Thank you. I've created a separate thread and posted the latest version here: https://groups.google.com/g/tiddlywiki/c/T0MP1Adzzk0
On Sunday, April 4, 2021 at 5:35:26 AM UTC+2 joshua....@gmail.com wrote: > This is very cool stuff Saq. Mahalo for sharing your experiments. > > Best, > Joshua Fontany > > On Friday, April 2, 2021 at 10:28:28 AM UTC-7 saq.i...@gmail.com wrote: > >> That's good to hear Mark. I wasn't looking forward to having to debug >> this if it was still an issue with the newer code. >> >> I'm still actively working on this, in fact I put in a little time on it >> even today, so there isn't any documentation as things are apt to change as >> I explore different implementation approaches. >> >> One caveat to be aware of: The way the code now works is that it saves >> the imported image as a normal base 64 tiddler in the wiki tiddler store in >> memory, while uploading it to the files folder in the background. Once it >> has been uploaded and successfully saved to the files folder, the base 64 >> image tiddler is converted into a _canonical_uri tiddler. All of this >> happens without any user interaction and doesn't seem to disrupt normal >> usage in any manner. This workflow guards against losing data because of >> any issues writing the file to the files folder, and also works around the >> fact that import process in TW is designed to be synchronous. >> >> Working on a local node.js server it is all almost instantaneous and you >> don't notice that the tiddler is first saved as base 64, but just a heads >> up that this is what happens in the background. On a remote server over a >> slow connection, or with very large media files, this could be an issue as >> the normal sync mechanism may start saving the base 64 tiddler while the >> file is uploading to the files folder. However, best dealing with that may >> require some core modifications and support in the file saving adaptor. The >> long term goal is to identify what changes would be needed and hopefully >> get support for saving to the files folder in the core. >> >> Not present in the version I uploaded for you this morning but now >> implemented, is a toolbar button and a tm-externalise-tiddler message that >> can be used to convert an existing base 64 image into a _canonical_uri >> tiddler with the actual image saved in the files folder. If that's >> something you need, I can publish that. >> >> Saq >> >> On Friday, April 2, 2021 at 7:09:06 PM UTC+2 Mark S. wrote: >> >>> Ok! That seems to fix it -- it creates zero byte stub files! (Linux, >>> Chromium). >>> >>> Thank you! >>> On Friday, April 2, 2021 at 12:11:47 AM UTC-7 saq.i...@gmail.com wrote: >>> >>>> Hi Mark, >>>> >>>> Very peculiar! This suggests that there is something about my >>>> environment that makes this work, whereas it otherwise fails. I've been >>>> using this as part of a highly customized TW, where the image import is >>>> via >>>> a custom UI and part of a larger workflow, so it is possible that there is >>>> some other part of it that is needed for this to work properly. >>>> >>>> However, I did realize that when I was testing yesterday to recreate >>>> your problem in a fresh wiki, I made the mistake of using the version of >>>> the plugin that I have been working on the last week. Since this is the >>>> code base I will be working on going forwards, it would be very helpful to >>>> know if this version still gives you the same issue on 5.1.23, before I >>>> try >>>> to do a deeper dive and identify and resolve the issue. >>>> >>>> https://saqimtiaz.github.io/sq-tw/temp/import-to-external-file.html >>>> >>>> Thanks again for the feedback and time invested in testing this. >>>> Regards, >>>> >>>> Saq >>>> >>>> On Friday, April 2, 2021 at 4:00:40 AM UTC+2 Mark S. wrote: >>>> >>>>> I just booted into Windows 10 and experienced the same error. In the >>>>> case of one of the image files, it corrected itself after I temporarily >>>>> changed the text in the canonical_uri tiddler. But the other two images >>>>> that I imported continued to have a non-zero byte file (1kib) in the >>>>> tiddlers directory. In addition, there was an oddity that the imported >>>>> image files have no mod dates, and so don't appear in the "recent" list. >>>>> >>>>> So I've tried it now on 3 file systems, 3 operating systems and two >>>>> different physical machines. >>>>> >>>>> Thanks! >>>>> >>>>> On Thursday, April 1, 2021 at 2:49:14 PM UTC-7 saq.i...@gmail.com >>>>> wrote: >>>>> >>>>>> Hi Mark, appreciate the additional testing and feedback. It at least >>>>>> helps rule out some potential causes. >>>>>> >>>>>> I'm still quite perplexed as I have been using this for several >>>>>> months now, with no issues across several server restarts. I >>>>>> consistently >>>>>> get 0 bytes image files in the tiddlers folder, which I presume is the >>>>>> expected behaviour. >>>>>> >>>>>> Anyway, I've actually worked on this some the last week, including >>>>>> rewriting portions and adding better redundancy and error handling in >>>>>> case >>>>>> of write/upload errors. I'll publish that once I get the chance, it will >>>>>> be >>>>>> interesting to see if the problem you are encountering still persists. >>>>>> >>>>>> On Thursday, April 1, 2021 at 11:38:39 PM UTC+2 Mark S. wrote: >>>>>> >>>>>>> Just to be clear, the 6 byte images were created using the kit from >>>>>>> your sandbox. >>>>>>> >>>>>>> Just in case it was an OS thing, I fired up my old Windows 7 >>>>>>> machine, created a new node.js instance (5.1.21 is what I have there), >>>>>>> dragged and dropped the items from your sandbox including the plugin, >>>>>>> restarted the instance. Then dropped in an image. It made the >>>>>>> _canonical_uri tiddler. I was able to insert the image as img[[ >>>>>>> wikitext. >>>>>>> Then I restarted the instance. Now the img[] image is replaced by the >>>>>>> "missing" icon. On the disk, the image file in the tiddlers directory >>>>>>> is 1k >>>>>>> rather than 6bytes. (1k is too small to be the image size, BTW). >>>>>>> >>>>>>> Then, just in case it related to TW version, I upgraded to 5.1.23 >>>>>>> and tested again. Once again it makes 1k files. But they won't display >>>>>>> via >>>>>>> img[] after restart. >>>>>>> >>>>>>> So the same problem appears to exist when making files on Windows >>>>>>> and on Linux. The only difference is that in one system it's making >>>>>>> 6byte >>>>>>> image files and in the other 1k byte files. Either way the image >>>>>>> doesn't >>>>>>> display with img[] after the node instance has been restarted. >>>>>>> >>>>>>> On Thursday, April 1, 2021 at 12:47:31 PM UTC-7 saq.i...@gmail.com >>>>>>> wrote: >>>>>>> >>>>>>>> Understood. As I'm unable to reproduce the problem of the 6 byte >>>>>>>> images being created, there isn't much I can do to further debug this >>>>>>>> without more information. >>>>>>>> >>>>>>>> Saq >>>>>>>> On Thursday, April 1, 2021 at 9:40:28 PM UTC+2 Mark S. wrote: >>>>>>>> >>>>>>>>> I explained that poorly. The way that I created the png files >>>>>>>>> (actually using your external files tool) generated 6 byte file >>>>>>>>> images. >>>>>>>>> *These* images do not display correctly. >>>>>>>>> >>>>>>>>> The zero-byte images generated when making external uri's by hand >>>>>>>>> *do* display correctly. >>>>>>>>> >>>>>>>>> So to be clear, AFAIK, there is nothing wrong with img[] >>>>>>>>> nomenclature under node.js. If there was still a way to mark an issue >>>>>>>>> as >>>>>>>>> resolved, I would do that. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> On Thursday, April 1, 2021 at 11:24:56 AM UTC-7 saq.i...@gmail.com >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> It would helpful to determine what the default behaviour of TW on >>>>>>>>>> node.js is for _canonical_uri tiddlers and saving to file. >>>>>>>>>> >>>>>>>>>> On Windows 10, latest Chrome and using latest pre-release with >>>>>>>>>> the server edition and no plugins, I tried manually creating a >>>>>>>>>> tiddler with: >>>>>>>>>> - title: test.png >>>>>>>>>> - type: image/png, >>>>>>>>>> - and an arbitrary _canonical_uri value of files/test.png >>>>>>>>>> >>>>>>>>>> In the tiddlers directory I get a 0 bytes file test.png and >>>>>>>>>> test.png.meta. >>>>>>>>>> I am not sure if the 0 byte expected behaviour on node.js for >>>>>>>>>> canonical_uri tiddlers or something new in 5.1.23 >>>>>>>>>> >>>>>>>>>> Using [img[test.png]] in another tiddler, the img is correctly >>>>>>>>>> resolved to have a src attribute files/test.png, both before and >>>>>>>>>> after >>>>>>>>>> restarting the node server. >>>>>>>>>> >>>>>>>>>> So I cannot reproduce the problem. It could be OS specific. >>>>>>>>>> >>>>>>>>>> It is worth considering that 5.1.23 had significant changes to >>>>>>>>>> the file saving code, so there may have been a regression. >>>>>>>>>> Also I am using the latest pre-release, which might be relevant >>>>>>>>>> as at least one bugfix since 5.1.23 seems to deal with missing text >>>>>>>>>> fields >>>>>>>>>> of tiddlers, see >>>>>>>>>> https://github.com/Jermolene/TiddlyWiki5/pull/5416/files >>>>>>>>>> >>>>>>>>>> If you can confirm that creating a _canonical_uri tiddler by hand >>>>>>>>>> works well for you on node.js, that will help narrow down whether >>>>>>>>>> this is a >>>>>>>>>> core issues or a plugin issue. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Saq >>>>>>>>>> On Thursday, April 1, 2021 at 7:43:50 PM UTC+2 Saq Imtiaz wrote: >>>>>>>>>> >>>>>>>>>>> Hmm. That's interesting. I'm seeing 0 byte image files but the >>>>>>>>>>> images do otherwise load correctly. >>>>>>>>>>> What OS and browser are you using? >>>>>>>>>>> >>>>>>>>>>> It would also be interesting to check how TW on node.js on your >>>>>>>>>>> system, with no plugins saves _canonical_uri tiddlers. Are there >>>>>>>>>>> any image >>>>>>>>>>> files created at all, or just a tid file? >>>>>>>>>>> >>>>>>>>>>> I've been working on this here and there over the last week, so >>>>>>>>>>> with a few more details I should be able to address this. >>>>>>>>>>> >>>>>>>>>>> On Thursday, April 1, 2021 at 7:24:36 PM UTC+2 Mark S. wrote: >>>>>>>>>>> >>>>>>>>>>>> The problem turned out to be that the way the _canonical_uri >>>>>>>>>>>> files were created was also creating 6byte image files in the >>>>>>>>>>>> tiddler >>>>>>>>>>>> folder which apparently confuses the system into thinking that it >>>>>>>>>>>> is >>>>>>>>>>>> dealing with base64 image files. >>>>>>>>>>>> >>>>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/939d7697-e78d-44ed-a286-86d1886adc70n%40googlegroups.com.
