Hi Saq, This plugin is very nice. Thanks for the contribution!
It runs beautifully - I had to make a small change in the JS to make it work on my TiddlyServer installation though! I changed "var uploadURL = '/api/upload';" to "var uploadURL = 'api/upload';" Because the initial / was making it call the api on the home directory instead of the subdirectory. Once I did that it worked perfectly! Thanks again, Adithya On Friday, April 2, 2021 at 11:28:28 AM UTC-6 saq.i...@gmail.com wrote: > That's good to hear Mark. I wasn't looking forward to having to debug this > if it was still an issue with the newer code. > > I'm still actively working on this, in fact I put in a little time on it > even today, so there isn't any documentation as things are apt to change as > I explore different implementation approaches. > > One caveat to be aware of: The way the code now works is that it saves the > imported image as a normal base 64 tiddler in the wiki tiddler store in > memory, while uploading it to the files folder in the background. Once it > has been uploaded and successfully saved to the files folder, the base 64 > image tiddler is converted into a _canonical_uri tiddler. All of this > happens without any user interaction and doesn't seem to disrupt normal > usage in any manner. This workflow guards against losing data because of > any issues writing the file to the files folder, and also works around the > fact that import process in TW is designed to be synchronous. > > Working on a local node.js server it is all almost instantaneous and you > don't notice that the tiddler is first saved as base 64, but just a heads > up that this is what happens in the background. On a remote server over a > slow connection, or with very large media files, this could be an issue as > the normal sync mechanism may start saving the base 64 tiddler while the > file is uploading to the files folder. However, best dealing with that may > require some core modifications and support in the file saving adaptor. The > long term goal is to identify what changes would be needed and hopefully > get support for saving to the files folder in the core. > > Not present in the version I uploaded for you this morning but now > implemented, is a toolbar button and a tm-externalise-tiddler message that > can be used to convert an existing base 64 image into a _canonical_uri > tiddler with the actual image saved in the files folder. If that's > something you need, I can publish that. > > Saq > > On Friday, April 2, 2021 at 7:09:06 PM UTC+2 Mark S. wrote: > >> Ok! That seems to fix it -- it creates zero byte stub files! (Linux, >> Chromium). >> >> Thank you! >> On Friday, April 2, 2021 at 12:11:47 AM UTC-7 saq.i...@gmail.com wrote: >> >>> Hi Mark, >>> >>> Very peculiar! This suggests that there is something about my >>> environment that makes this work, whereas it otherwise fails. I've been >>> using this as part of a highly customized TW, where the image import is via >>> a custom UI and part of a larger workflow, so it is possible that there is >>> some other part of it that is needed for this to work properly. >>> >>> However, I did realize that when I was testing yesterday to recreate >>> your problem in a fresh wiki, I made the mistake of using the version of >>> the plugin that I have been working on the last week. Since this is the >>> code base I will be working on going forwards, it would be very helpful to >>> know if this version still gives you the same issue on 5.1.23, before I try >>> to do a deeper dive and identify and resolve the issue. >>> >>> https://saqimtiaz.github.io/sq-tw/temp/import-to-external-file.html >>> >>> Thanks again for the feedback and time invested in testing this. >>> Regards, >>> >>> Saq >>> >>> On Friday, April 2, 2021 at 4:00:40 AM UTC+2 Mark S. wrote: >>> >>>> I just booted into Windows 10 and experienced the same error. In the >>>> case of one of the image files, it corrected itself after I temporarily >>>> changed the text in the canonical_uri tiddler. But the other two images >>>> that I imported continued to have a non-zero byte file (1kib) in the >>>> tiddlers directory. In addition, there was an oddity that the imported >>>> image files have no mod dates, and so don't appear in the "recent" list. >>>> >>>> So I've tried it now on 3 file systems, 3 operating systems and two >>>> different physical machines. >>>> >>>> Thanks! >>>> >>>> On Thursday, April 1, 2021 at 2:49:14 PM UTC-7 saq.i...@gmail.com >>>> wrote: >>>> >>>>> Hi Mark, appreciate the additional testing and feedback. It at least >>>>> helps rule out some potential causes. >>>>> >>>>> I'm still quite perplexed as I have been using this for several >>>>> months now, with no issues across several server restarts. I consistently >>>>> get 0 bytes image files in the tiddlers folder, which I presume is the >>>>> expected behaviour. >>>>> >>>>> Anyway, I've actually worked on this some the last week, including >>>>> rewriting portions and adding better redundancy and error handling in >>>>> case >>>>> of write/upload errors. I'll publish that once I get the chance, it will >>>>> be >>>>> interesting to see if the problem you are encountering still persists. >>>>> >>>>> On Thursday, April 1, 2021 at 11:38:39 PM UTC+2 Mark S. wrote: >>>>> >>>>>> Just to be clear, the 6 byte images were created using the kit from >>>>>> your sandbox. >>>>>> >>>>>> Just in case it was an OS thing, I fired up my old Windows 7 machine, >>>>>> created a new node.js instance (5.1.21 is what I have there), dragged >>>>>> and >>>>>> dropped the items from your sandbox including the plugin, restarted the >>>>>> instance. Then dropped in an image. It made the _canonical_uri tiddler. >>>>>> I >>>>>> was able to insert the image as img[[ wikitext. Then I restarted the >>>>>> instance. Now the img[] image is replaced by the "missing" icon. On the >>>>>> disk, the image file in the tiddlers directory is 1k rather than 6bytes. >>>>>> (1k is too small to be the image size, BTW). >>>>>> >>>>>> Then, just in case it related to TW version, I upgraded to 5.1.23 and >>>>>> tested again. Once again it makes 1k files. But they won't display via >>>>>> img[] after restart. >>>>>> >>>>>> So the same problem appears to exist when making files on Windows and >>>>>> on Linux. The only difference is that in one system it's making 6byte >>>>>> image >>>>>> files and in the other 1k byte files. Either way the image doesn't >>>>>> display >>>>>> with img[] after the node instance has been restarted. >>>>>> >>>>>> On Thursday, April 1, 2021 at 12:47:31 PM UTC-7 saq.i...@gmail.com >>>>>> wrote: >>>>>> >>>>>>> Understood. As I'm unable to reproduce the problem of the 6 byte >>>>>>> images being created, there isn't much I can do to further debug this >>>>>>> without more information. >>>>>>> >>>>>>> Saq >>>>>>> On Thursday, April 1, 2021 at 9:40:28 PM UTC+2 Mark S. wrote: >>>>>>> >>>>>>>> I explained that poorly. The way that I created the png files >>>>>>>> (actually using your external files tool) generated 6 byte file >>>>>>>> images. >>>>>>>> *These* images do not display correctly. >>>>>>>> >>>>>>>> The zero-byte images generated when making external uri's by hand >>>>>>>> *do* display correctly. >>>>>>>> >>>>>>>> So to be clear, AFAIK, there is nothing wrong with img[] >>>>>>>> nomenclature under node.js. If there was still a way to mark an issue >>>>>>>> as >>>>>>>> resolved, I would do that. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> On Thursday, April 1, 2021 at 11:24:56 AM UTC-7 saq.i...@gmail.com >>>>>>>> wrote: >>>>>>>> >>>>>>>>> It would helpful to determine what the default behaviour of TW on >>>>>>>>> node.js is for _canonical_uri tiddlers and saving to file. >>>>>>>>> >>>>>>>>> On Windows 10, latest Chrome and using latest pre-release with the >>>>>>>>> server edition and no plugins, I tried manually creating a tiddler >>>>>>>>> with: >>>>>>>>> - title: test.png >>>>>>>>> - type: image/png, >>>>>>>>> - and an arbitrary _canonical_uri value of files/test.png >>>>>>>>> >>>>>>>>> In the tiddlers directory I get a 0 bytes file test.png and >>>>>>>>> test.png.meta. >>>>>>>>> I am not sure if the 0 byte expected behaviour on node.js for >>>>>>>>> canonical_uri tiddlers or something new in 5.1.23 >>>>>>>>> >>>>>>>>> Using [img[test.png]] in another tiddler, the img is correctly >>>>>>>>> resolved to have a src attribute files/test.png, both before and >>>>>>>>> after >>>>>>>>> restarting the node server. >>>>>>>>> >>>>>>>>> So I cannot reproduce the problem. It could be OS specific. >>>>>>>>> >>>>>>>>> It is worth considering that 5.1.23 had significant changes to the >>>>>>>>> file saving code, so there may have been a regression. >>>>>>>>> Also I am using the latest pre-release, which might be relevant as >>>>>>>>> at least one bugfix since 5.1.23 seems to deal with missing text >>>>>>>>> fields of >>>>>>>>> tiddlers, see >>>>>>>>> https://github.com/Jermolene/TiddlyWiki5/pull/5416/files >>>>>>>>> >>>>>>>>> If you can confirm that creating a _canonical_uri tiddler by hand >>>>>>>>> works well for you on node.js, that will help narrow down whether >>>>>>>>> this is a >>>>>>>>> core issues or a plugin issue. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Saq >>>>>>>>> On Thursday, April 1, 2021 at 7:43:50 PM UTC+2 Saq Imtiaz wrote: >>>>>>>>> >>>>>>>>>> Hmm. That's interesting. I'm seeing 0 byte image files but the >>>>>>>>>> images do otherwise load correctly. >>>>>>>>>> What OS and browser are you using? >>>>>>>>>> >>>>>>>>>> It would also be interesting to check how TW on node.js on your >>>>>>>>>> system, with no plugins saves _canonical_uri tiddlers. Are there any >>>>>>>>>> image >>>>>>>>>> files created at all, or just a tid file? >>>>>>>>>> >>>>>>>>>> I've been working on this here and there over the last week, so >>>>>>>>>> with a few more details I should be able to address this. >>>>>>>>>> >>>>>>>>>> On Thursday, April 1, 2021 at 7:24:36 PM UTC+2 Mark S. wrote: >>>>>>>>>> >>>>>>>>>>> The problem turned out to be that the way the _canonical_uri >>>>>>>>>>> files were created was also creating 6byte image files in the >>>>>>>>>>> tiddler >>>>>>>>>>> folder which apparently confuses the system into thinking that it >>>>>>>>>>> is >>>>>>>>>>> dealing with base64 image files. >>>>>>>>>>> >>>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/463d01c1-e5e4-49c0-a060-df3081df6b18n%40googlegroups.com.
