[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
Hi Rick, On Wednesday, October 22, 2014 11:35:14 PM UTC+2, Rick Williams wrote: There is still the option, to create a completely static version of all tiddlers, no javascript involved. This could be served entirely by the nginx server. So every tiddler is a single page. There would be the need for a different layout, because at the moment the static version uses the same layout as the js version. Can you point me instructions for deployment of a static environment built from a standard TW5 wiki? If you have a look at the tw5.com tiddlywiki.info file [1], you can see the static build section. The first step/line: .. creates a static.html file, that is like a startpage. So imo it's important to have a TOC there :) second line: .. creates an alltiddlers.html. ... As the name says the whole TW in one tiddler (no js) So imo its nice to use the browser search third line: .. creates all tiddlers as single tiddlers into a static/ directory forth line: .. creates the needed CSS file. I'm using the exact same settings [2], to create a static version of the german translation. So you should be able to just copy paste the stuff into your tiddlywiki.info to test it. build static command: node tiddlywiki.js yourEdition --build static build all command: node tiddlywiki.js yourEdition --build -- To build your own static layout you have to have a look at the $:/core/templates/ tiddlers [3]. So it would be possible to create your own branding :) Template handling can be a bit tricky. .. But its all html5 + tw5 wikitext syntax. Used elements are html, macro definitions, filters and transclusions. Docs can be found at tiddlywiki.com But imo this will be the second step. I also talked with Jeremy at the last hangout #65 [4] about the difference between image tiddler lazy loading and images using _canonical_uri The link starts at the position and the discussion is about 10 minutes. There will be some background info, if you want to see it. Tiddler lazy loading would have been a solution for your too but I simply forgot about it. After our last discussion I think this mechanism fits better to your workflow, and would eliminate the conversion from fat image tiddlers to skinny tiddlers using the _canonical_uri. images can be fat but would be loaded in a second step. ... see the video have fun! mario PS: any success on loading speed? Please keep us updated. [1] https://github.com/Jermolene/TiddlyWiki5/blob/master/editions/tw5.com/tiddlywiki.info#L54 [2] https://github.com/Jermolene/TiddlyWiki5/blob/master/editions/de-AT/tiddlywiki.info#L25 [3] https://github.com/Jermolene/TiddlyWiki5/tree/master/core/templates [4] https://www.youtube.com/watch?v=hJ4feiD6iuYfeature=player_detailpage#t=4144 -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
build static command: node tiddlywiki.js yourEdition --build static build all command: node tiddlywiki.js yourEdition --build don't forget to define --output directory -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
Hi Mario, You asked a few questions, I'd like to try and respond: On Tuesday, October 21, 2014 3:55:12 AM UTC-4, PMario wrote: Can you describe your use case a bit closer, if that's possible. So I can see, if I should recommend it, or not :) - How many different wiki's do you need? I expect to have separate wiki's for different functions. They would be distinct, and not share any data. So, they may even be on different VMs. Although, perhaps we can run more than one on a server. - How many users? I'm expecting a relatively small number of users. Perhaps only 40 or so individuals and perhaps 5-15 concurrent. - Do you have many different editors? - or just consumers with a view content editors / creators? The use case scenario is primarily for IT staff. We have several areas that document their processes for how to handle particular issues, problems, or steps for implementation of service requests. The wiki would be used to describe these processes. We then have a system that presents real-time alarms and events to operations staff. The idea is to have the alarm relate to a process such that a staff member can 'right-click' an alarm/event and go directly to the wiki page (tiddler) that defines the process they should follow. The processes (tiddlers) will be maintained and edited primarily by tier-2 and -3 support staff. So, for this particular use case, we envision: - about 200 or so tiddlers equating to processes. - about 5-10 concurrent viewers - about 30 content creators but only expect about 10 maximum concurrent editors and they'd likely be editing different things. - There is still the option, to create a completely static version of all tiddlers, no javascript involved. This could be served entirely by the nginx server. So every tiddler is a single page. There would be the need for a different layout, because at the moment the static version uses the same layout as the js version. Can you point me instructions for deployment of a static environment built from a standard TW5 wiki? Thanks, again, Rick -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
On Tuesday, October 21, 2014 12:03:59 AM UTC+2, Rick Williams wrote: Did you do the import step? So import the small index.html into the server. You could select all. ... You did a backup right? I did do this later. But I still winder if it was really necessary. The image files were already in the images directory. Yes, the images are in the image directory already. But the server version doesn't know about it. The whole build step creates the index.html file. It creates the images It changes the image tiddlers (_canonical_uri, empty text) ... but it doesn't touch the image.png.tid or .jpg.tid files. So they are still big. If the server starts, it still uses those files. If the server is started and the index.html is imported, the .png.tid files are small. So it seemed to work right in your configuration according to the ls tiddlers/ ouput. Tiddlywiki server doesn't use the build info. So it doesn't know about the images directory to store new images. The TW server at the moment is a very very basic server to make TW development easier. OK, I see. Does this mean I need to rerun the 'build' step periodically to get new image files placed in the right directory? hmmm, That's why I wrote: I'm not sure, what happens if you do the build several times. If the tiddlers already have the _canonical_uri ... So take care. Since the (small) images are still tagged with external-image, they are touched everytime you run the build. So you need to check if everything is right after a new build run. I just didn't test this case, so I don't know. Also the import step is needed every time you did import images with the TW drag and drop action. The TW binary file handling isn't optimized for this workflow at the moment. So an alternative version would be to let nginx handle the file upload. I did a fast search and found this: https://coderwall.com/p/swgfvw I did a search: https://www.google.at/search?q=file+upload+ui+for+nginxtbs=qdr:y and found: http://www.howtoforge.com/displaying-upload-progress-with-nginx-on-debian-wheezy If nginx does the upload handling, there is the problem, that we need to tell TW about the new file. ... As I wrote. The workflow needs to be improved for binary file handling. It also brings up another question about the future plans for the system. Do you indeed plan to have a more robust server implementation? I need to pass this on on to @Jeremy. After all, the value is in the sharing of information among many. I have no use for a personal wiki. One reason, that makes TiddlyWiki special and unique, is its focus on beeing an all-in-one single file Wiki. But for your configuration, this is actually a weak spot. You are right, _one_ value of a wiki is in sharing it :) And there are plenty of requests, in making the sharing easier. But most TW users _don't_ want to touch a server. They request file based sharing options. ... But in the TW hangouts we discussed making server more robust. There is a hangout today: https://plus.google.com/u/0/events/cd8h7qbbtethtk44i98cq2fmmis There is a very robust backend for TiddlyWiki. It's called TiddlyWeb. It's a python app, with a lot of options. In its basic configuration it uses a similar file based store as the TW node server. If you are python guru it should be relatively easy for you to handle it. If not, it can be overwhelming. Can you describe your usecase a bit closer, if that's possible. So I can see, if I should recommend it, or not :) - How many different wiki's do you need? - How many users? - Do you have many different editors? - or just consumers with a view content editors / creators? - There is still the option, to create a completely static version of all tiddlers, no javascript involved. This could be served entirely by the nginx server. So every tiddler is a single page. There would be the need for a different layout, because at the moment the static version uses the same layout as the js version. Thanks again for your help. you are welcome The load time is still pretty slow - 38 seconds or so. It does take pretty long over the network to serve the data. I'm turning my attention now to speeding that up. I have a NAT in the way that I maybe can avoid. ok, let us know if it worked out. Thx for posting your nginx configurations, it will help others, that want to use TW with a similar configuration. have fun! mario -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
I've added a ticket for properly enabling external images with the client-server configuration: https://github.com/Jermolene/TiddlyWiki5/issues/1000 (yay! 1,000 bugs!) The other route you may be able to explore at the moment is to use the existing lazy image support, along with caching the .html file in nginx. Best wishes Jeremy. On Tue, Oct 21, 2014 at 8:55 AM, PMario pmari...@gmail.com wrote: On Tuesday, October 21, 2014 12:03:59 AM UTC+2, Rick Williams wrote: Did you do the import step? So import the small index.html into the server. You could select all. ... You did a backup right? I did do this later. But I still winder if it was really necessary. The image files were already in the images directory. Yes, the images are in the image directory already. But the server version doesn't know about it. The whole build step creates the index.html file. It creates the images It changes the image tiddlers (_canonical_uri, empty text) ... but it doesn't touch the image.png.tid or .jpg.tid files. So they are still big. If the server starts, it still uses those files. If the server is started and the index.html is imported, the .png.tid files are small. So it seemed to work right in your configuration according to the ls tiddlers/ ouput. Tiddlywiki server doesn't use the build info. So it doesn't know about the images directory to store new images. The TW server at the moment is a very very basic server to make TW development easier. OK, I see. Does this mean I need to rerun the 'build' step periodically to get new image files placed in the right directory? hmmm, That's why I wrote: I'm not sure, what happens if you do the build several times. If the tiddlers already have the _canonical_uri ... So take care. Since the (small) images are still tagged with external-image, they are touched everytime you run the build. So you need to check if everything is right after a new build run. I just didn't test this case, so I don't know. Also the import step is needed every time you did import images with the TW drag and drop action. The TW binary file handling isn't optimized for this workflow at the moment. So an alternative version would be to let nginx handle the file upload. I did a fast search and found this: https://coderwall.com/p/swgfvw I did a search: https://www.google.at/search?q=file+upload+ui+for+nginxtbs=qdr:y and found: http://www.howtoforge.com/displaying-upload-progress-with-nginx-on-debian-wheezy If nginx does the upload handling, there is the problem, that we need to tell TW about the new file. ... As I wrote. The workflow needs to be improved for binary file handling. It also brings up another question about the future plans for the system. Do you indeed plan to have a more robust server implementation? I need to pass this on on to @Jeremy. After all, the value is in the sharing of information among many. I have no use for a personal wiki. One reason, that makes TiddlyWiki special and unique, is its focus on beeing an all-in-one single file Wiki. But for your configuration, this is actually a weak spot. You are right, _one_ value of a wiki is in sharing it :) And there are plenty of requests, in making the sharing easier. But most TW users _don't_ want to touch a server. They request file based sharing options. ... But in the TW hangouts we discussed making server more robust. There is a hangout today: https://plus.google.com/u/0/events/cd8h7qbbtethtk44i98cq2fmmis There is a very robust backend for TiddlyWiki. It's called TiddlyWeb. It's a python app, with a lot of options. In its basic configuration it uses a similar file based store as the TW node server. If you are python guru it should be relatively easy for you to handle it. If not, it can be overwhelming. Can you describe your usecase a bit closer, if that's possible. So I can see, if I should recommend it, or not :) - How many different wiki's do you need? - How many users? - Do you have many different editors? - or just consumers with a view content editors / creators? - There is still the option, to create a completely static version of all tiddlers, no javascript involved. This could be served entirely by the nginx server. So every tiddler is a single page. There would be the need for a different layout, because at the moment the static version uses the same layout as the js version. Thanks again for your help. you are welcome The load time is still pretty slow - 38 seconds or so. It does take pretty long over the network to serve the data. I'm turning my attention now to speeding that up. I have a NAT in the way that I maybe can avoid. ok, let us know if it worked out. Thx for posting your nginx configurations, it will help others, that want to use TW with a similar configuration. have fun! mario -- Jeremy Ruston mailto:jeremy.rus...@gmail.com
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
:-) All together now ... 999 bottles of beer on the wall... / :-) On Tuesday, October 21, 2014 10:11:57 AM UTC-4, PMario wrote: On Tuesday, October 21, 2014 1:49:05 PM UTC+2, Jeremy Ruston wrote: https://github.com/Jermolene/TiddlyWiki5/issues/1000 (yay! 1,000 bugs!) yay ... when you closed it :) -m -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
Hi Mario,
Thanks for the follow up. I took a bit of vacation but I'm back now.
I followed your instructions and saw some slight differences. The index is
still pretty large and doesn't seem to put new image imports in the
./images folder.
1) I modified the tiddlywiki.info as you suggested:
{
build: {
index: [
--savetiddlers,[tag[external-image]],images,
--setfield,[tag[external-image]],_canonical_uri,$:/core/templates/canonical-uri-external-image,text/plain,
--setfield,[tag[external-image]],text,,text/plain,
--rendertiddler,$:/core/save/all,index.html,text/plain]
},
plugins: [
tiddlywiki/tiddlyweb,
tiddlywiki/filesystem,
tiddlywiki/codemirror,
tiddlywiki/highlight
],
themes: [
tiddlywiki/vanilla,
tiddlywiki/snowwhite
]
}
2) I added tags to my existing png and jpg tiddlers (manually).
3) I made the /images directory
4) I ran the /usr/bin/node /usr/bin/tiddlywiki /home/ncemonwiki/emonwiki
--output /home/ncemonwiki/emonwiki --build index
I then started the server and tried it.
My load time is now about 37 seconds. So it's better.
But, my index is still pretty large:
[ncemonwiki@stratus emonwiki]$ ls -alh
total 1.9M
drwxr-xr-x. 4 ncemonwiki ncemonwiki 89 Oct 20 11:45 .
drwx--x--x. 5 ncemonwiki ncemonwiki 4.0K Oct 8 12:57 ..
-rwxr-xr-x. 1 ncemonwiki ncemonwiki 329 Sep 26 14:56 .htpasswd
drwxrwxr-x. 2 ncemonwiki ncemonwiki 4.0K Oct 20 11:45 images
-rw-rw-r--. 1 ncemonwiki ncemonwiki 1.9M Oct 20 11:45 index.html
drwxr-xr-x. 2 ncemonwiki ncemonwiki 4.0K Oct 20 12:43 tiddlers
-rw-r--r--. 1 ncemonwiki ncemonwiki 618 Oct 20 11:45 tiddlywiki.info
[ncemonwiki@stratus emonwiki]$
My tiddlers did seem to get rewritten after the build:
[ncemonwiki@stratus tiddlers]$ ls -al | grep png
-rw-r--r--. 1 ncemonwiki ncemonwiki 181 Oct 20 11:45
BlockBDPU_Disabled_Port.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 175 Oct 20 11:45
BlockBDPUGuard_event.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 185 Oct 20 11:45
BlockBDPUGuard_event_rule.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 191 Oct 20 11:45
BlockBPDUGuard_Ticket_Sample.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 193 Oct 20 11:45
BPDUGuard_event_rule_criteria.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 197 Oct 20 11:45 Cisco IOS Device
Location.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 165 Oct 20 11:45 emonUtilBox.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 171 Oct 20 11:45
MajorMCServers.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 12656 Oct 20 12:46 soi_image001.png.tid
[ncemonwiki@stratus tiddlers]$ cat BlockBDPU_Disabled_Port.png.tid
_canonical_uri: ./images/BlockBDPU_Disabled_Port.png
created: 20141001175956296
modified: 20141020153521294
tags: external-image
title: BlockBDPU_Disabled_Port.png
type: image/png
[ncemonwiki@stratus tiddlers]$
However, notice the file soi_image001.png.tid. This was a new import.
On Monday, October 13, 2014 6:51:57 AM UTC-4, PMario wrote:
@Rick,
Any update about your issue?
-m
--
You received this message because you are subscribed to the Google Groups
TiddlyWiki group.
To unsubscribe from this group and stop receiving emails from it, send an email
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
You need to tell nginx now to serve the images. then [img[http://yourdomain.lan/image.png]] should work. -m -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
On Monday, October 20, 2014 7:05:51 PM UTC+2, Rick Williams wrote:
1) I modified the tiddlywiki.info as you suggested:
{
build: {
index: [
--savetiddlers,[tag[external-image]],images,
--setfield,[tag[external-image]],_canonical_uri,$:/core/templates/canonical-uri-external-image,text/plain,
--setfield,[tag[external-image]],text,,text/plain,
--rendertiddler,$:/core/save/all,index.html,text/plain]
},
plugins: [
tiddlywiki/tiddlyweb,
tiddlywiki/filesystem,
tiddlywiki/codemirror,
tiddlywiki/highlight
],
themes: [
tiddlywiki/vanilla,
tiddlywiki/snowwhite
]
}
looks good to me.
2) I added tags to my existing png and jpg tiddlers (manually).
3) I made the /images directory
4) I ran the /usr/bin/node /usr/bin/tiddlywiki /home/ncemonwiki/emonwiki
--output /home/ncemonwiki/emonwiki --build index
ok
I then started the server and tried it.
Did you do the import step? So import the small index.html into the server.
You could select all. ... You did a backup right?
My load time is now about 37 seconds. So it's better.
But, my index is still pretty large:
[ncemonwiki@stratus emonwiki]$ ls -alh
total 1.9M
drwxr-xr-x. 4 ncemonwiki ncemonwiki 89 Oct 20 11:45 .
drwx--x--x. 5 ncemonwiki ncemonwiki 4.0K Oct 8 12:57 ..
-rwxr-xr-x. 1 ncemonwiki ncemonwiki 329 Sep 26 14:56 .htpasswd
drwxrwxr-x. 2 ncemonwiki ncemonwiki 4.0K Oct 20 11:45 images
-rw-rw-r--. 1 ncemonwiki ncemonwiki 1.9M Oct 20 11:45 index.html
1.9 MByte ... that's ok. It should be loaded in about a second or so.
drwxr-xr-x. 2 ncemonwiki ncemonwiki 4.0K Oct 20 12:43 tiddlers
-rw-r--r--. 1 ncemonwiki ncemonwiki 618 Oct 20 11:45 tiddlywiki.info
[ncemonwiki@stratus emonwiki]$
My tiddlers did seem to get rewritten after the build:
I'm not sure, what happens if you do the build several times.
If the tiddlers already have the _canonical_uri ... So take care.
[ncemonwiki@stratus tiddlers]$ ls -al | grep png
-rw-r--r--. 1 ncemonwiki ncemonwiki 181 Oct 20 11:45
BlockBDPU_Disabled_Port.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 175 Oct 20 11:45
BlockBDPUGuard_event.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 185 Oct 20 11:45
BlockBDPUGuard_event_rule.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 191 Oct 20 11:45
BlockBPDUGuard_Ticket_Sample.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 193 Oct 20 11:45
BPDUGuard_event_rule_criteria.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 197 Oct 20 11:45 Cisco IOS Device
Location.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 165 Oct 20 11:45 emonUtilBox.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 171 Oct 20 11:45
MajorMCServers.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 12656 Oct 20 12:46 soi_image001.png.tid
[ncemonwiki@stratus tiddlers]$ cat BlockBDPU_Disabled_Port.png.tid
_canonical_uri: ./images/BlockBDPU_Disabled_Port.png
created: 20141001175956296
modified: 20141020153521294
tags: external-image
title: BlockBDPU_Disabled_Port.png
type: image/png
[ncemonwiki@stratus tiddlers]$
However, notice the file soi_image001.png.tid. This was a new import.
Tiddlywiki server doesn't use the build info. So it doesn't know about
the images directory to store new images.
The TW server at the moment is a very very basic server to make TW
development easier.
-
I'm still concerned about the 37 seconds. ...
If you open the dev console in the browser F12
Select the network tab and reload the page.
The timeline should show you, which element needs the time.
Sorry but I'm running out of ideas, atm :)
It probably is something really simple, we just can't see it.
@community: Anyone else? any ideas? nginx experts ?:)
-m
--
You received this message because you are subscribed to the Google Groups
TiddlyWiki group.
To unsubscribe from this group and stop receiving emails from it, send an email
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
Did you do the import step? So import the small index.html into the server.
You could select all. ... You did a backup right?
I did do this later. But I still winder if it was really necessary.
The image files were already in the images directory.
Tiddlywiki server doesn't use the build info. So it doesn't know about
the images directory to store new images.
The TW server at the moment is a very very basic server to make TW
development easier.
OK, I see. Does this mean I need to rerun the 'build' step
periodically to get new image files placed in the right directory?
It also brings up another question about the future plans for the
system. Do you indeed plan to have a more robust server implementation?
After all, the value is in the sharing of information among many. I have no
use for a personal wiki.
You need to tell nginx now to serve the images.
then [img[http://yourdomain.lan/image.png]] should work.
Ok. After modifying nginx to add location /images/ the images files are seen
as usual:
[root@stratus conf.d]# cat ncemonwiki.conf
# HTTPS server
proxy_cache_path /data/nginx/cache keys_zone=one:10m;
#
server {
listen 8443 ssl;
server_name stratus.emon.nc.gov;
ssl_certificate /etc/nginx/ssl/server.crt;
ssl_certificate_key /etc/nginx/ssl/server.key;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
proxy_cache one;
location / {
root/home/ncemonwiki/emonwiki;
auth_basic Restricted;
auth_basic_user_file /home/ncemonwiki/emonwiki/.htpasswd;
proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_headerHost $http_host;
proxy_set_headerX-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8080;
}
location /images/ {
root /home/ncemonwiki/emonwiki;
}
}
[root@stratus conf.d]#
Thanks again for your help.
The load time is still pretty slow - 38 seconds or so. It does take pretty
long over the network to serve the data. I'm turning my attention now to
speeding that up. I have a NAT in the way that I maybe can avoid.
--
You received this message because you are subscribed to the Google Groups
TiddlyWiki group.
To unsubscribe from this group and stop receiving emails from it, send an email
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
@Rick, Any update about your issue? -m -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
On Monday, October 6, 2014 6:28:32 PM UTC+2, Rick Williams wrote: We have tw5.1.2 installed for Node.js and with NGINX. We're using a daemon to run the wiki out of it's own directory (/home/ncemonwiki/emonwiki) and using SSL to access the nginx server running one the same server. At initial connect from Chrome browser, it takes about 60 seconds to server the wiki. I don't understand, where those 60 seconds come from. It should be not more than if you go to tiddlywiki.com. ... Assuming you have similar number of tiddlers and your server is not a watch. I'm not sure, if I understand we are using SSL to access the nginx server. you have a https:// domain? This shouldn't add that much overhead. What internet connection do you use? downlaod speed? What is your client hardware? Eventually, we'd also like to be able to navigate directly to individual tiddlers (with permanent url) with less than 3 second response. TiddlyWiki can build a static html version for every tiddler. So it would be easy to serve it with nginx. Depending on the location of your server and the location of the user it shouldn't be more than 400 ms for a static page. If I load tiddlywiki.com it needs about a second and consider this to be slow. I personally won't see a page, that needs 3 seconds. I close the tab after 2 :/ ... joking, but imo 3 seconds is frustrating. -m -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
Hi All,
I really appreciate the help and responses on this.
Let me describe my setup in more detail. Perhaps there is some
configuration changes that someone might suggest to help.
My server is in our local data center. Network and server performance
should not be any issue. Keep in mind, all responses are very fast after
the initial connection. It is just in loading the wiki initially where the
delay is - before the first (default) tiddler is presented.
The server itself is a VM with 4 cores, 8GB ram, running CentOS 7.
The config for NGINX is:
# HTTPS server
#
server {
listen 8443 ssl;
server_name stratus.emon.nc.gov;
ssl_certificate /etc/nginx/ssl/server.crt;
ssl_certificate_key /etc/nginx/ssl/server.key;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
location / {
root/home/ncemonwiki/emonwiki;
auth_basic Restricted;
auth_basic_user_file /home/ncemonwiki/emonwiki/.htpasswd;
proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_headerHost $http_host;
proxy_set_headerX-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8080;
}
}
The wiki starts via systemd:
[Service]
WorkingDirectory=/home/ncemonwiki/emonwiki
ExecStart=/usr/bin/node /usr/bin/tiddlywiki /home/ncemonwiki/emonwiki
--server
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=ncemonwiki
User=ncemonwiki
Group=ncemonwiki
Environment='NODE_ENV=production'
[Install]
WantedBy=multi-user.target
And the wiki is located under userid ncemonwiki in the home directory:
[ncemonwiki@stratus ~]$ ls
emonwiki
[ncemonwiki@stratus ~]$ cd emonwiki
[ncemonwiki@stratus emonwiki]$ ls -al
total 16
drwxr-xr-x. 3 ncemonwiki ncemonwiki 59 Sep 26 12:36 .
drwx--x--x. 5 ncemonwiki ncemonwiki 4096 Oct 7 09:15 ..
-rwxr-xr-x. 1 ncemonwiki ncemonwiki 329 Sep 26 14:56 .htpasswd
drwxr-xr-x. 2 ncemonwiki ncemonwiki 4096 Oct 6 16:12 tiddlers
-rw-r--r--. 1 ncemonwiki ncemonwiki 239 Sep 26 09:27 tiddlywiki.info
[ncemonwiki@stratus emonwiki]$ cat tiddlywiki.info
{
plugins: [
tiddlywiki/tiddlyweb,
tiddlywiki/filesystem,
tiddlywiki/codemirror,
tiddlywiki/highlight
],
themes: [
tiddlywiki/vanilla,
tiddlywiki/snowwhite
]
}[ncemonwiki@stratus emonwiki]$ cd tiddlers
[ncemonwiki@stratus tiddlers]$ ls -al
total 1544
drwxr-xr-x. 2 ncemonwiki ncemonwiki 4096 Oct 6 16:12 .
drwxr-xr-x. 3 ncemonwiki ncemonwiki 59 Sep 26 12:36 ..
-rw-r--r--. 1 ncemonwiki ncemonwiki 8186 Oct 3 10:20 ACS_ 5.5 Policy
Model.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 299311 Oct 1 13:59
BlockBDPU_Disabled_Port.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 213576 Oct 1 13:48
BlockBDPUGuard_event.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 158061 Oct 1 13:49
BlockBDPUGuard_event_rule.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 242672 Oct 1 14:00
BlockBPDUGuard_Ticket_Sample.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 49313 Oct 1 14:36
BPDUGuard_event_rule_criteria.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 100637 Sep 30 12:30 Cisco IOS Device
Location.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki166 Sep 26 13:40
$__config_PageControlButtons_Visibility_$__core_ui_Buttons_control-panel.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki173 Sep 29 08:15
$__config_ViewToolbarButtons_Visibility_$__core_ui_Buttons_more-tiddler-actions.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki126 Oct 2 10:53
$__DefaultTiddlers.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki827 Oct 6 16:11 Documentation on
eMon Systems and Tools.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 3352 Sep 26 09:27 emonParseMotd.py.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 40525 Sep 26 13:56 emonUtilBox.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 1135 Sep 30 13:39 emonwiki
Maintenance.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 2370 Oct 1 11:24 Formatted
sysLocation.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 2596 Sep 26 14:05 HP BSM_ Changing
NCFAST Script Password.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 1319 Sep 26 13:14 htpasswd.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 1289 Sep 26 09:27 Linux_ Find Open
Port Process.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki938 Sep 26 09:27 Linux_ Find When
System Last Started_Shutdown.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki509 Sep 26 09:27 Linux_ Using
Systemd.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 265508 Sep 26 13:56
MajorMCServers.png.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki181 Oct 6 15:47 MARS Archives.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki 4509 Oct 6 15:48 MARS Server Info.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki848 Sep 30 13:42 NCeMon Team
Responsibilities.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki824 Sep 30 13:46 NCeMon.tid
-rw-r--r--. 1 ncemonwiki ncemonwiki455 Oct 2 13:01
NCeMon_TiddlyWiki.tid
-rw-r--r--. 1
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
On Tuesday, October 7, 2014 3:23:48 PM UTC+2, Rick Williams wrote: The way I understand this is that we are simply forwarding the requests from nginx to the http server built in to the tiddlywiki program. jup. From the responses so far, I'm thinking there must be another way to configure this that I'm missing. I'm wondering where tiddlywiki actually stores the rendered html content. If you request a TW.html from the TW server, the TW server builds an html file including the TW core, but without the content tiddlers. The browser loads the core and makes a status request to the server If status is ok, the core requests all the content tiddlers. If the TW is loaded, the core makes the rendering .. so no communication to the server Only if you modify a tiddler, the content goes back and forth again. .. The only idea I have, is, that the Data center sets the vm to standby if there are no request for a longer time. So the initial load could be starting the VM + app ... This could be up to a minute. ... But I think, the server is in use. What about server load? -m -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
Re: [tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
If you request a TW.html from the TW server, the TW server builds an html file including the TW core, but without the content tiddlers. One small correction: the HTML file does include the content tiddlers. (It doesn't include them when used with TiddlyWeb). Best wishes Jeremy The browser loads the core and makes a status request to the server If status is ok, the core requests all the content tiddlers. If the TW is loaded, the core makes the rendering .. so no communication to the server Only if you modify a tiddler, the content goes back and forth again. .. The only idea I have, is, that the Data center sets the vm to standby if there are no request for a longer time. So the initial load could be starting the VM + app ... This could be up to a minute. ... But I think, the server is in use. What about server load? -m -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout. -- Jeremy Ruston mailto:jeremy.rus...@gmail.com -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
Hi PMario, That's definitely not the case. I've defined this server myself specifically for this. There is no competing load of any significance. On Tuesday, October 7, 2014 3:46:53 PM UTC-4, PMario wrote: On Tuesday, October 7, 2014 3:23:48 PM UTC+2, Rick Williams wrote: The way I understand this is that we are simply forwarding the requests from nginx to the http server built in to the tiddlywiki program. jup. From the responses so far, I'm thinking there must be another way to configure this that I'm missing. I'm wondering where tiddlywiki actually stores the rendered html content. If you request a TW.html from the TW server, the TW server builds an html file including the TW core, but without the content tiddlers. The browser loads the core and makes a status request to the server If status is ok, the core requests all the content tiddlers. If the TW is loaded, the core makes the rendering .. so no communication to the server Only if you modify a tiddler, the content goes back and forth again. .. The only idea I have, is, that the Data center sets the vm to standby if there are no request for a longer time. So the initial load could be starting the VM + app ... This could be up to a minute. ... But I think, the server is in use. What about server load? -m -- You received this message because you are subscribed to the Google Groups TiddlyWiki group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
[tw] Re: [TW5] Advice on Node.js Deployment and Wiki Performance
On Tuesday, October 7, 2014 10:43:27 PM UTC+2, Rick Williams wrote:
Hi PMario,
That's definitely not the case. I've defined this server myself
specifically for this. There is no competing load of any significance.
I think I know it.
WARNING: Be aware, that I didn't test the following steps. So ... *backup
;)*
your ls -al says: total 1544 !!
And there seem to be many .png.tid and .jpeg's with several 100kByte ... So
your TiddlyWikki will be huge.
IMO you should store your images in an image subfolder and use the
_cannonical_uri field for the image tiddlers.
So your TW stays small and nginx can load the static images on demand. ...
IMO Your startup time should be down quite a bit.
The tiddlywiki.info file has a configuration, that lets you create external
images with the TW build process.
see:
https://github.com/Jermolene/TiddlyWiki5/blob/master/editions/tw5.com/tiddlywiki.info#L32
You should add this to your tiddlywiki.info
build: {
index: [
--savetiddlers,[tag[external-image]],images,
--setfield,[tag[external-image]],_canonical_uri,$:/core/templates/canonical-uri-external-image,text/plain,
--setfield,[tag[external-image]],text,,text/plain,
--rendertiddler,$:/core/save/all,index.html,text/plain]
},
Then you should add a tag eg: external-image to your .png.tid, png.meta
files.
Same for the .jpg.meta and jpg.tid. ..
Manually or may be a sed script. ..
The tag needs to fit the tag in the tiddlywiki.info build section
Your build commands should look like this:
# I don't know if TW build creates the image dir so we do :)
# Just to be sure you have a backup?
mkdir /home/ncemonwiki/emonwiki/images
/usr/bin/node /usr/bin/tiddlywiki /home/ncemonwiki/emonwiki --output
/home/ncemonwiki/emonwiki --build index
Your /images directory should now contain all the images. ... if not
something went wrong.
The build command should create a index.html file, that now should be
relatively small.
(Just a test. Build it again without the --setfield parameters. That's the
file that was served in 60 seconds. .. I'm interested in the size of this
file!)
Now comes the tricky part. ... The build did create images, but didn't
rewrite the .png.tid files in the tiddlers folder.
The (small) index.html should be accessible from a browser. we need to
import it into the server version. ...
If you open the index.html file with a browser you should check some
images.
They should have no text content anymore but a field named _canonical_uri
that points to the image dir ...
- Start the server. Will still be slow.
- Drag and drop the small index.html file to the open TW in the browser
- It should display an import dialog.
- Select all the images, import and pray .
- The import should rewrite all the image tiddler in the tiddlers folder.
- so this may take some time.
If everything goes well ... your TW should be faster now.
If something doesn't make sense. Just ask.
have fun!
mario
--
You received this message because you are subscribed to the Google Groups
TiddlyWiki group.
To unsubscribe from this group and stop receiving emails from it, send an email
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
