By the way,
I realize today that since you removed the legends from config, I is
becoming very useful to write on the info page a line like legend: part1 |
part2 | part3
... but won't it get erased on pages such as index.game everytime we
rebuild an index from scratch? Maybe for Boltwire 6 (whoa already planning
Boltwire 6 when Boltwire 5 was released not so long ag ! It's amazing to
see so many things moving forward and going well for Boltwire!), but I
could be neat to creation of indexing a way to specify a legend to write in
the index file. If data vars are indexed, I would propose to directly reuse
the "legends" parameter of the search function.
What do you think?
Tiffany
Le lundi 11 janvier 2016 18:48:35 UTC+1, Tiffany Grenier a écrit :
>
> Thanks for your quick answer, and the extensive explanation.
>
> It appears I did not understand at all how the cron indexing was working.
> I like your suggestion of reindexing only changes pages with [(index
> list=game)] and automatically indexing from the last indexing time only.
> Then it would probably be enough to delete site.index.game to force a full
> indexing (not only from last indexing time).
>
> I would enjoy such features very much in the next release, or even one
> later. I have something working well for now, so this is not urgent.
>
> Cheers,
> Tiffany
>
> Le lundi 11 janvier 2016 16:03:42 UTC+1, Dan a écrit :
>>
>> Ok, I understand what you are saying now. And I note I didn't answer your
>> question, either...
>>
>> So here's the general way indexing is supposed to work as I remember it:
>>
>> 1) If you set indexing to cron in site.config, changed pages get logged
>> on page site.index.cron--and the pages are NOT indexed. You then HAVE TO
>> use a cron job to call some public page you create with the index function
>> on it. Otherwise nothing will happen. The idea of this is to completely
>> avoid page lags when you link indexing to page views.
>>
>> 2) Depending on the parameters you put on the cron index function several
>> things could happen:
>>
>> [(index)] -- nothing happens. no pages specified
>> [(index cron=10)] -- index 10 pages on site.index.cron. They will be
>> checked for all matching rules
>> [(index page.1,page.2,page.3)] -- index the 3 specified pages. They will
>> be checked for all matching rules
>> [(index rule=game)] -- indexes 25 (indexBatch) pages on the
>> site.index.game list using the game rule defined on site.index
>>
>> Now the auto indexing feature, if set to cron will never populate pages
>> like site.index.game. It only populates site.index.cron. Nor will it catch
>> pages changes by ftp. So yes, you do need to use the action.index script to
>> populate them. Using BoltWire's script capabilities to do that via a cron
>> job was brilliant. Never would have thought of doing that.
>>
>> Having said that, I see the merit in allowing a batch parameter in the
>> index function to optionally override the default setting. So [(index
>> rule=game batch=10)] should be able to overrule the default site.config
>> setting of 25. This is easily done by making these changes:
>>
>> Change the first line of BOLTFindex to
>> if ($args['rule'] != '') $out = BOLTindex($args['rule'], $args['batch']);
>>
>> Change the first two lines of BOLTindex to
>> function BOLTindex($rule='', $batch='') {
>> $batch = BOLTinit(BOLTconfig('indexBatch', 25), $batch);
>>
>> Another problem that occurs to me reflecting on all this is [(index)]
>> should default to the equivalent of cron=indexBatch. This requires a slight
>> rewrite of the index function. It also has the added advantage that you can
>> do [(index 10)] and it is the equivalent of [(index cron=10)]. I plan to
>> include the changes above and below to the next release, unless you see
>> some reason not to.
>>
>> function BOLTFindex($args) {
>> if ($args['rule'] != '') $out = BOLTindex($args['rule'], $args['batch']);
>> // works through a rule if no list
>> else {
>> $count = BOLTinit(BOLTconfig('indexBatch', 25), $args[1], $args['cron']);
>> if (is_number($count)) {
>> $cron = BOLTinit('site.index.cron', BOLTconfig('cronIndex'));
>> $content = trim(BOLTloadpage($cron));
>> if ($content == '') return BOLTmessage('index_current', $args);
>> $pages = explode("\n", $content);
>> $pages = array_unique($pages);
>> if (count($pages) > $count) {
>> $list = array_slice($pages, 0, $count);
>> $content = implode("\n", array_slice($pages, $count));
>> }
>> else {
>> $list = $pages;
>> $content = ' ';
>> }
>> BOLTsavepage($cron, $content);
>> }
>> else {
>> if ($count == '') return BOLTmessage('index_empty', $args);
>> $list = explode(',', $count);
>> }
>> $out = BOLTindexAuto($list); // indexes pages
>> }
>> BOLTmessage('index_complete', $args);
>> return $out;
>> }
>>
>> Ok, now for the big question--the problem you actually raised. How do I
>> automatically index changed pages which I have uploaded by ftp or via some
>> custom rule? Your answer is the right one of course. But it might be
>> possible to simplify it further by creating a plugin, or extending the
>> index function. Here's what I envision:
>>
>> 1) Either call it [(indexer rule=game)] or perhaps [(index list=game)].
>> The first is a plugin, the latter an extension of the core. Either way,
>> this tells BoltWire to create an updated list for the specified rule.
>> 2) The script simply looks up the rule, generates a pagelist based on the
>> rule parameters, and appends it to page site.index.game.
>> 3) To be really smart, we would somehow record the last time that rule
>> was "listed" and only include pages changes since then.
>>
>> If we could get #3 to work well, we could make this happen automatically
>> anytime you call [(index rule=game)] before starting the indexing process.
>> That way it just works. As for accomplishing #3, I propose saving the
>> last listed date as an info var hidden in a data var on site.index. It
>> might be a bit tricky as I suspect there could be situations where you miss
>> pages--but can't think of any. We could also always use the action index
>> anytime we wanted to reindex an entire group of pages.
>>
>> I'm sure I could pull this together in 10 or 15 minutes. Will wait for
>> your response to see if you have any thoughts or feedback?
>>
>> Cheers,
>> Dan
>>
>> On Sun, Jan 10, 2016 at 2:34 AM Tiffany Grenier <[email protected]>
>> wrote:
>>
>>> Hi,
>>>
>>> About the cron, I checked the BOLTFindex function, and it goes as
>>> follows:
>>> if ($args['rule'] != '') $out = BOLTindex($args['rule']); // works
>>> through a rule if no list
>>> else {
>>> if (is_number($args['cron'])) {
>>> So I don't see how rule and cron could be used at the same time...
>>>
>>> In the end, I created several cron commands to page
>>> action.index&rule=rule1, action.index&rule=rule2 etc. The current rate is
>>> one every two minutes, each every twenty minutes (yes, I have quite a
>>> number of indexes). It works fine, using the default indexBatch value and
>>> updating the list of pages to be indexed everytime a change is made on the
>>> website by members.
>>> I also added a "buildIndexes" script that I included with all the rules
>>> on a page I visit whenever I do massive changes from the FTP interface.
>>> Then I just have to way that the whole indexing has been processed by my
>>> index cron rules. If I know I will change main pages on a regular basis
>>> (use page not relevant at the moment, but you never know), I can also
>>> create a cron rule for this page. It spares me the necessity of login in
>>> and clicking my way through all indexes in action.index without starting
>>> the indexing process.
>>> It is a simple copy from tha action.index indexing building form, and
>>> goes as follows (although I don't think it could be of use for anyone else):
>>> code.script.inde
>>> authkey: indexing
>>> source: 'site.index::{?build}'
>>> reset: search='{=source} fmt=list count=false dir=pages'
>>> search:
>>> [if set {?build}]edit: content='{=search}' page='site.index.{?build}'[if
>>> ]
>>> That's it,
>>> Cheers,
>>> Tiffany/Maelite
>>>
>>>
>>>
>>>
>>>
>>> Le jeudi 7 janvier 2016 19:42:13 UTC+1, Dan a écrit :
>>>
>>>> Thank you!. I fixed the typo, and moved the messages you indicated to
>>>> site.messages.
>>>>
>>>> As for #3, I believe you can use the cron parameter to do the
>>>> equivalent. Batch is probably a better word, but I'm not sure the change
>>>> is
>>>> worth the disruption...
>>>>
>>>> [(index rule=site cron={?quantity})]
>>>>
>>>> Cheers,
>>>> Dan
>>>>
>>>>
>>>>
>>>> On Tue, Jan 5, 2016 at 2:57 AM Tiffany Grenier <[email protected]>
>>>> wrote:
>>>>
>>> Hi,
>>>>>
>>>>> 3 quick notes regarding indexing:
>>>>>
>>>>> - I just noticed a typo in BOLTindex:
>>>>> if ($myrule == '') return "Index rule //$myrule// not found";
>>>>> should be
>>>>> if ($myrule == '') return "Index rule //$rule// not found";
>>>>> Otherwise this warning gives no info at all...
>>>>> - Also, I think the "Indexing is current" and "No pages to index"
>>>>> warnings should be localized and moved to site.messages.
>>>>> - And lastly, a small question: if I define 3 indexing rules and
>>>>> set indexing to cron, nothing will happen, right? If so, could you
>>>>> please
>>>>> add a "batch" parameter to the indexing function such that I can put
>>>>> in
>>>>> cron.index (which is visited through a cron job):
>>>>> [(index rule=site batch={?quantity})]
>>>>> [(index rule=game batch={?quantity})]
>>>>> [(index rule=memberstats batch={?quantity})]
>>>>> I know it will kind of do that automaticaly everytime a page is
>>>>> modified in the website, but i just modified a whole bunch of pages
>>>>> through
>>>>> FTP and it's a real pain to re-index it all manually... I am heavily
>>>>> using
>>>>> content of index.game and index.memberstats and need them up-to-date
>>>>> the
>>>>> fastest way possible.
>>>>>
>>>>> Cheers,
>>>>> Tiffany
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "BoltWire" group.
>>>>>
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To post to this group, send email to [email protected].
>>>>
>>>>
>>>>> Visit this group at https://groups.google.com/group/boltwire.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "BoltWire" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> Visit this group at https://groups.google.com/group/boltwire.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"BoltWire" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/boltwire.
For more options, visit https://groups.google.com/d/optout.
