Urban,
It is done this way, because simply writing UTF-8 to the database tables
results in very random sorting (for diacritics). And by random, I mean
things like Z with a particular diacritic appearing in the middle of the
'A' entries - that's just broken in any language / locale.
By decomposing to the base and combining character, we guarantee that
the ordering is at least predictable, and sensible for those of us of an
English persuasion.
The normalisation that is applied to the strings that will be used for
sorting is configurable though - you simply need to specify the plugins
used for a particular order format, by defining the OrderFormatDelegate
in dspace.cfg - ie:
plugin.named.org.dspace.sort.OrderFormatDelegate= \
org.dspace.sort.OrderFormatTitleMarc21=title
In this case, wherever I specify that my sort column is of type 'title',
then it will use the OrderFormatTitleMarc21 class (an implementation of
OrderFormatDelegate) to normalise the string before writing it to the
database.
You can name a type anything you like - ie:
plugin.named.org.dspace.sort.OrderFormatDelegate= \
org.dspace.sort.OrderFormatTitleMarc21=marc21title
webui.itemlist.sort-option.1 = title:dc.title:marc21title
would work correctly - you just need to tie the 'name' of the plugin
(after the equals), with the last component of the sort-option
configuration.
In your case, you should look at the LocaleOrderingFilter (in
org.dspace.browse), that will normalise a String so that the resulting
value can be sorted in a way that is correct for a specified Locale.
You will need to create an OrderFormatDelegate implementation, but that
is as simple as:
public class OrderFormatLocale extends AbstractTextFilterOFD
{
{
filters = new TextFilter[] {
new LowerCaseAndTrim(),
new LocaleOrderingFilter() };
}
}
Note that the LocaleOrderingFilter produces a String that is NOT
readable, and is inappropriate to be passed to other filters like
LowerCaseAndTrim, hence the particular ordering presented above.
Also note, that this will always produce sort strings based on the
Locale that you define in 'webui.browse.sort.locale' - if your interface
supports being displayed in multiple locales, the ordering can't be
changed based on the locale being displayed. This is a limitation of
making the system performant - if we used a Java collator to do all the
sorting during display, the browse / search function would not scale to
large numbers of items.
G
Urban Andersson wrote:
> Hello all,
>
> One obstacle that I see in 1.5 is that terms containing diacritics are
> sorted by base character in the browse indexes - i.e. the letter "Ö"
> (oumlaut) is treated like "O" etc., which result in records not being
> sorted according to (national) standards (or the postgres locale).
>
> I notice that these characters are decomposed (diacritic stored after
> the base character) before writing the sorting terms to the database,
> and this result in the above.
> I know that this has been discussed in the past, at the DSUG meeting in
> Rome for instance, but I cannot remember the exact reason why it was
> done this way (but I remember that there was an explanation and I would
> be happy if someone could please repeat it).
>
> Modifying the utf-8 characters manually in bi_2_dis.sort_value et al
> will result in records being sorted correctly, but then the links from
> the indexes don't seem to work.
>
> Has anyone else looked into this and is there perhaps a known, quick
> (and possibly dirty) solution to get the most rudimentary sorting to
> work properly?
> Our 1.5 runs on Debian linux, Postgres and Tomcat 6.
>
>
> / Urban Andersson
>
This email has been scanned by Postini.
For more information please visit http://www.postini.com
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
DSpace-tech mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspace-tech
