I notice that on my local machine Mahara 17.04+ saves
João Jiménez Māori
in the title fields as 'João Jiménez Māori'
and in the description fields as 'João Jiménez Māori'
But on cluster machines in 16.10 it saves
in the title fields as 'Jo<C3><A3>o Jim<C3><A9>nez M<C4><81>ori'
in the description fields as 'João Jiménez M<C4><81>ori'
If I do a
SELECT 'João Jiménez Māori' AS test;
The both show the special chars correctly
But if I do
UPDATE view SET description = '<p>João Jiménez Māori</p>' where id =
10;
My local shows it like local above but cluster shows it like cluster
above.
So the cluster setup for postgres must be different in the way it
handles special utf8 characters
** Changed in: mahara
Milestone: 17.10.0 => 18.04.0
--
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
Matching subscriptions: Subscription for all Mahara Contributors -- please ask
on #mahara-dev or mahara.org forum before editing or unsubscribing it!
https://bugs.launchpad.net/bugs/1487274
Title:
Elasticsearch choking on non-ASCII characters
Status in Mahara:
Confirmed
Bug description:
In 15.10 I've added code to "quarantine" records that Elasticsearch
won't index. That is, if Elasticsearch errors out while processing a
batch of records, then I re-try each record individually. And if it
errors out while processing one of those individual records, I mark
the record as quarantined, and keep it in the
search_elasticsearch_queue table.
I've backported that to one of our large 15.04 sites, and since then
I've taken a look at the data in the records that have caused
Elasticsearch to choke. They all contain non-ASCII characters, i.e.
Unicode characters. These can be as simple as "e with an accent over
it", all the way up to exotic ones like emoji and the Unicode snowman.
I was not able to replicate this when testing on my local machine, but
it is certainly in place on our production servers, and bugs such as
Bug 1408577 make me think it's probably also present on some other
servers as well.
To manage notifications about this bug go to:
https://bugs.launchpad.net/mahara/+bug/1487274/+subscriptions
_______________________________________________
Mailing list: https://launchpad.net/~mahara-contributors
Post to : mahara-contributors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mahara-contributors
More help : https://help.launchpad.net/ListHelp
