[Mahara-contributors] [Bug 1514273] Re: Increase to "longtext" all text columns that hold serialized user data
** Changed in: mahara Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1514273 Title: Increase to "longtext" all text columns that hold serialized user data Status in Mahara: Fix Released Bug description: We've now had two bugs caused by serialized user data going into a text database column, and being truncated to 65KB by MySQL: Bug 1474659: import_entry_requests.entrycontent Bug 1513851: activity_queue.data We should try to get ahead of this issue by similarly increasing the size of any other columns in the database that typically store serialized user data. For instance, block_instance.configdata. If you created a text block that contains exactly 65KB of text, it would probably cause this same error, because, when serialized, it would tip to just over 65KB. I can't recall other serialized data columns off the top of my head, but I imagine there's a few. Additionally, if we have any other columns that store structured data with user content inside of it, we should expand those as well. (Are we storing any JSON-encoded values into the DB yet?). HTML is kind of an exception, because, while it will break if it's truncated, it won't generally cause a fatal error if we have broken HTML in the DB. Instead, htmlpurifier will filter it out when it gets displayed. To manage notifications about this bug go to: https://bugs.launchpad.net/mahara/+bug/1514273/+subscriptions ___ Mailing list: https://launchpad.net/~mahara-contributors Post to : mahara-contributors@lists.launchpad.net Unsubscribe : https://launchpad.net/~mahara-contributors More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1514273] Re: Increase to "longtext" all text columns that hold serialized user data
** Changed in: mahara Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1514273 Title: Increase to "longtext" all text columns that hold serialized user data Status in Mahara: Fix Committed Bug description: We've now had two bugs caused by serialized user data going into a text database column, and being truncated to 65KB by MySQL: Bug 1474659: import_entry_requests.entrycontent Bug 1513851: activity_queue.data We should try to get ahead of this issue by similarly increasing the size of any other columns in the database that typically store serialized user data. For instance, block_instance.configdata. If you created a text block that contains exactly 65KB of text, it would probably cause this same error, because, when serialized, it would tip to just over 65KB. I can't recall other serialized data columns off the top of my head, but I imagine there's a few. Additionally, if we have any other columns that store structured data with user content inside of it, we should expand those as well. (Are we storing any JSON-encoded values into the DB yet?). HTML is kind of an exception, because, while it will break if it's truncated, it won't generally cause a fatal error if we have broken HTML in the DB. Instead, htmlpurifier will filter it out when it gets displayed. To manage notifications about this bug go to: https://bugs.launchpad.net/mahara/+bug/1514273/+subscriptions ___ Mailing list: https://launchpad.net/~mahara-contributors Post to : mahara-contributors@lists.launchpad.net Unsubscribe : https://launchpad.net/~mahara-contributors More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1514273] Re: Increase to "longtext" all text columns that hold serialized user data
For the info, We had the following unserialize error multiple times when
upgrading from 1.10.2 to 15.04.5:
--
...
[DBG] 4f (lib/db/upgrade.php:4005) Updating TinyMCE emoticon locations in
mahara database
[WAR] 4f (blocktype/lib.php:601) unserialize(): Error at offset 1995 of 2050
bytes
Call stack (most recent first):
* log_message("unserialize(): Error at offset 1995 of 2050 bytes", 8, true,
true, "/app/apache2/htdocs/mahara/blocktype/lib.php", 601) at
/app/apache2/htdocs/mahara/lib/errors.php:441
* error(8, "unserialize(): Error at offset 1995 of 2050 bytes",
"/app/apache2/htdocs/mahara/blocktype/lib.php", 601, array(size 2)) at Unknown:0
* unserialize("a:6:{s:11:"artefactids";a:1:{i:0;i:3717;}s:11:"pro...") at
/app/apache2/htdocs/mahara/blocktype/lib.php:601
* BlockInstance->get("configdata") at
/app/apache2/htdocs/mahara/lib/db/upgrade.php:4036
* xmldb_core_upgrade("2014092313") at
/app/apache2/htdocs/mahara/lib/upgrade.php:359
* upgrade_core(object(stdClass)) at
/app/apache2/htdocs/mahara/lib/mahara.php:253
* upgrade_mahara(array(size 40)) at
/app/apache2/htdocs/mahara/admin/cli/upgrade.php:50
[WAR] 4f (lib/db/upgrade.php:4037) Invalid argument supplied for foreach()
Call stack (most recent first):
* log_message("Invalid argument supplied for foreach()", 8, true, true,
"/app/apache2/htdocs/mahara/lib/db/upgrade.php", 4037) at
/app/apache2/htdocs/mahara/lib/errors.php:441
* error(2, "Invalid argument supplied for foreach()",
"/app/apache2/htdocs/mahara/lib/db/upgrade.php", 4037, array(size 26)) at
/app/apache2/htdocs/mahara/lib/db/upgrade.php:4037
* xmldb_core_upgrade("2014092313") at
/app/apache2/htdocs/mahara/lib/upgrade.php:359
* upgrade_core(object(stdClass)) at
/app/apache2/htdocs/mahara/lib/mahara.php:253
* upgrade_mahara(array(size 40)) at
/app/apache2/htdocs/mahara/admin/cli/upgrade.php:50
...
--
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
Matching subscriptions: Subscription for all Mahara Contributors -- please ask
on #mahara-dev or mahara.org forum before editing or unsubscribing it!
https://bugs.launchpad.net/bugs/1514273
Title:
Increase to "longtext" all text columns that hold serialized user data
Status in Mahara:
Confirmed
Bug description:
We've now had two bugs caused by serialized user data going into a
text database column, and being truncated to 65KB by MySQL:
Bug 1474659: import_entry_requests.entrycontent
Bug 1513851: activity_queue.data
We should try to get ahead of this issue by similarly increasing the
size of any other columns in the database that typically store
serialized user data. For instance, block_instance.configdata. If you
created a text block that contains exactly 65KB of text, it would
probably cause this same error, because, when serialized, it would tip
to just over 65KB.
I can't recall other serialized data columns off the top of my head,
but I imagine there's a few.
Additionally, if we have any other columns that store structured data
with user content inside of it, we should expand those as well. (Are
we storing any JSON-encoded values into the DB yet?). HTML is kind of
an exception, because, while it will break if it's truncated, it won't
generally cause a fatal error if we have broken HTML in the DB.
Instead, htmlpurifier will filter it out when it gets displayed.
To manage notifications about this bug go to:
https://bugs.launchpad.net/mahara/+bug/1514273/+subscriptions
___
Mailing list: https://launchpad.net/~mahara-contributors
Post to : mahara-contributors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mahara-contributors
More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1514273] Re: Increase to "longtext" all text columns that hold serialized user data
I researched all the serialized/unserialized values from Mahara code (16.04dev
- 2016021000) and found out the columns/rows that store serialized data and
could potentially errors in MySQL:
* Column: block_instance.configdata
* Column: config.value - Row: config.field IN ("cookieconsent_settings",
"filters", "footerlinks", "footercustomlinks", "openssl_history")
* Column: skin_fonts.variants
* Column: skin.viewskin
* Column: skin_favorites.favorites
* Column: usr_registration.extra
* Column: import_entry_requests.duplicateditemids
* Column: import_entry_requests.existingitemids
* Column: import_entry_requests.artefactmapping
* Column: blocktype_externalfeed_data.image
* Column: import_queue.data
As a safety, I think maybe we can change all those columns to the
longtext type even if some of them may never reach over 65kb
--
You received this bug notification because you are a member of Mahara
Contributors, which is subscribed to Mahara.
Matching subscriptions: Subscription for all Mahara Contributors -- please ask
on #mahara-dev or mahara.org forum before editing or unsubscribing it!
https://bugs.launchpad.net/bugs/1514273
Title:
Increase to "longtext" all text columns that hold serialized user data
Status in Mahara:
Confirmed
Bug description:
We've now had two bugs caused by serialized user data going into a
text database column, and being truncated to 65KB by MySQL:
Bug 1474659: import_entry_requests.entrycontent
Bug 1513851: activity_queue.data
We should try to get ahead of this issue by similarly increasing the
size of any other columns in the database that typically store
serialized user data. For instance, block_instance.configdata. If you
created a text block that contains exactly 65KB of text, it would
probably cause this same error, because, when serialized, it would tip
to just over 65KB.
I can't recall other serialized data columns off the top of my head,
but I imagine there's a few.
Additionally, if we have any other columns that store structured data
with user content inside of it, we should expand those as well. (Are
we storing any JSON-encoded values into the DB yet?). HTML is kind of
an exception, because, while it will break if it's truncated, it won't
generally cause a fatal error if we have broken HTML in the DB.
Instead, htmlpurifier will filter it out when it gets displayed.
To manage notifications about this bug go to:
https://bugs.launchpad.net/mahara/+bug/1514273/+subscriptions
___
Mailing list: https://launchpad.net/~mahara-contributors
Post to : mahara-contributors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mahara-contributors
More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1514273] Re: Increase to "longtext" all text columns that hold serialized user data
** Changed in: mahara Assignee: (unassigned) => Jean-Philippe Gaudreau (jp-gaudreau) ** Changed in: mahara Status: Confirmed => In Progress -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1514273 Title: Increase to "longtext" all text columns that hold serialized user data Status in Mahara: In Progress Bug description: We've now had two bugs caused by serialized user data going into a text database column, and being truncated to 65KB by MySQL: Bug 1474659: import_entry_requests.entrycontent Bug 1513851: activity_queue.data We should try to get ahead of this issue by similarly increasing the size of any other columns in the database that typically store serialized user data. For instance, block_instance.configdata. If you created a text block that contains exactly 65KB of text, it would probably cause this same error, because, when serialized, it would tip to just over 65KB. I can't recall other serialized data columns off the top of my head, but I imagine there's a few. Additionally, if we have any other columns that store structured data with user content inside of it, we should expand those as well. (Are we storing any JSON-encoded values into the DB yet?). HTML is kind of an exception, because, while it will break if it's truncated, it won't generally cause a fatal error if we have broken HTML in the DB. Instead, htmlpurifier will filter it out when it gets displayed. To manage notifications about this bug go to: https://bugs.launchpad.net/mahara/+bug/1514273/+subscriptions ___ Mailing list: https://launchpad.net/~mahara-contributors Post to : mahara-contributors@lists.launchpad.net Unsubscribe : https://launchpad.net/~mahara-contributors More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1514273] Re: Increase to "longtext" all text columns that hold serialized user data
Here's a patch for master branch: https://reviews.mahara.org/6058 -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1514273 Title: Increase to "longtext" all text columns that hold serialized user data Status in Mahara: In Progress Bug description: We've now had two bugs caused by serialized user data going into a text database column, and being truncated to 65KB by MySQL: Bug 1474659: import_entry_requests.entrycontent Bug 1513851: activity_queue.data We should try to get ahead of this issue by similarly increasing the size of any other columns in the database that typically store serialized user data. For instance, block_instance.configdata. If you created a text block that contains exactly 65KB of text, it would probably cause this same error, because, when serialized, it would tip to just over 65KB. I can't recall other serialized data columns off the top of my head, but I imagine there's a few. Additionally, if we have any other columns that store structured data with user content inside of it, we should expand those as well. (Are we storing any JSON-encoded values into the DB yet?). HTML is kind of an exception, because, while it will break if it's truncated, it won't generally cause a fatal error if we have broken HTML in the DB. Instead, htmlpurifier will filter it out when it gets displayed. To manage notifications about this bug go to: https://bugs.launchpad.net/mahara/+bug/1514273/+subscriptions ___ Mailing list: https://launchpad.net/~mahara-contributors Post to : mahara-contributors@lists.launchpad.net Unsubscribe : https://launchpad.net/~mahara-contributors More help : https://help.launchpad.net/ListHelp
[Mahara-contributors] [Bug 1514273] Re: Increase to "longtext" all text columns that hold serialized user data
On a related note, we should probably also add something to Pieforms to limit user input to 65KB unless it's flagged otherwise. Many of our plugins do this already on an ad-hoc basis with a Pieforms "maxlength" rule, but it would be good to have it turned on by default for all the pieform elements that take user text as input. -- You received this bug notification because you are a member of Mahara Contributors, which is subscribed to Mahara. Matching subscriptions: Subscription for all Mahara Contributors -- please ask on #mahara-dev or mahara.org forum before editing or unsubscribing it! https://bugs.launchpad.net/bugs/1514273 Title: Increase to "longtext" all text columns that hold serialized user data Status in Mahara: Confirmed Bug description: We've now had two bugs caused by serialized user data going into a text database column, and being truncated to 65KB by MySQL: Bug 1474659: import_entry_requests.entrycontent Bug 1513851: activity_queue.data We should try to get ahead of this issue by similarly increasing the size of any other columns in the database that typically store serialized user data. For instance, block_instance.configdata. If you created a text block that contains exactly 65KB of text, it would probably cause this same error, because, when serialized, it would tip to just over 65KB. I can't recall other serialized data columns off the top of my head, but I imagine there's a few. Additionally, if we have any other columns that store structured data with user content inside of it, we should expand those as well. (Are we storing any JSON-encoded values into the DB yet?). HTML is kind of an exception, because, while it will break if it's truncated, it won't generally cause a fatal error if we have broken HTML in the DB. Instead, htmlpurifier will filter it out when it gets displayed. To manage notifications about this bug go to: https://bugs.launchpad.net/mahara/+bug/1514273/+subscriptions ___ Mailing list: https://launchpad.net/~mahara-contributors Post to : mahara-contributors@lists.launchpad.net Unsubscribe : https://launchpad.net/~mahara-contributors More help : https://help.launchpad.net/ListHelp
