subject:"\[Django\] #18392\: Use utf8mb4 encoding with MySQL 5.5"

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2017-08-10 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Claude Paroz):

 * needs_better_patch:  1 => 0
 * needs_tests:  1 => 0


Comment:

 I provided a [https://github.com/django/django/pull/8886 more
 comprehensive patch].

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.684d7571368d9bbf7530cb3931529069%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2017-08-05 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Claude Paroz):

 Oh, now I realize that `utf8mb4` can also be set in DATABASES OPTIONS.
 Still, using it in Django by default is a strong signal.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.cf7b98305c191af520aaacf5091371a7%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2017-08-05 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Claude Paroz):

 I suggest to begin with a very minimal patch like
 [https://github.com/django/django/pull/8853 this PR], which will at least
 allow users to begin converting some database columns to `utf8mb4` through
 custom migrations, and use these columns in their code (where indexing
 doesn't come in their way).

 Working on index issues can come later, and will be needed to run the
 Django test suite with `utf8mb4`.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.676ad6b7218a2ac48435d877082515ce%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2016-09-19 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by timgraham):

 The skips for the tests added in 1a9f6db5ffd2d5e71d73340ab59476572e05a728
 should be removed or modified when completing this ticket. Should we skip
 them conditionally based on the test charset or should we require running
 the MySQL tests with utf8mb4?

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.0bcb2697c1810ae38a96ccf65e32125f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2016-06-15 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by clokep):

 * cc: clokep@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.a4740aba92ced8a25d91478f994e330b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2016-05-18 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by edmorley):

 * cc: emorley@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.b7047872f9f2794a9c52a8bd5c6486fc%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2016-03-08 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by collinanderson):

 Yes, I based my proposal off of what WordPress did. WordPress limited the
 length of the index without limiting the length of the field itself.
 Django currently doesn't have that option.

 https://code.djangoproject.com/ticket/18392#comment:26

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.09985b606b2921d1010b1e424dae0554%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2016-01-15 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by EmilStenstrom):

 Slightly off-topic but maybe interesting: The Wordpress team did convert
 all their users from utf8 to utf8mb4 according to [this blog
 post](https://make.wordpress.org/core/2015/04/02/the-utf8mb4-upgrade/).

 Looking at the code it seems they just dropped and recreated all the
 indexes: https://github.com/WordPress/WordPress/blob/master/wp-
 admin/includes/upgrade.php#L2687

 And then converted all the tables one by one:
 https://github.com/WordPress/WordPress/blob/master/wp-
 admin/includes/upgrade.php#L1951

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.81dcbc65db0419878a44db92ee696e19%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2016-01-12 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by kezabelle):

 * cc: django@… (added)


Comment:

 By way of consensus: I agree that if possible, going forward, Django ought
 to seek to use utf8mb4, because for better or worse, the world is becoming
 more mobile oriented, and with it more emoji-laden; the non-mb4
 charsets/collations choke on such things, which is just plain unfortunate.

 If a suitable backwards-compatible, migration-friendly patch can't be
 achieved, it should at least be mentioned in the docs somewhere, IMHO
 (searching for `utf8mb4` or `emoji` currently yields no results, so I
 assume there are none squirreled away). Whether that mention is a
 recommendation or not is probably still unclear.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.e805b98bd5a7888c194fe1df0b9a161f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2016-01-05 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by ask):

 * cc: ask@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.0d59066f9d7c9d1cfb50407e4aba66c0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-12-29 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by teeberg):

 * cc: teeberg (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.09b149bdda3c5f991d2cedc741cbacdf%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-12-27 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by ernestoalejo):

 * cc: contact@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.d339f30eae7f6ecc6174925f23039841%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-11-23 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by timgraham):

 Thanks Collin, in that case your proposal makes more sense to me. It could
 be nice to get a consensus from more MySQL users though.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.7d17a9a6a78db214ead94b08a6268de8%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-11-23 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by collinanderson):

 I'm not thinking of limiting the _name_ of the index. The issue is "the
 maximum number of characters that can be indexed".

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.2a5082f82bdc1264a5c3f43835cac495%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-11-23 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by timgraham):

 In case it's not clear, here's the problematic scenario I see:

 * Indexes are created with max index name length=191 and truncated
 accordingly.
 * Developer increases max index name length setting to 200 (no changes to
 index names in the database).
 * Now migrations can't operate on existing indexes because the 200
 character index names it generates aren't what's in the database.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.b1f460140947d0a03f7827a5ce5bfa38%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-11-22 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by collinanderson):

 I was thinking it wouldn't actually change the name of the index, but I
 haven't actually looked at the code. :)

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.16afd620e372565f82d92b4ebbf8fe30%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-11-20 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by timgraham):

 Will that setting work nicely with migrations though? I think we need to
 know the index names for some operations like `AlterField`. It seems
 problematic if we have a way that users can vary the index names without
 updating existing names.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.2f467b0033bcc98b2f376309428516c8%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-11-12 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by collinanderson):

 For a first step, I propose a a setting in DATABASES OPTIONS that tells
 Django the maximum index size to use when creating new indexes. I think it
 should default to 191. That way Django won't ever reduce the size of
 existing indexes, only new indexes.

 If we have that settings, it's at least _possible_ to use utf8mb4. We
 could then down the road have Django try to default to using utf8mb4
 encoding for new databases/tables/columns.

 (We need this for #20846.)

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.3e96afb03d76af3560ed239342ce1ec5%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-11-12 Thread Django

Comment (by teeberg):

Replying to [comment:24 aigarius]:
> [...] have Django migration backend generate all the code for the
migration [...]

Even the utf8 to utf8mb4 migration, which would be the easiest in terms of
required changes, may cause data loss if you have indexes that are longer
than 191 characters and should thus probably be inspected and fixed up
individually and manually. For that reason, it seems impossible to me to
automate. That being said, clear migration instructions would probably be
helpful for many users.

Maybe this could be another backend-specific setting, similar to what was
implemented for integer types in 1506c71a95cd7f58fbc6363edf2ef742c58d2487?
Although, it only applying to index columns may make it way more painful
to implement.

--
Ticket URL:
Django
The Web framework for perfectionists with deadlines.

--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/071.4c12c064ac2004e776dd8838976306b6%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-05-07 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by aigarius):

 https://mathiasbynens.be/notes/mysql-utf8mb4 has more information on this.
 Ideally it would be wonderful if Django users could set an option and have
 Django migration backend generate all the code for the migration to the
 full utf8mb4 encoding for all databases, tables, columns and corresponding
 changes to all indexes as well. And then on next major version this option
 could become the new default.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.2c043ab73222cdc223ccf36da69cd79d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2015-01-22 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  utf8mb4 mysql| Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by collinanderson):

 * cc: cmawebsite@… (added)


Comment:

 One solution would be to reduce the INDEX size to 191 for mysql, like the
 example above:

 col1 VARCHAR(500) CHARACTER SET utf8mb4, INDEX (col1(191))"

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.27c092dafcca69c7f14cb86ca05452d1%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2014-07-04 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  utf8mb4 mysql|  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  1
  Needs tests:  1|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by flisky):

 * cc: flisky (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.596f9aafe397b8d92cdb9812a2ed8d61%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2014-04-30 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  utf8mb4 mysql|  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  1
  Needs tests:  1|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by donturner):

 Replying to [comment:20 moggers87]:
 > Replying to [comment:19 anonymous]:
 > *snip*
 > >
 > > and {{{CREATE TABLE}}} or {{{ALTER TABLE}}} with the
 {{{ROW_FORMAT=DYNAMIC}}} attribute.
 > [[BR]]
 > I'm not aware of any way to specify options like `ROW_FORMAT=DYNAMIC`
 for create/alter statements in Django other than specifying `SET
 ROW_FORMAT=DYNAMIC` with 'init_command' - but that adds overhead to each
 connection (and feels a bit hacky imo)

 Having run into the same issue my conclusion is that the only proper
 solution is to switch to PostgreSQL. It would appear that MySQL has
 extremely bad support for indexing on utf8mb4 fields.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.694845d36998fec2aa65932ff58a91ab%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2013-11-21 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  utf8mb4 mysql|  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  1
  Needs tests:  1|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by moggers87):

 * cc: moggers87 (added)


Comment:

 Replying to [comment:19 anonymous]:
 *snip*
 >
 > and {{{CREATE TABLE}}} or {{{ALTER TABLE}}} with the
 {{{ROW_FORMAT=DYNAMIC}}} attribute.
 [[BR]]
 I'm not aware of any way to specify options like `ROW_FORMAT=DYNAMIC` for
 create/alter statements in Django other than specifying `SET
 ROW_FORMAT=DYNAMIC` with 'init_command' - but that adds overhead to each
 connection (and feels a bit hacky imo)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.18cc33e35376385180815f0a61d4777f%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2013-10-31 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  utf8mb4 mysql|  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  1
  Needs tests:  1|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by anonymous):

 This [http://dev.mysql.com/doc/refman/5.5/en/innodb-restrictions.html
 InnoDB restriction] was fixed in MySQL 5.5.14. You have to set the
 following:

 [http://dev.mysql.com/doc/refman/5.5/en/innodb-
 parameters.html#sysvar_innodb_large_prefix innodb_large_prefix]=ON
 innodb_file_per_table=ON
 innodb_file_format=Barracuda

 and {{{CREATE TABLE}}} or {{{ALTER TABLE}}} with the
 {{{ROW_FORMAT=DYNAMIC}}} attribute.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.0673de00126248861aa6d28cccb88a89%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2013-10-30 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  utf8mb4 mysql|  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  1
  Needs tests:  1|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by timo):

 #21308 describes an issue with running tests on MySQL when charset=utf8mb4
 that I've marked as a duplicate of this.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.33953276880da856ab60422e0b3e5433%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2013-08-19 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  utf8mb4 mysql|  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  1
  Needs tests:  1|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by cvrebert):

 * cc: cvrebert (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/071.32afac801f1f8e371ab91104ae671644%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2013-03-22 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  utf8mb4 mysql|  Needs documentation:  0
Has patch:  1|  Patch needs improvement:  1
  Needs tests:  1|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by aaugustin):

 * stage:  Design decision needed => Accepted


Comment:

 This is just another case of MySQL being purposefully and irreversibly
 brain-damaged. What it calls utf8 in actually a non-standard 3-bytes
 encoding unrelated to utf8.

 Django should give the option to use utf8mb4, and maybe recommend it in
 the docs (if it works well; non-default features of MySQL rarely work
 well).

 The comments above suggest this isn't possible right now, although I'm
 still confused as to the nature of the problem. Accepting on this basis.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2013-02-22 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  utf8mb4 mysql|  decision needed
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by akaariai):

 * type:  Uncategorized => New feature


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2013-02-14 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  utf8mb4 mysql|  decision needed
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by streeter):

 * cc: django@… (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-10-12 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  utf8mb4 mysql|  decision needed
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by evax):

 The fix mentioned above has been merged to master (
 https://github.com/farcepest/MySQLdb1) and released
 (http://pypi.python.org/pypi/MySQL-python/1.2.4b5)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-10-11 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  utf8mb4 mysql|  decision needed
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by evax):

 There is a fix for this issue in the upcoming MySQLdb-python release
 (https://github.com/farcepest/MySQLdb1/tree/utf8mb4)

 This should be used in together with the appropriate OPTIONS in the
 database config:
 {{{
 DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.mysql',
 'OPTIONS': {'charset': 'utf8mb4'},
 (...)
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-09-24 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  utf8mb4 mysql|  decision needed
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by kitsunde):

 * cc: kitsunde@… (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-08-16 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  utf8mb4 mysql|  decision needed
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by kmichel_wgs):

 As a workaround, you can make python understand 'utf8mb4' as an alias for
 'utf8':
 {{{
 import codecs
 codecs.register(lambda name: codecs.lookup('utf8') if name == 'utf8mb4'
 else None)
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-06-15 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  utf8mb4 mysql|  decision needed
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by rogeliorv):

 * keywords:  hack utf8mb4 mysql => utf8mb4 mysql
 * needs_better_patch:  0 => 1
 * has_patch:  0 => 1
 * needs_tests:  0 => 1


Comment:

 Replying to [comment:8 EmilStenstrom]:
 > Replying to [comment:7 rogeliorv]:
 > > As a way to test it. The hack consists in adding self.query('SET NAMES
 utf8mb4') in MySQLdb.connections in Connection.set_character_set function
 as shown here: http://pastebin.com/MW5BgRgP
 > >
 > > Of course the correct way would be to change this in django when
 setting up the cursor connection.
 >
 > Did your hack remove the exception? What was the rationale behind the
 hack? What's the next step?


 Yes, the hack removed the exception. The rationale followed was to make
 the mysql client to use a certain encoding.

 The next step is to make django's mysql connections to use utf8mb4 by
 default or otherwise make it more configurable. Since utf8bm4 is utf8
 compatible, there should be no extra changes in that regard.


 To achieve this django.db.base.cursor should be changed in class
 DatabaseWrapper function _cursor, (complete function definition here
 http://pastebin.com/A6dMEMd4):

 ''kwargs = {

   "conv": django_conversions,
   "charset": "utf8mb4",
   "use_unicode": True,
 }
 ''

 Unfortunately this won't work unless we also change MySQLdb.connections
 class Connection function set_character_set:


 Change the two bottom lines to (complete function definition here:
 http://pastebin.com/AMN1B8za)

 #Hack so data can be decoded/encoded using python's utf8 since
 # python does not know about mysql utf8mb4

 ''if charset == 'utf8mb4':''
 ''charset = 'utf8'''

 ''self.string_decoder.charset = charset''

 ''self.unicode_literal.charset = charset''


 This will guarantee you can use special characets like ☺

 Unlike the previous hack, which worked on reading/writing data, this patch
 only allows me to read data in utfmb4 format, but now I've hit an error on
 insertion/creation where I get 'Cursor' object has no attribute
 '_last_executed'.  I will report evidence on this error as I find it. All
 your help regarding this error is appreciated.

 You can reach me via twitter, @rogeliorv

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-06-15 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  hack utf8mb4 mysql   |  decision needed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by EmilStenstrom):

 Replying to [comment:7 rogeliorv]:
 > As a way to test it. The hack consists in adding self.query('SET NAMES
 utf8mb4') in MySQLdb.connections in Connection.set_character_set function
 as shown here: http://pastebin.com/MW5BgRgP
 >
 > Of course the correct way would be to change this in django when setting
 up the cursor connection.

 Did your hack remove the exception, or what was the rationale behind it?
 What's the next step here?

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-06-14 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:  hack utf8mb4 mysql   |  decision needed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by rogeliorv):

 * keywords:   => hack utf8mb4 mysql


Comment:

 As proof of concept I tried inserting into mysql the following values in a
 varchar.

 ☺

 And got a "Warning: Incorrect string value" from mysql (which python
 treats as an exception)

 As a way to test it. The hack consists in adding self.query('SET NAMES
 utf8mb4') in MySQLdb.connections in Connection.set_character_set function
 as shown here: http://pastebin.com/MW5BgRgP

 Of course the correct way would be to change this in django when setting
 up the cursor connection.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-06-05 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:   |  decision needed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by EmilStenstrom):

 Maybe this ticket just needs some exploration before going further: A
 simple test that tries to save and their fetch some 4-byte unicode
 characters to MySQL... I think that would reveal what the minimal changes
 are, for full unicode support to be possible.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-06-04 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:   |  decision needed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by kmtracey):

 I don't believe setting the connection charset to this new utf8mb4
 encoding (if we're talking to a server that supports it) would cause any
 problem with the indexing: that's a problem that comes into play when you
 change the database charset itself. Whether we need to change that isn't
 clear to me. It's possible that these new characters (stored in a DB with
 utf8mb4 encoding) would be transmitted over a connection that has charset
 set simply to utf8. If so then there's really no point in making any
 connection charset change. So one open question here is: do you get an
 error or incorrect behavior trying to read/write these characters over a
 utf8 connection created by Django when operating on a database with
 utf8mb4 encoding?

 Traditionally Django has not gone beyond advising in the documentation
 what charset to use (uft8) for the database. Django doesn't attempt to set
 the charset for stuff it creates to utf8, it just uses the default charset
 for the DB, which is set before Django ever gets involved. Given this new
 MySQL 5.5 support for "more better" unicode we probably need to update the
 docs to mention the new option for database charset.

 Having the index creation fail for a too-long !CharField is a nuisance. I
 know we've had tickets before that dealt with this issue, but I don't
 recall offhand what the status is. Django tries to disallow creation of a
 !CharField where the index creation will fail, but MySQL makes it
 incredibly difficult to figure out what the right value for "max allowed"
 is. This new encoding just makes for more of a mess there.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-05-28 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Design
 Keywords:   |  decision needed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by akaariai):

 * stage:  Accepted => Design decision needed


Comment:

 I am marking this as design decision needed. This will need a good
 solution which guarantees trouble-free upgrades for current users. While
 it would be nice to have full UTF8 support with MySQL, I don't think it is
 worth risking a breakage for existing users.

 So, looking for good solutions here. Does MySQL infer the used character
 set from some runtime variable (perhaps something set at CREATE DATABASE
 time)? If so, we could just check the used charset for the db, and use
 that for test database creation and index creation. If so, this would be a
 simple solution to this problem.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-05-28 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by EmilStenstrom):

 Ok, there's some trickiness here. Indexes in InnoDB tables can't be longer
 than 255 chars with utf8, but only 191 chars with utf8mb4. This means that
 the default indexes that Django makes for CharField(max_length=255) is too
 long, and will break things (break what? I'm running a migration that
 converts all my tables to utf8mb4 automatically, and setting utf8mb4 on a
 long charfield breaks because the index is too long).

 From the official docs:


 {{{
 "InnoDB has a maximum index length of 767 bytes, so for utf8 or utf8mb4
 columns, you can index a maximum of 255 or 191 characters, respectively.
 If you currently have utf8 columns with indexes longer than 191
 characters, you will need to index a smaller number of characters. In an
 InnoDB table, these column and index definitions are legal:

 col1 VARCHAR(500) CHARACTER SET utf8, INDEX (col1(255))

 To use utf8mb4 instead, the index must be smaller:

 col1 VARCHAR(500) CHARACTER SET utf8mb4, INDEX (col1(191))"

 }}}

 From: http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-
 upgrading.html

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-05-28 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by EmilStenstrom):

 I don't know enough about MySQL and the ORM to answer your questions, I
 hope someone else does.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-05-28 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
-+-
 Reporter:  EmilStenstrom|Owner:  nobody
 Type:  Uncategorized|   Status:  new
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by akaariai):

 * needs_better_patch:   => 0
 * needs_docs:   => 0
 * needs_tests:   => 0
 * stage:  Unreviewed => Accepted


Comment:

 To me it seems the character encoding is set on connect to 'charset':
 'utf8'. Is it enough to change this from 'utf8' to 'utf8mb4' to change
 this default. If not, where should this default encoding be defined? Does
 this mean that any text column must be VARCHAR(N) CHARACTER SET utf8mb4 on
 creation?

 I am accepting this as to me the change sounds valid.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

[Django] #18392: Use utf8mb4 encoding with MySQL 5.5

2012-05-28 Thread Django

#18392: Use utf8mb4 encoding with MySQL 5.5
--+
 Reporter:  EmilStenstrom |  Owner:  nobody
 Type:  Uncategorized | Status:  new
Component:  Database layer (models, ORM)  |Version:  1.4
 Severity:  Normal|   Keywords:
 Triage Stage:  Unreviewed|  Has patch:  0
Easy pickings:  0 |  UI/UX:  0
--+
 Background:
 I just ran into a problem with iPhone emoji characters being saved into
 MySQL. The text was cut off after the first emoji character. After some
 research I found which explains how it works:
 http://mzsanford.wordpress.com/2010/12/28/mysql-and-unicode/

 The recommendation is to use MySQL 5.5, and the "utf8mb4" encoding.

 Suggestion:
 Make "utf8mb4" the default encoding for MySQL 5.5 and up.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

44 matches

Mail list logo