Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  Marcos Daniel Petry  |Owner:  nobody
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  needsinfo
 Keywords:  oracle database  | Triage Stage:  Accepted
  nclob clob |
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Shai Berger):

 * cc: felixx (added)
 * status:  new => closed
 * resolution:   => needsinfo


Comment:

 Apparently this hasn't been a problem lately, and the original description
 and suggested solution have been made largely irrelevant.

 If you have a similar problem you can reproduce, please re-open, or, more
 usefully, just open a new ticket.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.90c2d97084665ff0e091fe57411a16e4%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  Marcos Daniel Petry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  oracle database  | Triage Stage:  Accepted
  nclob clob |
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"950171d7b20a148a964281aed16aff3d2c734ab4" 950171d]:
 {{{
 #!CommitTicketReference repository=""
 revision="950171d7b20a148a964281aed16aff3d2c734ab4"
 Refs #11487 -- Removed redundant test_long_string() test.

 Redundant with model_regress.tests.ModelTests.test_long_textfield
 since 3ede430b9a94e3c2aed64d2cf898920635bdf4ae.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.fa73ea1b4b93c9bffd40da181aa8e220%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  oracle database  | Triage Stage:  Accepted
  nclob clob |
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by RohitV24):

 We are currently experiencing the same issue while inserting large strings
 (between size 2000-4000 characters) into the Oracle database.The
 NLS_NCHAR_CHARACTERSET on our database is  AL16UTF16 which assigns 2 bytes
 for a char .From digging in a little deeper, it looks like a string is
 mapped to cx_Oracle.STRING which is then mapped to either a VARCHAR,
 NVARCHAR or LONG in Oracle and the conversion to long in case of long
 values is causing the error. It looks like the issue with 4000 characters
 was fixed by setting the input size to cx_Oracle.CLOB when it reached the
 character limit. Using 2000( for utf-16) seems to work fine and solve the
 problem. Would setting the comparison value to 1000 (taking into
 consideration other encoding formats) before setting it to CLOB be the fix
 for this issue?

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.b89b51fc047d22dabf54331700af6792%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  oracle database  | Triage Stage:  Accepted
  nclob clob |
Has patch:  1|  Needs documentation:  0
  Needs tests:  1|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Arpit10jain):

 Is there any update on this issue? we are still experiencing this issue
 with 2000 characters. It happens usually when it has unicode characters in
 it. I read about the work around to solve this issue but its not an ideal
 solution, Does anyone have a fix for this.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.5915f3491635e476feb3cb88ed0a91da%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  1|
Easy pickings:  0|
-+-

Comment (by timo):

 Sorry, I was referring to a test for the patch that was added as an
 attachment since the ticket was reopened (`long_string.diff`).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.d907f45922fc419e868820fa524d5222%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  1|
Easy pickings:  0|
-+-

Comment (by ikelly):

 The test is at backends.OracleChecks.test_long_string and was added in
 [11285].  I note that it claims to test strings longer than 4000
 characters but actually tests a string of exactly 4000 characters; this is
 probably not important as the Oracle limitation is actually 4000
 ''bytes'', and the string in question is more than 4000 bytes in any
 encoding.

 The test is probably not reliable though, as the details of reproducing
 this issue seem to be dependent on the configuration of the database.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.af52ae807f10f10896f7ba2fcb75d577%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  1|
Easy pickings:  0|
-+-
Changes (by timo):

 * needs_tests:  0 => 1


Comment:

 I'm not an Oracle expert to comment on the fix, but I don't see a test
 that's integrated as part of Django's test suite so marking as "needs
 tests".

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.92c477c562ec3b0c6fcbe6adc828fef8%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by graham.boyle@…):

 and the NLS database parameters of interest are:
 NLS_CHARACTERSET AL32UTF8
 NLS_NCHAR_CHARACTERSET AL16UTF16
  and django_session.session_data is nclob

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.d660752973edb27f411c5dfc22b90a24%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by graham.boyle@…):

 We're seeing this when putting more than 4 message.success(..) on a page.
 It seems the success messages go onto the session, and the session_data
 gets too big.

 The len() of the string going into the SESSION_DATA field is 2120, but
 that's a unicode string, so I _strongly suspect_ the byte length of the
 data going into the django_session.session_data column is twice that.
 Looking at the HEX values of django_session.session_data already in the
 database, we're seeing 0x6C00 for "1", for example.
 We are using TextFields in our models with no problem, even with large
 chunks of text, and even when creating a new row. We see the sql being run
 to save the session is
 u'INSERT INTO "DJANGO_SESSION" ("SESSION_KEY", "SESSION_DATA",
 "EXPIRE_DATE") SELECT %s, %s, %s FROM DUAL'
 from a call to cursor.execute(sql, params)
 at line 937 of django\db\models\sql\compiler.py in execute_sql()

 We note that there's a bulk_insert_sql() in
 django.db.backends.oracle.base.py (line 411) that's doing inserts from
 selects off dual.

 We're using Oracle 11g, Django 1.5, Python 2.7.3, the python running under
 Windows 7 Professional (64 bit), Service Pack 1.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/065.dd2b2895c82fce22150dda703c8a7765%40djangoproject.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by mboersma):

 * cc: mboersma (removed)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by shai):

 * cc: shai@… (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-
Changes (by CollinAnderson):

 * cc: CollinAnderson (removed)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
 Reporter:  mdpetry  |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:  oracle database  |  Needs documentation:  0
  nclob clob |  Patch needs improvement:  0
Has patch:  1|UI/UX:  0
  Needs tests:  0|
Easy pickings:  0|
-+-

Comment (by aaugustin):

 #20201 was probably a duplicate.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
   Reporter:  mdpetry|  Owner:  nobody
   Type:  Bug| Status:  reopened
  Milestone:  1.1|  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:  oracle database
Needs documentation:  0  |  nclob clob
Patch needs improvement:  0  |  Has patch:  1
  UI/UX:  0  |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by aaugustin):

 * ui_ux:   => 0


Comment:

 #9152 was a duplicate.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
   Reporter:  mdpetry|  Owner:  nobody
   Type:  Bug| Status:  reopened
  Milestone:  1.1|  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:  oracle database
Needs documentation:  0  |  nclob clob
Patch needs improvement:  0  |  Has patch:  1
 |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by kimus.linuxus@…):

 * easy:   => 0


Comment:

 Same problem here and changing column to CLOB resolved my issue with
 ORA-12704: character set mismatch

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
-+-
   Reporter:  mdpetry|Owner:  nobody
 Status:  reopened   |Milestone:  1.1
  Component:  Database   |  Version:  SVN
  layer (models, ORM)| Keywords:  oracle database nclob
 Resolution: |  clob
   Triage Stage:  Accepted   |Has patch:  1
Needs documentation:  0  |  Needs tests:  0
Patch needs improvement:  0  |
-+-
Changes (by Goldan):

 * status:  closed => reopened
 * keywords:  oracle database => oracle database nclob clob
 * resolution:  fixed =>
 * cc: Goldan (added)


Comment:

 This issue seems to be a bit more complicated.


 {{{
 self.input_size = Database.CLOB
 }}}


 causes the following error when inserting more than 4000 characters.


 {{{
 DatabaseError: ORA-12704: character set mismatch
 }}}


 Changing it to


 {{{
 self.input_size = Database.LONG_STRING
 }}}


 (as initially suggested by the reporter) works ok.

 I've attached the patch and a test case that fails with CLOB.

 I'm using cx_Oracle version 5.0.4 with unicode support and Oracle9i
 Enterprise Edition Release 9.2.0.1.0 - Production.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  0  

Needs_better_patch:  0 |  
---+
Comment (by kmtracey):

 11487_fix.diff also fixes this test failure:

 {{{
 ==
 FAIL: Doctest: regressiontests.model_regress.models.__test__.API_TESTS
 --
 Traceback (most recent call last):
   File "d:\u\kmt\django\trunk\django\test\_doctest.py", line 2180, in
 runTest
 raise self.failureException(self.format_failure(new.getvalue()))
 AssertionError: Failed doctest test for
 regressiontests.model_regress.models.__test__.API_TESTS
   File
 "D:\u\kmt\django\trunk\tests\regressiontests\model_regress\models.py",
 line unknown line number, in API_TESTS

 --
 File
 "D:\u\kmt\django\trunk\tests\regressiontests\model_regress\models.py",
 line ?, in regressiontests.model_regress.models.__test__.API_TESTS
 Failed example:
 len(a4.article_text)
 Expected:
 5000
 Got:
 2500
 }}}

 for my test install of Oracle (also on a Windows box).  I've seen this
 failure ever since I set up a test Oracle install, but never found time to
 dig into what might be causing it.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  0  

Needs_better_patch:  0 |  
---+
Comment (by rafax):

 Replying to [comment:17 ikelly]:
 > Thanks, rafax.  I've actually tested with the same set of versions and
 not reproduced the bug, so there must be something more to it.  I suspect
 it has something to do with the database encodings.

 Or with the OS you run - I was able to reproduce on my work machine
 running Windows XP (with default CP-1250 encoding), while on my personal
 notebook running Ubuntu (with UTF based locale) it works OK.

 > About the test case, it appears to be saving a string of exactly 4000
 characters, not more than 4000 characters.  Is that intentional?

 I thought it will certainly be longer than 4000 this way, in fact
 something like
 {{{
 >>> len(''.join(unicode(x) for x in xrange(2000)))
 6890
 }}}
  should be enough.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  0  

Needs_better_patch:  0 |  
---+
Comment (by jbronn):

 Replying to [comment:17 ikelly]:
 > Thanks, rafax.  I've actually tested with the same set of versions and
 not reproduced the bug, so there must be something more to it.  I suspect
 it has something to do with the database encodings.
 >

 I was able to reproduce the bug using Oracle 11, Python 2.5.4, and
 cx_Oracle 4.4.1.  However, when using cx_Oracle 5.0.1 it's not a problem.
 I'm doing this on Windows, so this may be why I can reproduce.

 > About the test case, it appears to be saving a string of exactly 4000
 characters, not more than 4000 characters.  Is that intentional?  Also,
 does it still replicate if you use an NCLOB column instead of a CLOB
 column?  Since that is the default for Django, that is what we should be
 testing.

 I thought that at first, but if you evaluate `len(''.join([unicode(x) for
 x in xrange(4000)]))` you see it is really 14,890 characters long.  I also
 updated the test to use a `NCLOB` column, and the test case still fails
 without the patch.

 Ian, look over my patch and make sure it doesn't break anything else --
 I've run the full test suite but I want a second run on a different system
 to make sure there are no unintended side-effects.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  0  

Needs_better_patch:  0 |  
---+
Changes (by ikelly):

  * needs_tests:  1 => 0

Comment:

 Thanks, rafax.  I've actually tested with the same set of versions and not
 reproduced the bug, so there must be something more to it.  I suspect it
 has something to do with the database encodings.

 About the test case, it appears to be saving a string of exactly 4000
 characters, not more than 4000 characters.  Is that intentional?  Also,
 does it still replicate if you use an NCLOB column instead of a CLOB
 column?  Since that is the default for Django, that is what we should be
 testing.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by rafax):

 Sorry for the anonymous comment from yesterday.[[BR]]
 I managed to replicate the bug using Oracle 10, Python 2.6.2, Django SVN,
 cx_Oracle 4.4.1, which might indicate the improper behavior is connected
 with the version of Oracle/ cx_Oracle.[[BR]]
 I wrote a simple test case as suggested by ikelly, if there is something
 that should be corrected in it please let me know.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Changes (by CollinAnderson):

 * cc: CollinAnderson (added)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Changes (by mboersma):

 * cc: mboersma, ikelly (added)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by ikelly):

 And it hasn't been committed yet because it still needs a regression test.
 Tests aren't optional.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by Alex):

 This ticket is still open because no code has been committed to the Django
 repository.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by anonymous):

 so is the bug fixed? why is the ticket still open? :\

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by anonymous):

 Replying to [comment:8 dg]:
 > I belive it should say "as _a_ CLOB" (just a grammar nitpick ;))

 I just attached a patch with the correct grammar.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by dg):

 I belive it should say "as _a_ CLOB" (just a grammar nitpick ;))

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by mdpetry):

 Replying to [comment:6 Alex]:
 > Please upload the diffs from the root of the source tree, not just for
 the single file.  Also there's no need to keep uploading the source file,
 just the diff is enough.
 Sorry Alex, I'm still adjusting to the process of submission of patches

 I believe that now the file is ok:
 [http://code.djangoproject.com/attachment/ticket/11487/base.3.diff
 base.3.diff]

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by Alex):

 Please upload the diffs from the root of the source tree, not just for the
 single file.  Also there's no need to keep uploading the source file, just
 the diff is enough.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Comment (by mdpetry):

 Using CLOB also works perfectly! I've added the file with the diff
 ([http://code.djangoproject.com/attachment/ticket/11487/base.2.diff
 base.2.diff]) and source code
 ([http://code.djangoproject.com/attachment/ticket/11487/base.2.py
 base.2.py])

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #11487: Oracle encoding bug when saving more than 4000 characters

[Django]

#11487: Oracle encoding bug when saving more than 4000 characters
---+
  Reporter:  mdpetry   | Owner:  nobody 

Status:  new   | Milestone:  1.1

 Component:  Database layer (models, ORM)  |   Version:  SVN

Resolution:|  Keywords:  oracle 
database
 Stage:  Accepted  | Has_patch:  1  

Needs_docs:  0 |   Needs_tests:  1  

Needs_better_patch:  0 |  
---+
Changes (by ikelly):

  * summary:  Oracle does not register in the batabase items with more than
  4000 characters => Oracle encoding bug when
  saving more than 4000 characters
  * version:  1.0 => SVN
  * stage:  Unreviewed => Accepted

Comment:

 Glad you were able to get this working.  Would you mind testing out for me
 the same change but using Database.CLOB rather than Database.LONG_STRING?
 The reason is I'm concerned that LONG_STRING might cause problems in other
 configurations.  Also, the patch in #10566 already changes it from NCLOB
 to CLOB, so I'd prefer a solution that both tickets can agree on.

 Also, since I can't reproduce this myself, would you please write up a
 test case for this?  It doesn't need to be complicated.  Just inserting a
 large string, reading it back, and checking that it's the same would be
 fine.  The main thing is we need to be sure the string used will actually
 trigger the bug.

 Thanks!

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---