Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: reopened | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 1 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Changes (by claudep): * needs_better_patch: 1 => 0 * has_patch: 0 => 1 Comment: Just attached a patch using the idea on comment 12 from andialbrecht. I choose to use the encode-at-the-last-moment approach. Note that only the addresses really used by SMTP to send the messages are idna encoded, not the addresses in the message headers. -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:18> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: reopened | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 1 | ---+ Comment (by philomat): Replying to [comment:16 jezdez]: > the encoding with idna needs to happen earlier in the life of an !EmailMessage, say in the `__init__` when the different recipient attributes are handled anyway (see 2). I think encoding during `__init__` is a bad idea since you might want to change those object variables at a later time. Maybe set private object vars during `__init__` and make `to, cc, bcc` properties delivering encoded values? -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:17> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: reopened | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 1 | ---+ Changes (by jezdez): * needs_better_patch: 0 => 1 Comment: Replying to [comment:13 philomat]: > The relevant Wikipedia article (German only, http://de.wikipedia.org/wiki/E-Mail- Adresse#Der_Dom.C3.A4nenteil_.28Domain_Part.29) says that with the introduction of IDN, nothing changes technically with regard to the SMTP protocol: Characters above ASCII code #127 are illegal. It is the client's responsibility to convert to an IDNA string. > > Hence, andialbrechts approach seems to be absolutely correct. Indeed, and the actual culprit seems to be the fact that to, cc and bcc attributes of an !EmailMessage are only idna encoded when calling its message() method (and blankly passed by the SMTP backend by using the recipients() method (see 1)). In other words, the encoding with idna needs to happen earlier in the life of an !EmailMessage, say in the `__init__` when the different recipient attributes are handled anyway (see 2). 1: http://code.djangoproject.com/browser/django/trunk/django/core/mail/message.py?rev=14216#L173 2: http://code.djangoproject.com/browser/django/trunk/django/core/mail/message.py?rev=14216#L121 -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:16> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: reopened | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by adamnelson): One problem that this ticket exposes is that now we allow unicode email addresses for user accounts (#9764) but those people will never be able to receive email through Django - until it supports send() to IDN email addresses. I almost wonder if #9764 should be undone with regards to the email address until Django can send to IDN email addresses. -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:15> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: reopened | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by adamnelson): @philomat is right. Even in gmail, they won't accept IDN domain names like: t...@façonnable.com -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:14> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: reopened | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by philomat): The relevant Wikipedia article (German only, http://de.wikipedia.org/wiki/E-Mail- Adresse#Der_Dom.C3.A4nenteil_.28Domain_Part.29) says that with the introduction of IDN, nothing changes technically with regard to the SMTP protocol: Characters above ASCII code #127 are illegal. It is the client's responsibility to convert to an IDNA string. Hence, andialbrechts approach seems to be absolutely correct. -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:13> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld)
---+
Reporter: harm | Owner: nobody
Status: reopened | Milestone: 1.3
Component: django.core.mail | Version: 1.2
Resolution:| Keywords:
Stage: Accepted | Has_patch: 0
Needs_docs: 0 | Needs_tests: 0
Needs_better_patch: 0 |
---+
Comment (by andialbrecht):
Maybe we need something like this in the SMTP backend:
{{{
>>> email = u'f...@bär.com'
>>> name, domain = email.split('@', 1)
>>> email = '@'.join([name, domain.encode('idna')])
>>> email
<<< u'f...@xn--br-eda5w.com'
}}}
But I'm not very familiar with IDN and the depths of SMTP, so this
approach could be wrong.
--
Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:12>
Django <http://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld)
---+
Reporter: harm | Owner: nobody
Status: reopened | Milestone: 1.3
Component: django.core.mail | Version: 1.2
Resolution:| Keywords:
Stage: Accepted | Has_patch: 0
Needs_docs: 0 | Needs_tests: 0
Needs_better_patch: 0 |
---+
Comment (by philomat):
The exact same steps to reproduce as listed by the original poster will
spawn this traceback *if* you are using the SMTP backend. The email tests
are probably not using it, which is why the error went unnoticed.
So, for the following example would show the problem:
{{{
>>> email = u'f...@bär.com'
>>> send_mail('subject','message','f...@example.com',[email])
}}}
--
Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:11>
Django <http://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: reopened | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by jezdez): Without giving specific details what actually spawned this traceback, I don't see a way to see the reason. Providing a test case would be the best way. -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:10> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld)
---+
Reporter: harm | Owner: nobody
Status: reopened | Milestone: 1.3
Component: django.core.mail | Version: 1.2
Resolution:| Keywords:
Stage: Accepted | Has_patch: 0
Needs_docs: 0 | Needs_tests: 0
Needs_better_patch: 0 |
---+
Changes (by philomat):
* status: closed => reopened
* resolution: fixed =>
Comment:
While one part of the problem is fixed, the SMTP backend still needs to be
fixed since it is not expecting non-ASCII characters.
{{{
File
"/Sites/vendor/Django-1.3-alpha-1/django/core/mail/__init__.py", line 61,
in send_mail
connection=connection).send()
File "/Sites/vendor/Django-1.3-alpha-1/django/core/mail/message.py",
line 186, in send
return self.get_connection(fail_silently).send_messages([self])
File
"/Sites/vendor/Django-1.3-alpha-1/django/core/mail/backends/smtp.py", line
85, in send_messages
sent = self._send(message)
File
"/Sites/vendor/Django-1.3-alpha-1/django/core/mail/backends/smtp.py", line
101, in _send
email_message.message().as_string())
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/smtplib.py",
line 703, in sendmail
(code,resp)=self.rcpt(each, rcpt_options)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/smtplib.py",
line 457, in rcpt
self.putcmd("rcpt","TO:%s%s" % (quoteaddr(recip),optionlist))
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/smtplib.py",
line 318, in putcmd
self.send(str)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/smtplib.py",
line 305, in send
self.sock.sendall(str)
File "", line 1, in sendall
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in
position 16: ordinal not in range(128)
}}}
--
Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:9>
Django <http://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: new | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by SmileyChris): Is the try/except even needed? If the name gets always encoded to str using the charset, why not just assume we should do this for the email address to? -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:6> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: new | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by jezdez): Hm, I can't seem to reproduce the error, tbh. Can someone confirm the test breakage? -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:5> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: new | Milestone: 1.3 Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Changes (by jezdez): * milestone: => 1.3 -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:4> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: new | Milestone: Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by andialbrecht): For the record, support for IDN in validate_email() was added in r12474 (#9764). -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:3> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: new | Milestone: Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Comment (by anonymous): I added such test to regressiontests/mail.py and it worked fine. -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:2> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
Re: [Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld) ---+ Reporter: harm | Owner: nobody Status: new | Milestone: Component: django.core.mail | Version: 1.2 Resolution:| Keywords: Stage: Accepted | Has_patch: 0 Needs_docs: 0 | Needs_tests: 0 Needs_better_patch: 0 | ---+ Changes (by lukeplant): * needs_better_patch: => 0 * stage: Unreviewed => Accepted * needs_tests: => 0 * needs_docs: => 0 -- Ticket URL: <http://code.djangoproject.com/ticket/14301#comment:1> Django <http://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To post to this group, send email to django-upda...@googlegroups.com. To unsubscribe from this group, send email to django-updates+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-updates?hl=en.
[Django] #14301: django crashes on email address that passed validate_email() (utf8-tld)
#14301: django crashes on email address that passed validate_email() (utf8-tld)
--+-
Reporter: harm | Owner: nobody
Status: new | Milestone:
Component: django.core.mail | Version: 1.2
Keywords:| Stage: Unreviewed
Has_patch: 0 |
--+-
== description ==
It seems validate_email() and send_email() in django don't agree on the
validness of email addresses whenever utf8 is used in the tld.
I'm not sure what is currently allowed the domain part, there is an
experimental RFC 5335, but at least django should agree with itself on
this matter.
== steps to reproduce ==
{{{
#!python
python manage.py shell
>>> from django.core.validators import validate_email
>>> from django.core.mail import send_mail
>>> email = u'chw08...@nyc.odn.ne.j\uff43'
>>> validate_email(email)
>>> send_mail('subject','message','f...@example.com',[email])
}}}
== result ==
{{{
Traceback (most recent call last):
File "", line 1, in
File "/opt/python2.5/lib/python2.5/site-
packages/django/core/mail/__init__.py", line 61, in send_mail
connection=connection).send()
File "/opt/python2.5/lib/python2.5/site-
packages/django/core/mail/message.py", line 175, in send
return self.get_connection(fail_silently).send_messages([self])
File "/opt/python2.5/lib/python2.5/site-
packages/django/core/mail/backends/smtp.py", line 85, in send_messages
sent = self._send(message)
File "/opt/python2.5/lib/python2.5/site-
packages/django/core/mail/backends/smtp.py", line 101, in _send
email_message.message().as_string())
File "/opt/python2.5/lib/python2.5/site-
packages/django/core/mail/message.py", line 147, in message
msg['To'] = ', '.join(self.to)
File "/opt/python2.5/lib/python2.5/site-
packages/django/core/mail/message.py", line 86, in __setitem__
name, val = forbid_multi_line_headers(name, val, self.encoding)
File "/opt/python2.5/lib/python2.5/site-
packages/django/core/mail/message.py", line 70, in
forbid_multi_line_headers
result.append(formataddr((nm, str(addr
UnicodeEncodeError: 'ascii' codec can't encode character u'\uff43' in
position 21: ordinal not in range(128)
}}}
== expected result ==
Either
1. validate_email() should reject the email address,
or
2. send_email() should handle the message and attempt to deliver it.
== version ==
django 1.2
--
Ticket URL: <http://code.djangoproject.com/ticket/14301>
Django <http://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
