Oh, I found my old patch to force UTF-8 on Android. I didn't test it:
see attached file.
It would be nice to start a wiki page to collect all informations on
the Python port to Android.
Victor
2015-01-30 21:04 GMT+01:00 Victor Stinner <[email protected]>:
> Android provides a minimal support of locales. Most functions return a
> fake result, do nothing. I'm not sure that it supports any codec. To
> support Android, we may force UTF-8 for the filesystem encoding, as
> done on Mac OS X.
>
> Victor
>
> 2015-01-30 19:04 GMT+01:00 Ryan Gonzalez <[email protected]>:
>> No...
>>
>> ...but I think I found the issue with grep. Try applying the attached patch
>> to the Python/frozenmain.c. It comments out the locale handling.
>>
>> It seems that Python always calls its strdup function on the locale string.
>> On Android, this can apparently be null (as seen in the bug report you
>> linked to).
>>
>> On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton <[email protected]> wrote:
>>>
>>> I don't have gdb on device; does the following tell you where Python's
>>> strdup is called?
>>>
>>> >> _PyMem_RawStrdup
>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>
>>> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez <[email protected]> wrote:
>>> > Is it possible at all to get a stack trace of the crash using gdb? Try
>>> > the
>>> > steps here.
>>> >
>>> > That way we can see where Python's own strdup function is getting
>>> > called.
>>> >
>>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <[email protected]>
>>> > wrote:
>>> >>
>>> >> Absolutely. Good thing I have addr2line on device
>>> >>
>>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>>> >> 0008bbc8
>>> >> _PyMem_RawStrdup
>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>> >> /bld/python/Python-3.4.2 $
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <[email protected]> wrote:
>>> >> > Could you try the steps at
>>> >> > http://stackoverflow.com/a/11369475/2097780?
>>> >> > They
>>> >> > allow you to get a better idea of where libc is crashing.
>>> >> >
>>> >> > Cyd Haselton <[email protected]> wrote:
>>> >> >>
>>> >> >> Managed to get this out of logcat:
>>> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>>> >> >> 11914 (python) (libc)
>>> >> >>
>>> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc ]
>>> >> >> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
>>> >> >> (python)
>>> >> >>
>>> >> >> Less detail than strace but it seems to be that python is
>>> >> >> segfaulting
>>> >> >> libc...
>>> >> >>
>>> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <[email protected]>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>>> >> >>> <[email protected]>
>>> >> >>> wrote:
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> What I see in the strace:
>>> >> >>>>
>>> >> >>>> ... load libpython3.4m.so.1.0
>>> >> >>>> ... load libm
>>> >> >>>> ... open /dev/__properties__ and do something to it
>>> >> >>>> (what?)
>>> >> >>>> ... get current time
>>> >> >>>> ... allocate memory
>>> >> >>>> ... getuid
>>> >> >>>> ... segfault
>>> >> >>>>
>>> >> >>>> That's not a lot to go on, but it doesn't look as if it has
>>> >> >>>> started
>>> >> >>>> to
>>> >> >>>> load modules yet.
>>> >> >>>>
>>> >> >>>> Does /dev/__properties__ ring a bell? Not to me.
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>> >> >>> is the code that works with that file.
>>> >> >>>
>>> >> >>> This explains it a bit (slides 24-29). Looks like something to do
>>> >> >>> with
>>> >> >>> interprocess communication. Likely has nothing to do with Python
>>> >> >>> itself.
>>> >> >>>
>>> >> >>> Maybe this would be useful?
>>> >> >>>
>>> >> >>>>
>>> >> >>>> That stack trace would be really helpful.
>>> >> >>>>
>>> >> >>>> On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>>> >> >>>> <[email protected]>
>>> >> >>>> wrote:
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>> Apologies...I'm not sure what a stack track is, but I do have
>>> >> >>>>> the
>>> >> >>>>> strace. Nearest I can tell, it happens due to an open call,
>>> >> >>>>> though
>>> >> >>>>> I
>>> >> >>>>> am probably wrong.
>>> >> >>>>> Attaching the strace output to this email. I'm going to head
>>> >> >>>>> back
>>> >> >>>>> to
>>> >> >>>>> the documentation and to back out of some Android-related
>>> >> >>>>> changes
>>> >> >>>>> in
>>> >> >>>>> _localemodule.c
>>> >> >>>>>
>>> >> >>>>> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>>> >> >>>>> <[email protected]>
>>> >> >>>>> wrote:
>>> >> >>>>>>
>>> >> >>>>>> There could be a million differences relevant (unicode, ints,
>>> >> >>>>>> ...).
>>> >> >>>>>> Perhaps
>>> >> >>>>>> the importlib bootstrap is failing. Perhaps the dynamic loading
>>> >> >>>>>> code
>>> >> >>>>>> changed. Did you get a stack track? (IIRC strace shows a
>>> >> >>>>>> syscall
>>> >> >>>>>> trace
>>> >> >>>>>> --
>>> >> >>>>>> also useful, but doesn't tell you precisely how
>>> >> >>>>>> it segfaulted.)
>>> >> >>>>>>
>>> >> >>>>>> On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>>> >> >>>>>> <[email protected]>
>>> >> >>>>>> wrote:
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>> All,
>>> >> >>>>>>> I recently ditched my attempts to port Python 2.7.8 to Android
>>> >> >>>>>>> in
>>> >> >>>>>>> favor of Python 3.4.2. Unfortunately, after using the same
>>> >> >>>>>>> configure
>>> >> >>>>>>> options in the same environment, and modifying the setup.py as
>>> >> >>>>>>> needed,
>>> >> >>>>>>> the newly built binary throws a segfault when the
>>> >> >>>>>>> generate-posix-vars
>>> >> >>>>>>> portion of the build is reached...and when it is run as well
>>> >> >>>>>>> (i.e.
>>> >> >>>>>>> ./python --help, ./python -E -S -m sysconfig, or similar)
>>> >> >>>>>>>
>>> >> >>>>>>> I took a strace of ./python, however I'm a bit lost when
>>> >> >>>>>>> reviewing
>>> >> >>>>>>> it.
>>> >> >>>>>>> Any ideas as to what may be going on...i.e. why Python 2.7
>>> >> >>>>>>> works
>>> >> >>>>>>> but
>>> >> >>>>>>> 3.x throws a segfault?
>>> >> >>>>>>>
>>> >> >>>>>>> Thanks in advance,
>>> >> >>>>>>> Cyd
>>> >> >>>>>>> ________________________________
>>> >> >>>>>>>
>>> >> >>>>>>> Python-Dev mailing list
>>> >> >>>>>>>
>>> >> >>>>>>> [email protected]
>>> >> >>>>>>> https://mail.python.org/mailman/listinfo/python-dev
>>> >> >>>>>>> Unsubscribe:
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>> --
>>> >> >>>>>> --Guido van Rossum (python.org/~guido)
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> --
>>> >> >>>> --Guido van Rossum (python.org/~guido)
>>> >> >>>>
>>> >> >>>> ________________________________
>>> >> >>>>
>>> >> >>>> Python-Dev mailing list
>>> >> >>>> [email protected]
>>> >> >>>> https://mail.python.org/mailman/listinfo/python-dev
>>> >> >>>> Unsubscribe:
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> --
>>> >> >>> Ryan
>>> >> >>> If anybody ever asks me why I prefer C++ to C, my answer will be
>>> >> >>> simple:
>>> >> >>> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>> >> >>> that
>>> >> >>> was
>>> >> >>> nul-terminated."
>>> >> >>> Personal reality distortion fields are immune to contradictory
>>> >> >>> evidence.
>>> >> >>> -
>>> >> >>> srean
>>> >> >>> Check out my website: http://kirbyfan64.github.io/
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>> >> > Check out my website: http://kirbyfan64.github.io/
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Ryan
>>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>> > nul-terminated."
>>> > Personal reality distortion fields are immune to contradictory evidence.
>>> > -
>>> > srean
>>> > Check out my website: http://kirbyfan64.github.io/
>>
>>
>>
>>
>> --
>> Ryan
>> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> nul-terminated."
>> Personal reality distortion fields are immune to contradictory evidence. -
>> srean
>> Check out my website: http://kirbyfan64.github.io/
>>
>> _______________________________________________
>> Python-Dev mailing list
>> [email protected]
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>>
diff -r 1927f47a1838 Modules/main.c
--- a/Modules/main.c Wed Jul 30 19:24:32 2014 -0400
+++ b/Modules/main.c Fri Aug 01 14:15:50 2014 +0200
@@ -540,7 +540,7 @@ Py_Main(int argc, wchar_t **argv)
oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL));
setlocale(LC_ALL, "");
for (p = strtok(buf, ","); p != NULL; p = strtok(NULL, ",")) {
-#ifdef __APPLE__
+#if defined(__APPLE__) || defined(__ANDROID__)
/* Use utf-8 on Mac OS X */
unicode = PyUnicode_FromString(p);
#else
diff -r 1927f47a1838 Objects/unicodeobject.c
--- a/Objects/unicodeobject.c Wed Jul 30 19:24:32 2014 -0400
+++ b/Objects/unicodeobject.c Fri Aug 01 14:15:50 2014 +0200
@@ -3205,13 +3205,22 @@ wcstombs_errorpos(const wchar_t *wstr)
static int
locale_error_handler(const char *errors, int *surrogateescape)
{
+
if (errors == NULL) {
+#ifdef __ANDROID__
+ *surrogateescape = 1;
+#else
*surrogateescape = 0;
+#endif
return 0;
}
if (strcmp(errors, "strict") == 0) {
+#ifdef __ANDROID__
+ *surrogateescape = 1;
+#else
*surrogateescape = 0;
+#endif
return 0;
}
if (strcmp(errors, "surrogateescape") == 0) {
@@ -3340,7 +3349,7 @@ PyUnicode_EncodeFSDefault(PyObject *unic
{
#ifdef HAVE_MBCS
return PyUnicode_EncodeCodePage(CP_ACP, unicode, NULL);
-#elif defined(__APPLE__)
+#elif defined(__APPLE__) || defined(__ANDROID__)
return _PyUnicode_AsUTF8String(unicode, "surrogateescape");
#else
PyInterpreterState *interp = PyThreadState_GET()->interp;
@@ -3626,7 +3635,7 @@ PyUnicode_DecodeFSDefaultAndSize(const c
{
#ifdef HAVE_MBCS
return PyUnicode_DecodeMBCS(s, size, NULL);
-#elif defined(__APPLE__)
+#elif defined(__APPLE__) || defined(__ANDROID__)
return PyUnicode_DecodeUTF8Stateful(s, size, "surrogateescape", NULL);
#else
PyInterpreterState *interp = PyThreadState_GET()->interp;
@@ -4811,7 +4820,7 @@ onError:
return NULL;
}
-#ifdef __APPLE__
+#if defined(__APPLE__) || defined(__ANDROID__)
/* Simplified UTF-8 decoder using surrogateescape error handler,
used to decode the command line arguments on Mac OS X.
diff -r 1927f47a1838 Python/bltinmodule.c
--- a/Python/bltinmodule.c Wed Jul 30 19:24:32 2014 -0400
+++ b/Python/bltinmodule.c Fri Aug 01 14:15:50 2014 +0200
@@ -24,7 +24,7 @@
#ifdef HAVE_MBCS
const char *Py_FileSystemDefaultEncoding = "mbcs";
int Py_HasFileSystemDefaultEncoding = 1;
-#elif defined(__APPLE__)
+#elif defined(__APPLE__) || defined(__ANDROID__)
const char *Py_FileSystemDefaultEncoding = "utf-8";
int Py_HasFileSystemDefaultEncoding = 1;
#else
diff -r 1927f47a1838 Python/fileutils.c
--- a/Python/fileutils.c Wed Jul 30 19:24:32 2014 -0400
+++ b/Python/fileutils.c Fri Aug 01 14:15:50 2014 +0200
@@ -18,7 +18,7 @@
#include <fcntl.h>
#endif /* HAVE_FCNTL_H */
-#ifdef __APPLE__
+#if defined(__APPLE__) || defined(__ANDROID__)
extern wchar_t* _Py_DecodeUTF8_surrogateescape(const char *s, Py_ssize_t size);
#endif
@@ -63,7 +63,7 @@ PyObject *
Py_RETURN_NONE;
}
-#if !defined(__APPLE__) && !defined(MS_WINDOWS)
+#if !defined(__APPLE__) && !defined(__ANDROID__) && !defined(MS_WINDOWS)
extern int _Py_normalize_encoding(const char *, char *, size_t);
/* Workaround FreeBSD and OpenIndiana locale encoding issue with the C locale.
@@ -213,7 +213,7 @@ encode_ascii_surrogateescape(const wchar
}
#endif /* !defined(__APPLE__) && !defined(MS_WINDOWS) */
-#if !defined(__APPLE__) && (!defined(MS_WINDOWS) || !defined(HAVE_MBRTOWC))
+#if !defined(__APPLE__) && !defined(__ANDROID__) && (!defined(MS_WINDOWS) || !defined(HAVE_MBRTOWC))
static wchar_t*
decode_ascii_surrogateescape(const char *arg, size_t *size)
{
@@ -260,7 +260,7 @@ decode_ascii_surrogateescape(const char
wchar_t*
_Py_char2wchar(const char* arg, size_t *size)
{
-#ifdef __APPLE__
+#if defined(__APPLE__) || defined(__ANDROID__)
wchar_t *wstr;
wstr = _Py_DecodeUTF8_surrogateescape(arg, strlen(arg));
if (size != NULL) {
@@ -403,7 +403,7 @@ oom:
char*
_Py_wchar2char(const wchar_t *text, size_t *error_pos)
{
-#ifdef __APPLE__
+#if defined(__APPLE__) || defined(__ANDROID__)
Py_ssize_t len;
PyObject *unicode, *bytes = NULL;
char *cpath;
_______________________________________________
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
