Re: [C++-sig] [boost python] : how to pass a tuple of lists from python to C++

[HOUSSEN Franck]

OK, I understand this is a type related problem. I found a workaround
(dummyTest3.py - init numpy arrays with list seems to work).

But, I unfortunately do not get how to change the code to get dummyTest2.py
to work ?!... Should I read reinterpret_cast'ed data sizeof(double) by
sizeof(double) : seems cryptic (?!), what's the natural way to do that ?

Also a question raises : if the size of the data type you want to pass from
python to C++ is smaller than sizeof(double), python will always allocate
MORE memory than needed ? Why is that ? In case the size is large, the
extra memory allocated may be large and trigger problems like slowdown
(swapping) or out-of-memory, no ?... Why is this like so ?

Can somebody help to fix the dummy code : this would be the best
explanation !... :D

Franck

Le mar. 11 févr. 2020 à 14:57, Jakob van Santen <jvansan...@gmail.com> a
écrit :

>
> > On Feb 11, 2020, at 13:24, HOUSSEN Franck <fghous...@gmail.com> wrote:
> >
> > Finally able to reproduce the "real" problem with a "dummy" example :
> seems that, at python side, when you use "np.append" C++ get screwed data
> ?!... (note that with or without "np.append" all is OK at python side).
> > Can somebody help here ? Known problem ? Possible workaround or fix ?
> Should I open a bug for that ? (if yes where)
>
> This is a side-effect of how np.append() broadcasts its arguments to find
> a common type. You've asked it to append a Python float (which is secretly
> a double) to an array of np.float32. The only safe way to do that is to
> cast both to numpy.float64, which np.append() happily does for you. Before
> you reinterpret_cast the data pointer of an ndarray to a given type, you
> need to check whether the dtype is compatible (and also, strictly speaking,
> whether the flags are CARRAY or CARRAY_RO).
>
> There's a whole lot more background information in the Numpy C API docs:
> https://docs.scipy.org/doc/numpy/reference/c-api.html
>
> Cheers,
> Jakob
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig@python.org
> https://mail.python.org/mailman/listinfo/cplusplus-sig
>

#include <iostream>

#include <boost/python.hpp>
#include <boost/python/numpy.hpp>

namespace bp = boost::python;
namespace np = boost::python::numpy;

void doStuffs(boost::python::tuple & t) {
  boost::python::extract<np::ndarray> lsIntExt(t[0]);
  if (lsIntExt.check()) {
    np::ndarray lsInt = lsIntExt(); // Conversion to np::ndarray.
    int nbInt = lsInt.shape(0);
    int * ptrInt = reinterpret_cast<int*>(lsInt.get_data());
    for(auto k = 0; ptrInt && k < nbInt; k++) {std::cout << "ptrInt[" << k << "] = " << ptrInt[k] << ", " << ptrInt+k << std::endl;};
  }

  boost::python::extract<np::ndarray> lsFloatExt(t[1]);
  if (lsFloatExt.check()) {
    np::ndarray lsFloat = lsFloatExt(); // Conversion to np::ndarray.
    int nbFloat = lsFloat.shape(0);
    float * ptrFloat = reinterpret_cast<float*>(lsFloat.get_data());
    for(auto k = 0; k < nbFloat; k++) {std::cout << "ptrFloat[" << k << "] = " << ptrFloat[k] << ", " << ptrFloat+k << std::endl;};
  }

  boost::python::extract<np::ndarray> lsDoubleExt(t[2]);
  if (lsDoubleExt.check()) {
    np::ndarray lsDouble = lsDoubleExt(); // Conversion to np::ndarray.
    int nbDouble = lsDouble.shape(0);
    double * ptrDouble = reinterpret_cast<double*>(lsDouble.get_data());
    for(auto k = 0; k < nbDouble; k++) {std::cout << "ptrDouble[" << k << "] = " << ptrDouble[k] << ", " << ptrDouble+k << std::endl;};
  }
}

BOOST_PYTHON_MODULE(dummy) {
  np::initialize();
  def("doStuffs", doStuffs);
}

#!/usr/bin/env python

import numpy as np
ls = [1, 2, 3]
lsInt = np.array(ls, dtype='int32')
ls = [1.2, 2.3, 3.4]
lsFloat = np.array(ls, dtype='float32')
ls = [4.5, 5.6, 6.7]
lsDouble = np.array(ls, dtype='float64')

t = (lsInt, lsFloat, lsDouble)
import dummy

print(lsInt)
print(lsFloat)
print(lsDouble)
dummy.doStuffs(t)

#!/usr/bin/env python

import numpy as np
lsInt = np.array([1, 2, 3], dtype='int32')
lsFloat = np.array([1.2, 2.3, 3.4], dtype='float32')
lsDouble = np.array([4.5, 5.6, 6.7], dtype='float64')

t = (lsInt, lsFloat, lsDouble)
import dummy

print(lsInt)
print(lsFloat)
print(lsDouble)
dummy.doStuffs(t)

Makefile
Description: Binary data

#!/usr/bin/env python

import numpy as np
lsInt = np.array([], dtype='int32')
lsInt = np.append(lsInt, 1)
lsInt = np.append(lsInt, 2)
lsInt = np.append(lsInt, 3)
lsFloat = np.array([], dtype='float32')
lsFloat = np.append(lsFloat, 1.2)
lsFloat = np.append(lsFloat, 2.3)
lsFloat = np.append(lsFloat, 3.4)
lsDouble = np.array([], dtype='float64')
lsDouble = np.append(lsDouble, 4.5)
lsDouble = np.append(lsDouble, 5.6)
lsDouble = np.append(lsDouble, 6.7)

t = (lsInt, lsFloat, lsDouble)
import dummy

print(lsInt)
print(lsFloat)
print(lsDouble)
dummy.doStuffs(t)

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@python.org
https://mail.python.org/mailman/listinfo/cplusplus-sig