Ok, thank you for the fast answer as always.
Moreover the specs for the JSON datatype for MySQL say:
A JSON column cannot have a default value.
So I think I will just skip the default value.
Thank you by the way again!
_
-. .´ |∞∞∞∞
', ; |∞∞∞∞∞∞
˜˜ |∞∞∞∞∞∞∞∞∞ RdB
,., |∞∞∞∞∞∞
.' '. |∞∞∞∞
-' `’
https://rdb.is
On 30 agosto 2018 a 17:09:33, Mike Bayer (mike...@zzzcomputing.com) scritto:
On Thu, Aug 30, 2018 at 6:32 AM, Ruben Di Battista
<rubendibatti...@gmail.com> wrote:
> Ehi Mike, thank you :).
>
> I just went ahead in implementing a custom type for a Python object that
is
> a (subclass of) numpy array and that is stored as JSON in the DB.
>
> This is the Python class:
>
> class ElevationMask(np.ndarray):
> r""" A class to represent an elevation mask.
>
> This class proxies the functionality of a numpy array encoding the
> elevation mask as a functional dependency of the type:
>
> .. math::
>
> \epsilon_\text{mask} = \epsilon_\text{mask}(\theta)
>
> So the elevation mask is described as the mapping
> :math:`\epsilon_\text{mask}` that for each value of azimuth angle
> :math:`\theta` maps the corresponding elevation angle value
>
>
> Args:
> elevation_mask(numpy.ndarray): The elevation mask as numpy array.
> It's
> an 2xN array. First row is the azimuth values. Second row is
> elevation values
> Attributes:
> azimuth(numpy.ndarray): The values of the azimuth angles that are
> part
> of the elevation mask
> elevation(numpy.ndarray): The corrisponding elevation values to the
> azimuth values
> """
>
> def __new__(cls, elevation_mask):
> obj = np.asarray(elevation_mask).view(cls)
> rows, cols = obj.shape
>
> if not(rows == 2):
> raise IndexError("The elevation mask array given has not the "
> "right shape. It needs to be a 2xN array.")
>
> return obj
>
> @property
> def azimuth(self):
> return self[0, :]
>
> @property
> def elevation(self):
> return self[1, :]
>
> def __call__(self, azimuth_value):
>
> # Interpolation bounds checking
> if azimuth_value < self.azimuth.min() or \
> azimuth_value > self.azimuth.max():
> raise ValueError("The required azimuth "
> "value is out of interpolation bounds ")
>
> return np.interp(azimuth_value,
> self.azimuth,
> self.elevation)
>
> @staticmethod
> def _to_json(obj):
> """ This method returns the dict representation for JSON encoding
> """
>
> return {
> '__type__': type(obj).__name__,
> 'entries': {
> 'azimuth': obj.azimuth.tolist(),
> 'elevation': obj.elevation.tolist()
> }
> }
>
> def to_json(self):
> """ This actually dumps the instance into the JSON string """
>
> return json.dumps(self, default=self._to_json)
>
> @staticmethod
> def _from_json(json_dict):
> """ This reconstitutes the Python object from the JSON serialization
> """
> if '__type__' in json_dict:
> if json_dict['__type__'] == ElevationMask.__name__:
> mask_entries = json_dict['entries']
> azimuth = mask_entries['azimuth']
> elevation = mask_entries['elevation']
>
> el_mask_array = np.array([
> azimuth,
> elevation
> ])
>
> return ElevationMask(el_mask_array)
>
> return json_dict
>
> @classmethod
> def from_json(self, json_dict):
> return json.loads(json_dict, object_hook=self._from_json)
>
>
> So it's basically a 2xN array.
>
> Then I have the actual custom type
>
> from sqlalchemy.dialects.mysql import JSON
>
> class JSONElevationMask(TypeDecorator):
> """ This type emits a JSON object into the Database. When instead
> loading
> the JSON from the database, it transforms the JSON in a ElevationMask
> object (i.e. a numpy-like object with few additional features """
>
> impl = JSON
>
> def process_bind_param(self, value, dialect):
> if value is not None:
> return value.to_json()
>
> def process_load_param(self, value, dialect):
> if value is not None:
> return ElevationMask.from_json(value)
>
> It seems to work as expected. The only problem I'm experiencing is when I
> setup a default value for a Column of this type:
>
> elevation_mask = Column('elevationMask', JSONElevationMask,
> default=ElevationMask([
> [0, 2*PI],
> [0, 0]])
> )
it looks like that object you have has many special behaviors,
including __new__, __call__, and is also a descendant of a C object,
so some combination of those things which I can't exactly reproduce
here is causing SQLAlchemy's check for passing of a "callable" to
fail. The "default" parameter of a Column can accept a Python
callable, so this object can't be passed as is since it is already a
non-compatible callable. Your best bet is to just pass the JSON
itself as a SQL object, e.g. text("'%s'" %
ElevationMask(...).to_json()). SQLAlchemy will use the text()
construct as is to generate a SQL string that will be inline with the
INSERT statement.
>
> When I do this, I get
> File
>
"/Users/rubendibattista/.envs/pyggdrasill/lib/python2.7/site-packages/sqlalchemy/sql/schema.py",
> line 1259, in __init__
> args.append(ColumnDefault(self.default))
> File
>
"/Users/rubendibattista/.envs/pyggdrasill/lib/python2.7/site-packages/sqlalchemy/sql/schema.py",
> line 2067, in __init__
> arg = self._maybe_wrap_callable(arg)
> File
>
"/Users/rubendibattista/.envs/pyggdrasill/lib/python2.7/site-packages/sqlalchemy/sql/schema.py",
> line 2100, in _maybe_wrap_callable
> argspec = util.get_callable_argspec(fn, no_self=True)
> File
>
"/Users/rubendibattista/.envs/pyggdrasill/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
> line 330, in get_callable_argspec
> return get_callable_argspec(fn.__call__, no_self=no_self)
> File
>
"/Users/rubendibattista/.envs/pyggdrasill/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
> line 317, in get_callable_argspec
> if no_self and (_is_init or fn.__self__):
> ValueError: The truth value of an array with more than one element is
> ambiguous. Use a.any() or a.all()
>
> That is because a `numpy` array cannot be checked for truth. Is there a
way
> to monkey patch that `if no_self` in order to use the right method for
the
> array to be true (so .any() or .all())?
>
>
> On Monday, July 23, 2018 at 7:51:15 PM UTC+2, Mike Bayer wrote:
>>
>> On Mon, Jul 23, 2018 at 1:28 PM, Ruben Di Battista
>> <rubendi...@gmail.com> wrote:
>> > Hello,
>> > I need to store a matrix into the database and I was evaluating the
>> > possibility to have a column of JSON type in MySQL to store it. What I
>> > would
>> > like to achieve is the possibility of operating on a numpy array when
>> > manipulating that column on Python while keeping a "meaningful" data
>> > type on
>> > the DB, if possible.
>> >
>> > I was reading the Custom Types section of the documentation and, even
>> > though
>> > the application I envisage looks like just an extension to the JSON
>> > datatype, I believe to have understood that the `TypeDecorator`
approach
>> > can't store a state (i.e. the numpy array). Is that correct? So do I
>> > need to
>> > use the `UserDefined` subclassing method?
>>
>> when you want to use a special kind of datatype on the Python side,
>> and there is already a SQLAlchemy type on the database side that does
>> what you want, you use TypeDecorator. Types don't "store" states,
>> database columns do - you'd need to define how to marshal your numpy
>> array into JSON and back out again. you don't need to use
>> UserDefinedType. If you can create numpy->json and json->numpy
>> functions, I can show you how to stick that onto TypeDecorator.
>>
>>
>>
>>
>>
>>
>>
>> >
>> > Thanks in advance,
>> >
>> > --
>> > SQLAlchemy -
>> > The Python SQL Toolkit and Object Relational Mapper
>> >
>> > http://www.sqlalchemy.org/
>> >
>> > To post example code, please provide an MCVE: Minimal, Complete, and
>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>> > description.
>> > ---
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "sqlalchemy" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to sqlalchemy+...@googlegroups.com.
>> > To post to this group, send email to sqlal...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/sqlalchemy.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and
Verifiable Example. See http://stackoverflow.com/help/mcve for a full
description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
