First, thank you to both Avraham and James. In my view, it isn't said often
enough on the internet, and I've had many questions go without a response
at all. So please understand that I mean it very much when I say I deeply
appreciate the time you took to help me.
Now as to the matter at hand: The field in question is 'sunsetdate',
referring to the date something ends. It must be optional since most of my
objects/data don't, or haven't, ended. The first time I tried this I had
the empty string, '', in all rows in that column.
(cannon)malikarumi@Tetuoan2:~/Projects/cannon/jamf$ python manage.py
loaddata essell/fixtures/test22byhand.json
Traceback (most recent call last):
File
"/home/malikarumi/Projects/cannon/local/lib/python2.7/site-packages/django/core/serializers/python.py",
line 174, in Deserializer
raise base.DeserializationError.WithData(e, d['model'],
d.get('pk'), field_value)
django.core.serializers.base.DeserializationError: Problem installing
fixture
'/home/malikarumi/Projects/cannon/jamf/essell/fixtures/test22byhand.json':
[u"'' value has an invalid date format. It must be in YYYY-MM-DD format."]:
(essell.Code:pk=None) field_value was ''
I replaced “”, - the empty string - with ',', that is, nothing, no string,
no quote, nothing but the comma
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
django.core.serializers.base.DeserializationError: Problem installing
fixture
'/home/malikarumi/Projects/cannon/jamf/essell/fixtures/test22byhand.json':
No JSON object could be decoded
So now I assume this is because there are no quotes on sunsetdate. Note how
that one error in that one field makes the whole document 'not json'.
Now I have to put the date in the way they want it, which is not what I
want but not the end of the world since I don't have to display this field
if there is no actual sunset date.
File
"/home/malikarumi/Projects/cannon/local/lib/python2.7/site-packages/django/core/serializers/python.py",
line 174, in Deserializer
raise base.DeserializationError.WithData(e, d['model'],
d.get('pk'), field_value)
django.core.serializers.base.DeserializationError: Problem installing
fixture
'/home/malikarumi/Projects/cannon/jamf/essell/fixtures/test22byhand.json':
[u"'null' value has an invalid date format. It must be in YYYY-MM-DD
format."]: (essell.Code:pk=None) field_value was 'null'
django.core.serializers.base.DeserializationError: Problem installing
fixture
'/home/malikarumi/Projects/cannon/jamf/essell/fixtures/test22byhand.json':
[u"'0000-00-00' value has the correct format (YYYY-MM-DD) but it is an
invalid date."]: (essell.Code:pk=None) field_value was '0000-00-00'
Last night before posting I hacked serializers/python.py to make an
exception for sunsetdate, and that apparently worked, but now I have this:
File
"/home/malikarumi/Projects/cannon/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py",
line 2390, in get_db_prep_value
value = uuid.UUID(value)
File "/usr/lib/python2.7/uuid.py", line 134, in __init__
raise ValueError('badly formed hexadecimal UUID string')
ValueError: Problem installing fixture
'/home/malikarumi/Projects/cannon/jamf/essell/fixtures/test22byhand.json':
badly formed hexadecimal UUID string
The only uuid currently in this fixture is the one I got from Django when I
put another model (only one row) in with a fixture (no date field) and it
worked. I needed the uuid of that object to put into the foreign key of the
model I am having trouble with now.
Finally, this morning after reading your responses I decided to try putting
in null with no comma, since I had already tried it with no quotes, and of
course that didn't work. But then in fooling with it I put null in right
next to the colon, with no spaces or quotes, followed by a comma, and this
time null turned blue in my code editor, which suggests it was working. But
then I got the 'badly formed hexadecimal UUID string error again.
In sum, there is a special syntax to using null which is not obvious, but
thanks for making me take another look and try again. The uuid error seems
to be something else. There is a bugfix on github from November of 2014, so
I don't (yet) know why I'm having this issue. I may post that later as a
separate question. Thanks again!
On Wed, Feb 17, 2016 at 3:51 AM, James Schneider <jrschneide...@gmail.com>
wrote:
>
>
> On Tue, Feb 16, 2016 at 4:35 PM, Malik Rumi <malik.a.r...@gmail.com>
> wrote:
>
>> There are a ton of answers to this question out there - if you can wade
>> through all the ones that refer to forms and not models. The consensus
>> seems to be that datetime has to be set to both blank=true and null=true.
>> And when I put in one test row of data manually through the admin, I could
>> leave my datetime field blank with no problems. But when I tried to load
>> the 89 other instances through a fixture, I got error after error.
>>
>
> A blank form field is different than a missing field in a JSON dump, as
> each is handled differently within the Django workflow. A programmatic way
> to look at this would be: (blank=True) != (null=True).
>
> blank=True controls whether or not a form will accept an empty value in
> the form field as part of the validation mechanism for that field. It has
> no bearing on model objects created through other means, such as part of a
> fixture, or some view code that runs foo.save() somewhere.
>
> Conversely, null=True only applies to models and saving to the database,
> and even then, is not part of the validation mechanism when foo.save() is
> called. It really just serves as a flag to the model inspection mechanism
> for generating the schema that the database uses. Enforcement of missing
> fields or fields with a Null/None value is done at the database level, not
> at the model level within Python/Django. Obviously there are ways to inject
> validation at the model level to prevent such issues, but very few, if any,
> are used by default.
>
> In form fields that accept arbitrary text input, a blank field is
> submitted as an empty string. DateTime form fields are actually the same
> under the hood, but most use some fancy JS to pretty it up with date-time
> pop up selectors. If left blank in the form, an empty string is sent
> through the form submission. Django then coerces the data (empty string)
> for the DateTime form field to None for use later by the view/model, which
> is not the case for other CharField-type fields that are left as an empty
> string. Compare the empty values:
>
> https://docs.djangoproject.com/en/1.9/ref/forms/fields/#charfield
> https://docs.djangoproject.com/en/1.9/ref/forms/fields/#datetimefield
>
> When a blank (empty) value is provided through a form for a DateTime
> *form* field, the DateTime *model* field will be populated with None, and
> the resulting foo.save() operation will save the DateTime *model* field
> as NULL in the database.
>
> That's why for *model* CharField definitions (and other similar model
> fields), it is a common pattern to only see blank=True, and NOT null=True,
> because model CharFields are either saved with the text that was submitted,
> or an empty string, but never as NULL (even with null=True).
>
>
>>
>> As near as I can tell, if you leave a json value blank, loaddata says it
>> isn't a json document. But if you put the empty string, '', Null, None,
>> "Null", "None" or "0000-00-00" in there, it is rejected as not valid date
>> format. Therefore, it seems there is no way to do this through a fixture.
>> My question: Is this correct? If so, is there any alternative to doing it
>> all manually? Thanks.
>>
>
> When you are talking fixtures, everything I just mentioned about forms is
> thrown out (unless of course you are manually submitting the data in your
> fixtures programmatically through Django forms, which is not entirely
> uncommon since you can take advantage of form validation to eliminate/clean
> bad data rows during an import). I only mentioned forms to help understand
> why a 'blank' value in the Admin seems to work fine but your fixture is
> failing.
>
> The likely reason your fixture is failing is because you are expecting the
> coercion of a missing/blank field to be handled like it is when data is
> submitted through the form process. Django is expecting fixture data to be
> pre-processed already from a known-good data source, and I believe that the
> data is inserted using very low-level database API calls, and is not using
> the standard ORM magic that we all know and love.
>
> When you say you "leave a json value blank", what do you mean? Are you
> removing the field entirely from the object definition? It appears that you
> are keeping the field defined, but you want the equivalent of Python's
> None. Python equates None to null (bare keyword in JSON), so your field
> should probably be something like "updated_on: null," (without quotes). See
> the table here:
> https://docs.python.org/3/library/json.html#py-to-json-table
>
> Anyway, it sounds like the data source you are using is not correctly
> serializing what is to become your problematic DateTime model fields. You
> have these choices:
>
>
> 1. Fix the data source and have it generate/serialize to a proper
> representation of your data, using either a real date or null to represent
> an 'empty' or 'no date specified' value. An empty string does not make
> sense as a value for a DateTime field (hence the coercion gymnastics that
> form fields perform). Preferred.
> 2. Write a custom management command to manually de-serialize the data
> and create it using the ORM and forms to take advantage of the Django
> batteries. Not as graceful or efficient, but does grant you pretty explicit
> control over how incoming data is filtered, cleaned, and inserted into the
> database if you need to go that far. It probably isn't as bad as it sounds.
> Most of my management commands are less than 10 lines, aside from imports,
> but I'm not doing any fancy (de)serialization of incoming data, either.
>
> I'm hoping that this entire email wasn't just a TL;DR; use null (without
> quotes) to represent an empty datetime field in JSON...
>
> It would also be helpful to see the errors and tracebacks that you are
> getting when loading your fixtures. I could be completely wrong here,
> especially if the error is something silly like your JSON file is missing a
> comma or semi-colon.
>
> -James
>
> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/db367bcf-8cbf-4001-8976-f602e93576d5%40googlegroups.com
>> <https://groups.google.com/d/msgid/django-users/db367bcf-8cbf-4001-8976-f602e93576d5%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-users/Q4zybgExDyY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CA%2Be%2BciW%3Ddm%3DM9Eh10Y8zcB%2Bvjx6E0UQC6a0mzsyyELmb2x2qzA%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-users/CA%2Be%2BciW%3Ddm%3DM9Eh10Y8zcB%2Bvjx6E0UQC6a0mzsyyELmb2x2qzA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAKd6oByb3O7vhYcuVmWiqrrJf1ErHg7Mg4fyvQoht%2BhyvTWTLw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
