Re: formset initialization question

[Margie]

Ok - so one more day of working through this did the trick and I got
through all of my issues and am successfully using modelformset, both
when creating new objects and when editing existing ones.  I thought
I'd recap here in case anyone is searching for formset info, and also
to give any developers that are reading this some examples of new and
different usage issues.


First, in order to create a formset for a bunch of objects that don't
yet exist (ie, the form is the user's way of creating the objects) I
am setting initial inside _construct_form(), as Malcolm suggested:

def _construct_form(self, i, **kwargs):
kwargs["tile"] = self.tiles[i]
kwargs["initial"] = {'tile':self.tiles[i].id}
return super(TaskDetailCustomBaseFormSetForCreate,
self)._construct_form(i, **kwargs)

I think that this is the best way to create initial data with a
modelformset.  I was never successful using the initial argument when
constructng the formset from the formset factory class and I think
that would probably require a lot more hacking get it to work.  Once
you use the initial argument, ie like this:

TaskTileDetailFormSet = modelformset_factory(TaskTileDetail,
 
form=TaskTileDetailForm,
 
formset=TaskTileDetailCustomBaseFormSetForCreate)


taskTileDetailFormSet = TaskTileDetailFormSet(data=data,
  tiles=tileList,
  initial=
[{"tile":tile.id} for tile in tileList])

it seems that the modelformset code thinks you have 'n' existing
objects, and you get errors like this:

-> kwargs['instance'] = self.get_queryset()[i]
(Pdb) n
IndexError: 'list index out of range'

So for anyone using modelformsets - put your initial data into
construct_form().

One thing that I spent some time thinking about was whether I could
identify which form was which by indexing into my formset during the
POST.  I think Malcolm's suggestion was that I create the formset for
the GET, and then create the same formset for the POST (but with
postdict data this time).  I think the point was that since I am
creating the data myself during the POST, I should know what's what
and shouldn't be able to just index in at the right point.  After
thinking about this for some time, I think there is a problem with
this approach, at least for my app.


Let's say my app displays a form that presents the users with book
titles for a set of books, and the user is filling out the author and
submitting the form.  Now, suppose the app also allows a user to
change the books in the group (ie, via some other url).  I think this
would make it impossible to use the indexing method because the
formset that I create during POST might not be the same as the formset
I create during the GET.  IE, I'd run into this case if one user goes
to the form where they fill out authors and then sits on that form for
awhile, and meanwhile some other user goes and changes the books in
the group.  For error checking purposes, I think I need to send the
book titles as hidden inputs with the form, that when I proceses the
POST, if some of those titles are now gone, I can identify which
titles are still valid and also generate good error messags for the
ones that are not.


This example is sort of contrived, but in the system I'm designing, it
could happen and I'd like to be able to cover it gracefully.   The
code I created to check for this error is pretty ugly, in that it
reaches right into the django source private space.  This assumes that
fieldName is some "hidden" field that lets me identify each form.  The
code below compares the post data to the initial data, in a similar to
manner to the get_changed_data() function in the source.  Here it is:

def getFormContinuityError(form, fieldName):
"""Confirm that the 'initial' value for fieldName is the same as
the 'data' value for fieldName.
This protects from case where the fieldNames have changed between
the get and post.
"""
data_value = form.fields[fieldName].widget.value_from_datadict
(form.data, form.files, form.add_prefix(fieldName))
initial_value = form.initial.get(fieldName, form.fields
[fieldName].initial)
has_changed = form.fields[fieldName].widget._has_changed
(initial_value, data_value)
return has_changed

def getFormSetContinuityError(formSet, fieldName):
continuityError = 0
for form in formSet.forms:
if getFormContinuityError(form, fieldName):
continuityError = 1
break
return continuityError

I will probably no doubt regret this code some day when I update to a
new version of django!

A second thing that I needed to do that is perhaps different from what
other people have needed was to save the formset forms even if they
are empty.  In my particular app, the user is creating tasks.  They
put in a description of the task and each task has 1 or more results,
one per "tile".  Associated with each tile is an optional set of
"special 

Re: formset initialization question

[Malcolm Tredinnick]

On Wed, 2009-03-04 at 22:17 -0800, Margie wrote:
> Here's a "book" analogy to what I am doing.  I am sending out a bunch
> of forms to the user.  Each form contains the title of a book (my
> "tile") and an input field where the user is being asked to input the
> author of the book.  When the view.py code receives the post data, for
> each form where the user has input the author, the view.py code will
> create a book object with the title from the form and the author that
> the user input.
> 
> So when I get an author, I need to identify which title it goes with.
> When I was writing the code I felt uncertain about identifying which
> title the form was for based on index. I thought that would work, but
> it seemed somehow safer to simply have it encoded in the same form.
> That way I don't have to match up indexes - instead I just ask the
> form what the title is. So that's why I sent the title as a hidden
> input.

Okay, that makes sense. So it's a different hidden value for each form
then. That was something I hadn't been sure about in these threads: at
one point it looked like it was always the same value, at other times,
it was a different value.

> 
> The reason that has_changed() was false was simply that in my
> debugging, on the client side I was not filling out the form.  I was
> just leaving it blank.  Then in view.py I was looking at the form to
> verify that I could get back the book title, and that's when I
> encountered the fact that cleaned_data was empty.

Okay. So I think you now understand why that's happening and that it's
not a bug.

I'd also guess it's not really a problem, at least in the book analogy
you give. If they don't provide an author for the book, you just don't
do anything with that title (the form won't appear to have been filled
in). That seems like fairly reasonable behaviour: only create an object
when the user supplies some data.


> 
> What can I say ... I've been trying a lot of different things to
> really get a handle on this and it's taken me down a lot of strange
> paths.

Yeah, that's understandable. Sometimes running around a problem a lot
doesn't help with keeping a perspective on which approaches are working.
That's why I'm happy to keep going with threads like this and asking
different questions, since reorienting oneself from time to time
sometimes takes an extra set of eyes.

Regards,
Malcolm



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: formset initialization question

[Margie]

Here's a "book" analogy to what I am doing.  I am sending out a bunch
of forms to the user.  Each form contains the title of a book (my
"tile") and an input field where the user is being asked to input the
author of the book.  When the view.py code receives the post data, for
each form where the user has input the author, the view.py code will
create a book object with the title from the form and the author that
the user input.

So when I get an author, I need to identify which title it goes with.
When I was writing the code I felt uncertain about identifying which
title the form was for based on index. I thought that would work, but
it seemed somehow safer to simply have it encoded in the same form.
That way I don't have to match up indexes - instead I just ask the
form what the title is. So that's why I sent the title as a hidden
input.

The reason that has_changed() was false was simply that in my
debugging, on the client side I was not filling out the form.  I was
just leaving it blank.  Then in view.py I was looking at the form to
verify that I could get back the book title, and that's when I
encountered the fact that cleaned_data was empty.

What can I say ... I've been trying a lot of different things to
really get a handle on this and it's taken me down a lot of strange
paths.

I'm sure that you are right that I can identify the "book" based on
its index in the formset.

Once again, thanks for your insights.

Margie

On Mar 4, 8:42 pm, Malcolm Tredinnick 
wrote:
> On Wed, 2009-03-04 at 00:28 -0800,Margiewrote:
>
> [...]
>
> > After debugging into the source some more, I'm finding that my the
> > problem is related to the fact that empty_permitted is getting set to
> > True when all of my forms are considered "extra.
>
> Which is reasonable. "Extra" here means "optional".
>
> > Specifically, in site-
> > packages/django/forms/forms.py, in the full_clean() function, I find
> > that when empty_permitted is set to True, this code executes, which
> > drops me out of the code without setting cleaned_data at all:
>
> >         # If the form is permitted to be empty, and none of the form
> > data has
> >         # changed from the initial data, short circuit any validation.
> >         if self.empty_permitted and not self.has_changed():
> >             return
>
> This will only happen if none of the data has changed. That's the point
> of the call to has_changed().
>
>
>
> > It seems to me that if I pass in initial when I create the
> > TaskTileDetailFormSet, ie, like this:
>
> >     initial = []
> >     for tile in tileList:
> >         initial.append({'tile': tile.id})
>
> >     taskTileDetailFormSet = TaskTileDetailFormSet(data,
> > tiles=tileList, initial=initial)
>
> > then empty_permitted gets set to False ibecause my initial_form_count
> > is non-zero.  IE, in site-packages/django/forms/formsets.py, in
> > _construct_form(), in thh code that looks like this:
>
> >         # Allow extra forms to be empty.
> >         if i >= self._initial_form_count:
> >             defaults['empty_permitted'] = True
>
> > But if I pass initial in via my own _construct_form() function as you
> > suggested, then I have no initial data, so all of my forms are
> > "extra".  IN this case self._initial_form_count is 0, and it seems
> > that the result is that cleaned_data doesn't get set correctly.
>
> That would only be the case if none of the data for the form has
> changed. If you've changed something on the form (from the initial
> data's perspective), has_changed() should be returning True.
>
>
>
> > I am probably far from undrestanding this, but if what I said is
> > atually true, it seems like this is actually a bug?  The bug being
> > that cleaned_data is not getting set correctly when the forms are
> > created as "extra" forms.  Perhaps cleaned_data is not supposed to get
> > set in this case?
>
> Formsets are not entirely broken. Let's not think of zebras in
> preference to horses when we hear the hoof-beats. If formsets and empty
> forms didn't work, the entire admin application would be broken in
> interesting ways, for example.
>
> You've found the right pieces of code, but you aren't examining why they
> are falling through to skipping validation. The conditional test is
> important. Why isn't has_changed() returning True for your particular
> setup? Do you have to also subclass that based on the data changes
> you're making, perhaps?
>
> >   The whole reason that I happened upon this is
> > beacuse I am trying to identify which form is which, so I was looking
> > at cleaned_data['tile'].  I set that myself anyway, so I can just look
> > at data['tile'].
>
> This is something I don't understand. If you are setting up the "tile"
> each time in your view, why is it having to be sent to the form? Either
> it's data you take from the form, or it's data you can always supply in
> the view. I've decided I don't understand the problem you're trying to
> solve here, so can you please 

Re: formset initialization question

[Malcolm Tredinnick]

On Wed, 2009-03-04 at 00:28 -0800, Margie wrote:
[...]
> After debugging into the source some more, I'm finding that my the
> problem is related to the fact that empty_permitted is getting set to
> True when all of my forms are considered "extra. 

Which is reasonable. "Extra" here means "optional".

> Specifically, in site-
> packages/django/forms/forms.py, in the full_clean() function, I find
> that when empty_permitted is set to True, this code executes, which
> drops me out of the code without setting cleaned_data at all:
> 
> # If the form is permitted to be empty, and none of the form
> data has
> # changed from the initial data, short circuit any validation.
> if self.empty_permitted and not self.has_changed():
> return

This will only happen if none of the data has changed. That's the point
of the call to has_changed().

> It seems to me that if I pass in initial when I create the
> TaskTileDetailFormSet, ie, like this:
> 
> initial = []
> for tile in tileList:
> initial.append({'tile': tile.id})
> 
> taskTileDetailFormSet = TaskTileDetailFormSet(data,
> tiles=tileList, initial=initial)
> 
> then empty_permitted gets set to False ibecause my initial_form_count
> is non-zero.  IE, in site-packages/django/forms/formsets.py, in
> _construct_form(), in thh code that looks like this:
> 
> # Allow extra forms to be empty.
> if i >= self._initial_form_count:
> defaults['empty_permitted'] = True
> 
> But if I pass initial in via my own _construct_form() function as you
> suggested, then I have no initial data, so all of my forms are
> "extra".  IN this case self._initial_form_count is 0, and it seems
> that the result is that cleaned_data doesn't get set correctly.

That would only be the case if none of the data for the form has
changed. If you've changed something on the form (from the initial
data's perspective), has_changed() should be returning True.
> 
> I am probably far from undrestanding this, but if what I said is
> atually true, it seems like this is actually a bug?  The bug being
> that cleaned_data is not getting set correctly when the forms are
> created as "extra" forms.  Perhaps cleaned_data is not supposed to get
> set in this case?

Formsets are not entirely broken. Let's not think of zebras in
preference to horses when we hear the hoof-beats. If formsets and empty
forms didn't work, the entire admin application would be broken in
interesting ways, for example.

You've found the right pieces of code, but you aren't examining why they
are falling through to skipping validation. The conditional test is
important. Why isn't has_changed() returning True for your particular
setup? Do you have to also subclass that based on the data changes
you're making, perhaps?

>   The whole reason that I happened upon this is
> beacuse I am trying to identify which form is which, so I was looking
> at cleaned_data['tile'].  I set that myself anyway, so I can just look
> at data['tile'].

This is something I don't understand. If you are setting up the "tile"
each time in your view, why is it having to be sent to the form? Either
it's data you take from the form, or it's data you can always supply in
the view. I've decided I don't understand the problem you're trying to
solve here, so can you please (re-)explain that? I've gone back over the
thread and I can't see a clear statement of the need for all this
customisation, so thinking of alternative approaches is hard.

Once you have somehow identified the formset, identifying each form
inside it doesn't seem to be necessary, particularly if they're all new
objects (after all, each form in the formset is numbered, so unique
identifier + offset in formset is a unique way of identifying each
component form). Why not just have a single identifier for the entire
formset?

If you can do that, then the problem falls out easily: the identifier
doesn't go in the formset at all. It goes into a separate form class
that only contains this hidden field. Remember that Django form classes
correspond to a fragment of an HTML form, so you can pass multiple form
instance (or a form instance plus a formset instance) to the template.

Alternatively, you could override Formset.__init__ and put the
identifier into the ManagementForm instance. That doesn't feel quite as
nice to me, since it's kind of relying on a bunch of implementation
details (I'd almost like ManagementForm to be overridable, probably as a
class attribute).

Regards,
Malcolm


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: formset initialization question

[Margie]

Yes, I am defining _construct_form() as you do.  I had just not
thought to set initial there - but that makes sense.  For reference,
here is my current code, with it setting initial in _construct_form.
But you may want to read on below before bothering to look at it
because I have a theory about why cleaned_data is not getting set.

class TaskTileDetailCustomBaseFormSetForCreate(BaseFormSet):
def __init__(self, *args, **kwargs):
self.tiles = list(kwargs.pop("tiles"))
self.extra = len(self.tiles)
super(TaskTileDetailCustomBaseFormSetForCreate, self).__init__
(*args, **kwargs)

def _construct_form(self, i, **kwargs):
kwargs["tile"] = self.tiles[i]
kwargs["initial"] = {'tile':self.tiles[i].id}
return super(TaskTileDetailCustomBaseFormSetForCreate,
self)._construct_form(i, **kwargs)

def makeTaskTileDetailFormSetForCreate(tileList, data=None):
TaskTileDetailFormSet = formset_factory(form=TaskTileDetailForm,
formset=TaskTileDetailCustomBaseFormSetForCreate)
taskTileDetailFormSet = TaskTileDetailFormSet(data,
tiles=tileList)
return taskTileDetailFormSet



After debugging into the source some more, I'm finding that my the
problem is related to the fact that empty_permitted is getting set to
True when all of my forms are considered "extra. Specifically, in site-
packages/django/forms/forms.py, in the full_clean() function, I find
that when empty_permitted is set to True, this code executes, which
drops me out of the code without setting cleaned_data at all:

# If the form is permitted to be empty, and none of the form
data has
# changed from the initial data, short circuit any validation.
if self.empty_permitted and not self.has_changed():
return



It seems to me that if I pass in initial when I create the
TaskTileDetailFormSet, ie, like this:

initial = []
for tile in tileList:
initial.append({'tile': tile.id})

taskTileDetailFormSet = TaskTileDetailFormSet(data,
tiles=tileList, initial=initial)

then empty_permitted gets set to False ibecause my initial_form_count
is non-zero.  IE, in site-packages/django/forms/formsets.py, in
_construct_form(), in thh code that looks like this:

# Allow extra forms to be empty.
if i >= self._initial_form_count:
defaults['empty_permitted'] = True

But if I pass initial in via my own _construct_form() function as you
suggested, then I have no initial data, so all of my forms are
"extra".  IN this case self._initial_form_count is 0, and it seems
that the result is that cleaned_data doesn't get set correctly.

I am probably far from undrestanding this, but if what I said is
atually true, it seems like this is actually a bug?  The bug being
that cleaned_data is not getting set correctly when the forms are
created as "extra" forms.  Perhaps cleaned_data is not supposed to get
set in this case?  The whole reason that I happened upon this is
beacuse I am trying to identify which form is which, so I was looking
at cleaned_data['tile'].  I set that myself anyway, so I can just look
at data['tile'].  However, it seems that none of my post data is
getting put in cleaned_data, so that still seems like a general
problem.

Margie






On Mar 3, 11:17 pm, Malcolm Tredinnick 
wrote:
> On Tue, 2009-03-03 at 22:42 -0800, Margie wrote:
> > Hi Malcolm - Sorry, another formset question.  Maybe you'll know the
> > answer offhand, have been trying to figure this out for awhile now.
> > I'm trying to set the initial value for a form field inside the
> > constructor for the form.  The high level goal is to send out the
> > 'tile' field as a hidden input so that I can get it back in the POST
> > so that I can figure out wich 'tile' each form in the forset
> > corresonds to.
>
> > For example, I'm trying to do this in my form constructor
> > (specifically see the ==> line)
>
> > class TaskTileDetailForm(forms.ModelForm):
> >     class Meta:
> >         model=TaskTileDetail
> >         exclude=('task')
>
> If this is a cut-and-paste, this line is almost certainly a bug. The
> "exclude" parameter should be a tuple or a list, so ("task",) or
> ["task"] (note the trailing comma in the tuple case).
>
>
>
> >     def __init__(self, tile, *args, **kwargs):
>
> This line is a probably going to be a problem at some point. You cannot
> just add new positional arguments into the front of the __init__ call
> like this. When Django calls the constructor, it could well be passing
> the "data" parameter as the first argument.
>
> Instead, pass it in as a keyword argument. To wit:
>
>         def __init__(self, *args, **kwargs):
>            tile = kwargs.pop("tile")
>            super(TaskTileDetailForm, self).__init__(*args, **kwargs)
>            ...
>
> That preserves the ordering of positional arguments.
>
> >         super(TaskTileDetailForm, self).__init__(*args, **kwargs)
> >         self.fields['tile'].widget = forms.HiddenInput()
> 

Re: formset initialization question

[Malcolm Tredinnick]

On Tue, 2009-03-03 at 22:42 -0800, Margie wrote:
> Hi Malcolm - Sorry, another formset question.  Maybe you'll know the
> answer offhand, have been trying to figure this out for awhile now.
> I'm trying to set the initial value for a form field inside the
> constructor for the form.  The high level goal is to send out the
> 'tile' field as a hidden input so that I can get it back in the POST
> so that I can figure out wich 'tile' each form in the forset
> corresonds to.
> 
> For example, I'm trying to do this in my form constructor
> (specifically see the ==> line)
> 
> class TaskTileDetailForm(forms.ModelForm):
> class Meta:
> model=TaskTileDetail
> exclude=('task')

If this is a cut-and-paste, this line is almost certainly a bug. The
"exclude" parameter should be a tuple or a list, so ("task",) or
["task"] (note the trailing comma in the tuple case).

> 
> def __init__(self, tile, *args, **kwargs):

This line is a probably going to be a problem at some point. You cannot
just add new positional arguments into the front of the __init__ call
like this. When Django calls the constructor, it could well be passing
the "data" parameter as the first argument.

Instead, pass it in as a keyword argument. To wit:

def __init__(self, *args, **kwargs):
   tile = kwargs.pop("tile")
   super(TaskTileDetailForm, self).__init__(*args, **kwargs)
   ...

That preserves the ordering of positional arguments.

> super(TaskTileDetailForm, self).__init__(*args, **kwargs)
> self.fields['tile'].widget = forms.HiddenInput()
> ==> self.fields['tile'].initial = tile.id
> 
> If I do this, then later when processing my POST data there is no
> 'tile' key in cleaned_data.  IE, I get a KeyError when I do this:
> if taskTileDetailForm.cleaned_data["tile"] in taskTiles:
> 
> If I intiaizize it with the initial arg when creating the formset, the
> cleaned_data["tile"] key is set just fine, ie when I use initial as
> shown at the ==> below:

It seems like the problem is how is "tile" meant to be known to each
form? When a formset is constructed, it just constructs and essentially
normal form, passing in any "data" and "initial" parameters. If you need
to do any extra setup on the form -- particularly dealing with passing
in extra parameters -- then you have to do something similar to what I
did in my blog post ([1], for those who don't know what we're talking
about).

So, either, pass in "initial" to the formset, or override the
_construct_form() method, as in the blog post. That's precisely why I
had to do it that way in my example: I needed to pass in some custom
information to each form as it was constructed as part of the formset.

The difference between your case and mine, is that your extra data
corresponds directly to a form field (in mine, the question text was
auxiliary data that wasn't a form field). So I would probably set it up
using the "initial" parameter, if it was me. It would involve far less
formset base class customisation.

[1]
http://www.pointy-stick.com/blog/2009/01/23/advanced-formset-usage-django/


> def makeTaskTileDetailFormSetForCreate(tileList, data=None):
> TaskTileDetailFormSet = formset_factory
> (form=TaskTileDetailForm, formset=CustomBaseFormSe)
> 
> initial = []
> for tile in tileList:
> initial.append({'tile': tile.id})

Since this is a read-only attribute, you could probably just write

initial = [{"tile": tile.id}] * len(tileList)

That will make each element of initial a reference to the same
dictionary, but that's not a problem since the initial data is only
read, never changed.

Regards,
Malcolm


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---