Dear Steve, Florian Jaeger and I had an off-list conversation about this. Since the random slope of group by items is estimating a distribution over items, some of the items can be omitted and the model will still fit. In fact, it actually has a reasonable good estimate for the item-specific effects of group. Please see attached code for results.
In these data there are some convergence issues, but these seem to be highly dependent on the particular items excluded (and the particular distribution of participants across groups). This gets quite a bit worse as the number of overlapping items gets smaller and smaller. A more systematic study with simulated data is definitely in order, but I haven’t done that. An alternative approach which won’t suffer from the same convergence issues is to use a Bayesian model. That might be the right way to go in this case. Thanks, Florian, for discussion of this! Matt On Wed, Apr 24, 2019 at 7:50 AM Matt Goldrick < matt-goldr...@northwestern.edu> wrote: > Dear Steve, > > I'm not sure if anyone responded to this previously (apologies for > duplication if so). > > - You can certainly build the model with a random intercept for items > and a fixed effect for group. > - You cannot have a random slope for group, as not all items have > observations at both levels of group. > - This may be anticonservative, as you're failing to take in to > account the dependencies between observations of items across different > levels of group. > > The various alternatives (analyzing only overlapping items, only distinct > items, or all items) all seem to have drawbacks, so I'm not entirely sure > what to recommend. One way to proceed would be to conduct all of the > analyses and show how the estimate for the group effect varies as a > function of which set of items is included. If it's similar across > analyses, that might increase your confidence in the effect. > > mG > > On Fri, Mar 29, 2019 at 7:55 PM Steve Jones <midtown...@gmail.com> wrote: > >> Hello, >> >> Is it possible to compare group A and B responses in a mixed-effects >> linear model if the items that each group saw were not 100% the same? Each >> group saw mostly overlapping items (~200 items) but some distinct items >> (~100 items). Since I am interested in the group difference, can I treat >> items as a random effect in such a model? Is there any concern that the >> group differences may be driven by either the overlapping or the distinct >> items only? >> >> SJ >> >>
nonOverlapping.R
Description: Binary data
