Thanks for checking it out. In the mean time, I'm working around the
problem by overriding maxcharlength right after setting db, like this:
db = DAL(...)
db._adapter.maxcharlength = 65535
Now I just need to restore a bunch of truncated data :-(
On Jan 22, 1:06 pm, Massimo Di Pierro <massimo.dipie...@gmail.com>
wrote:
> I agree that is the problem:
>
> Field("description", "string", length=2048),
>
> according to:http://dev.mysql.com/doc/refman/5.0/en/char.html
> "Values in VARCHAR columns are variable-length strings. The length can
> be specified as a value from 0 to 255 before MySQL 5.0.3, and 0 to
> 65,535 in 5.0.3 and later versions".
>
> We set varchar max to 255 to avoid problems with older versions of
> mysql. Since problems were reported by users.
> There should not have been a migration in your case. I will look in
> this more.
>
> On Jan 22, 2:43 pm, "Jonathan Z." <jzem...@gmail.com> wrote:
>
>
>
> > It's probably also important to note that the "before" table existed
> > since 1.85.x. I keep the "migrate=False" flag set on it, until I make
> > a change. As I noted previously, the change happened on 1.91.6. So
> > the issue is data loss affected by changes in the web2py DAL.
>
> > I've been digging around, and my guess is that this line (3694) in
> > dal.Table is the problem:
>
> > field.length = min(field.length,self._db and
> > self._db._adapter.maxcharlength or INFINITY)
>
> > Since maxcharlength for MySQLAdapter is 255, this has the affect of
> > silently truncating my specified field lengths.
>
> > On Jan 22, 11:48 am, "Jonathan Z." <jzem...@gmail.com> wrote:
>
> > > Massimo,
>
> > > Here is an example of one of the tables that got a truncated field:
>
> > > before...
>
> > > db.define_table("thing",
> > > Field("user_id", db.auth_user, notnull=True, readable=False,
> > > writable=False),
> > > Field("title", "string", length=128, notnull=True, unique=True,
> > > required=True),
> > > Field("slug", "string", length=128, notnull=True, unique=True,
> > > required=True),
> > > Field("state", "string", length=10, notnull=True, required=True,
> > > default="created", requires=IS_IN_SET(("created", "started",
> > > "completed"))),
> > > Field("description", "string", length=2048),
> > > Field("created", "datetime", notnull=True, readable=False,
> > > writable=False, default=request.now),
> > > Field("updated", "datetime", readable=False, writable=False,
> > > update=request.now),
> > > Field("uuid", "string", length=16, notnull=True, unique=True,
> > > readable=False, writable=False),
> > > format="%(title)s"
> > > )
>
> > > after...
>
> > > db.define_table("thing",
> > > Field("user_id", db.auth_user, notnull=True, readable=False,
> > > writable=False),
> > > Field("title", "string", length=128, notnull=True, unique=True,
> > > required=True),
> > > Field("slug", "string", length=128, notnull=True, unique=True,
> > > required=True),
> > > Field("state", "string", length=10, notnull=True, required=True,
> > > default="created", requires=IS_IN_SET(("created", "started",
> > > "completed"))),
> > > Field("description", "string", length=2048),
> > > Field("things", "list:string"), # NEW FIELD
> > > Field("created", "datetime", notnull=True, readable=False,
> > > writable=False, default=request.now),
> > > Field("updated", "datetime", readable=False, writable=False,
> > > update=request.now),
> > > Field("uuid", "string", length=16, notnull=True, unique=True,
> > > readable=False, writable=False),
> > > format="%(title)s"
> > > )
>
> > > After this change (addition of a new field) and subsequent DB
> > > migration, the "description" field was truncated from 2048 characters
> > > to 255. Note that this change didn't alter the field length on
> > > sqlite. But it did truncate the description field on mysql.
>
> > > On Jan 22, 10:55 am, Massimo Di Pierro <massimo.dipie...@gmail.com>
> > > wrote:
>
> > > > Can you tell us what the model was before and after migration?
>
> > > > Massimo
>
> > > > On Jan 22, 10:02 am, "Jonathan Z." <jzem...@gmail.com> wrote:
>
> > > > > I have a number of fields specifying a length > 255 (some as large as
> > > > > 2K). On a recent schema migrate, web2py truncated all of these fields
> > > > > to 255, resulting in substantial data loss.
>
> > > > > My database is mysql. I'm on version 1.91.6 of web2py. I'm pretty
> > > > > sure this has something to do with the new DAL.
