[web2py] From csv to web2py-model

[Johann Spies]

I have to build new models from csv-data files regularly and have written a
small, primitive script to assist me in this.

I have attached it.  Maybe someone finds it useful.

Usage: python model_from_csv.py <somefile.csv>

It will print out a dictionary with a as keys the fields and values the
maximum length of that field and after that the proposed web2py model.
The field types will be either default (string) or text if the length of
the field > 512.

Regards
Johann

-- 
Because experiencing your loyal love is better than life itself,
my lips will praise you.  (Psalm 63:3)

#!/usr/bin/python
# -*- coding: latin-1 -*-

import sys, csv, sys, re
x = re.compile('["|\n\r]')
v = re.compile('[/ @*.-]')


leernaam = sys.argv[1]
f = open(leernaam, 'r')
eerstereel = [re.sub(x, '', y)for y in f.readline().split(',')]
f.close

reader = csv.DictReader(open(leernaam), delimiter = ',')
csv.field_size_limit(1000000)

leernaam = re.sub('__', '_', leernaam.lower())
leernaam = re.sub('-', '_', leernaam)
c = re.compile('.csv')
leernaam = re.sub(c, '', leernaam)

maksimum = {}

for ry in reader:
    for k in ry.keys():
        l = len(ry[k])
        if k in maksimum.keys():
            if l > maksimum[k]:
                maksimum[k] = l
        else:
            maksimum[k] = l

print maksimum
print
l = len(maksimum)
besigmet = 0
spasies = "    "
print ('db.define_table("%s",' % (leernaam))
for i in eerstereel:
    it = re.sub(v, '_', i)
    item = re.sub('__', '_', it).lower()
    besigmet = besigmet + 1
    if (maksimum[i] > 1024):
        print ('%sField("%s","text")' % (spasies, item)),
    elif (maksimum[i] > 512):
        print ('%sField("%s","string", length=1024")' % (spasies, item)),
    else:
        print ('%sField("%s")' % (spasies, item)),
    if besigmet == l:
        print(')')
    else:
        print (',')