Hi Delzad,
Column order is not preserved in the App Engine datastore. You can access
columns in whatever fashion you like, though.
-Nick Johnson
On Fri, Apr 16, 2010 at 5:58 PM, Delzad <delzad.da...@gmail.com> wrote:
> Hi Nick
>
> yes..the problem is resolved...
>
> I used the --has_header option with appcfg command
>
> thats done for now..
>
> by the way i have another doubt...
>
> does the data get uploaded and stored in the same manner as it is in
> the csv file and in the config file?
>
> the reason i ask this, is because my config file lists the Data in the
> same order as it is in the csv file.
>
> but the Data when viewed in the App engine data store..does not follow
> that order..
>
> is there anything I can do to correct that?
>
> Thanks anyways...for your comments
>
> Del
>
>
>
> On Apr 16, 1:54 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> wrote:
> > Hi Delzad,
> >
> > As I described, the issue here is with date parsing, not with the
> > bulkloader. You need to try the expression with a sample date in the
> > interactive Python console, and tweak it until you determine what's wrong
> > with the dates you're trying to format.
> >
> > -Nick
> >
> >
> >
> > On Thu, Apr 15, 2010 at 11:38 PM, Delzad <delzad.da...@gmail.com> wrote:
> > > Hi there!
> >
> > > Mate I have tried every thing!
> >
> > > 1. I even removed the header line of the csv file, kept just the
> > > values.
> > > when I did that...the error now is that the tuple object is not
> > > callable
> >
> > > 2. Secondly, i even used google docs, same error.
> >
> > > here is the one liner.
> >
> > > **loaders.py**
> > > ('author_birthdate', lambda x: datetime.datetime.strptime(x, '%d/%m/
> > > %Y').date())
> >
> > > **models.py**
> > > 1. ) author_birthdate = db.StringProperty()
> > > or
> > > 2. ) author_birthdate = db.DateProperty()
> >
> > > i even tried
> > > author_birthdate = db.DateTimeProperty()
> >
> > > **command used to upload data**
> > > appcfg.py upload_data --config_file=loaders.py --filename=del1.csv --
> > > kind=new_delzad ray-tech
> >
> > > I'm going exactly as per the tutorial, but the problem lies in the
> > > date format...which is where i am confused.
> >
> > > Note: this same command worked brilliant when I passed the date as a
> > > string.
> >
> > > look forward to your views on this one!
> >
> > > regards
> >
> > > Del
> >
> > > On Apr 15, 4:52 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> > > wrote:
> > > > Hi Delzad,
> >
> > > > Can you provide a oneliner (datetime.datetime.striptime(...)) that
> > > > reproduces this behaviour? The problem is with date formatting, so it
> > > will
> > > > be useful to remove the extraneous issues of the bulkloader from the
> > > > equation.
> >
> > > > -Nick Johnson
> >
> > > > On Thu, Apr 15, 2010 at 1:46 AM, Delzad <delzad.da...@gmail.com>
> wrote:
> > > > > Hi Nick!
> >
> > > > > thanks for the post dear,
> >
> > > > > yes I did try to send in an example date..and it gave back that
> same
> > > > > string and said that the format did not match, and it inserted an
> > > > > extra set of Inverted quotes..
> >
> > > > > I have a feeling it's something to do with my CSV file...
> >
> > > > > here is the code..
> >
> > > > > ***loaders.py******
> >
> > > > > import datetime
> > > > > from google.appengine.ext import db
> > > > > from google.appengine.tools import bulkloader
> > > > > import models
> >
> > > > > class BookLoader(bulkloader.Loader):
> > > > > def __init__(self):
> > > > > bulkloader.Loader.__init__(self, 'Book',
> > > > > [('title', str),
> > > > > ('author', str),
> > > > > ('copyright_year', str),
> > > > > ('author_birthdate', lambda x:
> > > > > datetime.datetime.strptime(x, '%m/%d/%Y').date()),
> > > > > ])
> >
> > > > > loaders = [BookLoader]
> >
> > > > > *****models.py*******
> > > > > from google.appengine.ext import db
> >
> > > > > class Book(db.Model):
> > > > > title = db.StringProperty()
> > > > > author = db.StringProperty()
> > > > > copyright_year = db.StringProperty()
> > > > > author_birthdate = db.DateProperty()
> >
> > > > > ******new_delzad.csv *******
> >
> > > > > title author copyright_year author_birthdate
> > > > > app_en Delzad 2010 12/25/2012
> >
> > > > > ****** ERROR MESSAGE******
> > > > > File "/usr/lib/python2.6/_strptime.py", line 325, in _strptime
> > > > > (data_string, format))
> > > > > ValueError: time data "'author_birthdate'" does not match format
> '%m/
> > > > > %d/%Y'
> > > > > [INFO ] Backing off due to errors: 1.0 seconds
> > > > > [INFO ] An error occurred. Shutting down...
> > > > > [ERROR ] Error in WorkerThread-0: time data "'author_birthdate'"
> > > > > does not match format '%m/%d/%Y'
> >
> > > > > these inverted commas, which i have entered in the loaders.py is
> the
> > > > > format which I am specifying right?
> >
> > > > > the csv file in Excel spreadsheet does not contain any inverted
> > > > > commas...
> >
> > > > > so this kind of leaves me a bit baffled..where could I be going
> wrong?
> >
> > > > > can i just remove the strptime thing and add just a simple date
> > > > > format?
> >
> > > > > thanks dear
> >
> > > > > regards
> >
> > > > > Delzad
> >
> > > > > On Apr 14, 3:09 pm, "Nick Johnson (Google)" <
> nick.john...@google.com>
> > > > > wrote:
> > > > > > Hi Delzad,
> >
> > > > > > Can you provide sample data? Have you tried passing an example
> date
> > > > > directly
> > > > > > to datetime.datetime.strptime with that format string, to see if
> it
> > > > > works?
> >
> > > > > > -Nick Johnson
> >
> > > > > > On Wed, Apr 14, 2010 at 1:00 AM, Delzad <delzad.da...@gmail.com>
> > > wrote:
> > > > > > > Hi there!
> >
> > > > > > > Bit confused about the data stores ability to take in data.
> >
> > > > > > > i'm trying to upload a data value in the format, %m/%d/%Y, but
> i
> > > keep
> > > > > > > getting the error message: ValueError: time data 'Date' does
> not
> > > match
> > > > > > > format '%m/%d/%Y'.
> >
> > > > > > > My CSV file contains data in this exact same format.
> >
> > > > > > > here is the main bit of my loader file.
> >
> > > > > > > class loaders(bulkloader.Loader):
> > > > > > > def __init__(self):
> > > > > > > bulkloader.Loader.__init__(self, 'New_aviva',
> > > > > > > [('Date',lambda x:
> > > > > > > datetime.datetime.strptime(x, '%m/%d/%Y').date()),
> > > > > > > ('Open', int),
> > > > > > > ('high', int),
> > > > > > > ('low', int),
> > > > > > > ('close', int),
> > > > > > > ('volume', int),
> > > > > > > ('adjclose', int)
> > > > > > > ])
> >
> > > > > > > Although I have managed to fix this error...if I take in the
> date
> > > > > > > property as a STRING, it works just fine, but then I have
> issues
> > > with
> > > > > > > the GQL in the data store, as I would not be able to access
> them in
> > > > > > > any order.
> >
> > > > > > > So i need to take in date as a DATE value.
> >
> > > > > > > Guidance appriciated.
> >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to the
> Google
> > > > > Groups
> > > > > > > "Google App Engine" group.
> > > > > > > To post to this group, send email to
> > > google-appengine@googlegroups.com
> > > > > .
> > > > > > > To unsubscribe from this group, send email to
> > > > > > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> >
> > > <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
> >
> >
> > > > > <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
> >
> > > <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
> <google-appengine%25252bunsubscr...@googlegroups.com<google-appengine%2525252bunsubscr...@googlegroups.com>
> >
> >
> > > > > > > .
> > > > > > > For more options, visit this group at
> > > > > > >http://groups.google.com/group/google-appengine?hl=en.
> >
> > > > > > --
> > > > > > Nick Johnson, Developer Programs Engineer, App Engine Google
> Ireland
> > > Ltd.
> > > > > ::
> > > > > > Registered in Dublin, Ireland, Registration Number: 368047
> > > > > > Google Ireland Ltd. :: Registered in Dublin, Ireland,
> Registration
> > > > > Number:
> > > > > > 368047
> >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "Google App Engine" group.
> > > > > To post to this group, send email to
> google-appengine@googlegroups.com
> > > .
> > > > > To unsubscribe from this group, send email to
> > > > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> >
> > > <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
> >
> >
> > > > > .
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/google-appengine?hl=en.
> >
> > > > --
> > > > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland
> Ltd.
> > > ::
> > > > Registered in Dublin, Ireland, Registration Number: 368047
> > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
> > > Number:
> > > > 368047
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appengine@googlegroups.com
> .
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> >
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
> >
> > --
> > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd.
> ::
> > Registered in Dublin, Ireland, Registration Number: 368047
> > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
> Number:
> > 368047
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> .
> > For more options, visit this group athttp://
> groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. ::
Registered in Dublin, Ireland, Registration Number: 368047
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
