Yeah, those are diagnostic logs. They effectively go into a ring buffer per
logging level, so the maximum data stored at any given time is capped. The
more you log, the more frequently you would have to download the logs to
avoid missing any. You would also be competing with log space with any
other log messages generated by your application.
What I would suggest instead is either to increment counters in memcache,
and flush them to the datastore periodically if you need durability (I'm
assuming you can tolerate some chance of data loss here). If you cannot
tolerate any loss of data, then I would suggest enqueueing tasks to a task
queue for each request that maintains a summary in memcache and/or the
datastore.
On Wed, Mar 10, 2010 at 4:59 PM, Spines <kwste...@gmail.com> wrote:
> I'm talking about the logs that get written when I call
> Logger.info("something").
>
> Basically this is what I'm thinking: I have certain data that needs to
> get written very often, but hardly ever needs to be read (stuff like
> what users view what pages of my site). The datastore is optimized
> for read efficiency. So, I want to output this data to the logs. I
> will have an offsite computer download these logs, do calculations on
> them, and upload the result of the calculations to the datastore.
>
> On Mar 10, 2:03 pm, Don Schwarz <schwa...@google.com> wrote:
> > Are you talking about request logs or diagnostic logs? Although we
> conflate
> > them a bit in both the Admin Console viewer and the appcfg command, but
> they
> > are stored and tracked separately.
> >
> >
> >
> > On Wed, Mar 10, 2010 at 4:01 PM, Spines <kwste...@gmail.com> wrote:
> > > Hmm, that is my biggest concern, log reliability. Can someone from
> > > Google confirm whether or not I can rely on the logs having all of the
> > > log data? Or might certain entries just disappear?
> >
> > > On Mar 10, 1:24 pm, thierry Le conniat <thlec...@euriware.fr> wrote:
> > > > Hello,
> > > > I think google log are stored in file.
> > > > My experience about log reliability is that when the app is very
> > > > strong working, not all the log are stored.
> > > > It's confusing, but i can't not explain it.
> >
> > > > Bye
> >
> > > > On 10 mar, 22:04, Spines <kwste...@gmail.com> wrote:
> >
> > > > > Where does Google store the logs when you do a Logging statement?
> > > > > Logging statements seem to be pretty fast, so it doesn't seem like
> > > > > they are stored in the datastore.
> >
> > > > > How reliable are the logs? If I do a logging statement and it
> > > > > succeeds, is it pretty much guaranteed that it will show up in the
> > > > > logs?
> >
> > > > > How much past history of logs is stored?
> >
> > > > > The reason I'm interested in this is because I'm making a question
> and
> > > > > answer website, and I want to keep track of views by each unique
> > > > > logged in user to each question, and display the view count on the
> > > > > question page. So if 10 different users visit the question page 100
> > > > > times, it still only counts as 10 unique views.
> >
> > > > > I have an offsite computer that does background processing for my
> app.
> > > > > I'm planning to have this offsite computer download the logs about
> > > > > every 30 minutes, and calculate what the view count should be for
> each
> > > > > question based off of the logs. By doing this, I don't have to
> create
> > > > > a datastore entity for each different question each user views.
> >
> > > > > What do you guys think? Does anyone see any problems with this?
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine for Java" group.
> > > To post to this group, send email to
> > > google-appengine-j...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com><google-appengine-java%2B
> unsubscr...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine-java?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
