Thanks Don,
I thought about the task queue, but that caps at being able to execute
like 5 tasks per second right?
So, as long as the log data doesn't get full before I download it then
it would be fine?
On Mar 10, 3:42 pm, Don Schwarz <schwa...@google.com> wrote:
> Yeah, those are diagnostic logs. They effectively go into a ring buffer per
> logging level, so the maximum data stored at any given time is capped. The
> more you log, the more frequently you would have to download the logs to
> avoid missing any. You would also be competing with log space with any
> other log messages generated by your application.
>
> What I would suggest instead is either to increment counters in memcache,
> and flush them to the datastore periodically if you need durability (I'm
> assuming you can tolerate some chance of data loss here). If you cannot
> tolerate any loss of data, then I would suggest enqueueing tasks to a task
> queue for each request that maintains a summary in memcache and/or the
> datastore.
>
>
>
> On Wed, Mar 10, 2010 at 4:59 PM, Spines <kwste...@gmail.com> wrote:
> > I'm talking about the logs that get written when I call
> > Logger.info("something").
>
> > Basically this is what I'm thinking: I have certain data that needs to
> > get written very often, but hardly ever needs to be read (stuff like
> > what users view what pages of my site). The datastore is optimized
> > for read efficiency. So, I want to output this data to the logs. I
> > will have an offsite computer download these logs, do calculations on
> > them, and upload the result of the calculations to the datastore.
>
> > On Mar 10, 2:03 pm, Don Schwarz <schwa...@google.com> wrote:
> > > Are you talking about request logs or diagnostic logs? Although we
> > conflate
> > > them a bit in both the Admin Console viewer and the appcfg command, but
> > they
> > > are stored and tracked separately.
>
> > > On Wed, Mar 10, 2010 at 4:01 PM, Spines <kwste...@gmail.com> wrote:
> > > > Hmm, that is my biggest concern, log reliability. Can someone from
> > > > Google confirm whether or not I can rely on the logs having all of the
> > > > log data? Or might certain entries just disappear?
>
> > > > On Mar 10, 1:24 pm, thierry Le conniat <thlec...@euriware.fr> wrote:
> > > > > Hello,
> > > > > I think google log are stored in file.
> > > > > My experience about log reliability is that when the app is very
> > > > > strong working, not all the log are stored.
> > > > > It's confusing, but i can't not explain it.
>
> > > > > Bye
>
> > > > > On 10 mar, 22:04, Spines <kwste...@gmail.com> wrote:
>
> > > > > > Where does Google store the logs when you do a Logging statement?
> > > > > > Logging statements seem to be pretty fast, so it doesn't seem like
> > > > > > they are stored in the datastore.
>
> > > > > > How reliable are the logs? If I do a logging statement and it
> > > > > > succeeds, is it pretty much guaranteed that it will show up in the
> > > > > > logs?
>
> > > > > > How much past history of logs is stored?
>
> > > > > > The reason I'm interested in this is because I'm making a question
> > and
> > > > > > answer website, and I want to keep track of views by each unique
> > > > > > logged in user to each question, and display the view count on the
> > > > > > question page. So if 10 different users visit the question page 100
> > > > > > times, it still only counts as 10 unique views.
>
> > > > > > I have an offsite computer that does background processing for my
> > app.
> > > > > > I'm planning to have this offsite computer download the logs about
> > > > > > every 30 minutes, and calculate what the view count should be for
> > each
> > > > > > question based off of the logs. By doing this, I don't have to
> > create
> > > > > > a datastore entity for each different question each user views.
>
> > > > > > What do you guys think? Does anyone see any problems with this?
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine for Java" group.
> > > > To post to this group, send email to
> > > > google-appengine-j...@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2B
> > > > unsubscr...@googlegroups.com><google-appengine-java%2B
> > unsubscr...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine-java?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to
> > google-appengine-j...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2B
> > unsubscr...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
