[web logs -> db] On Tuesday 26 May 2009 00.27:03 Michael Bayer wrote: > the best thing to do would be to experiment with some various schemas > and see what works best
Also, it's extremely important to keep in mind that SQL databases can only
work well with big tables if you create the right indices. What kind of
index to create depends on the database you'll use and on which queries
you'll run. (And: don't use a small example data set to decide which
indices to build. A database often will change its query plan, sometimes
drastically, depending on how much data is in a table.) So: create your
schema, fill it with a few million log entries and then look at which
queries might need which indices.
In the case of web logs, you'll probably want an index on every base field
(tinestamp, user, IP, URL string), but depending on your exact queries,
combined indices on multiple column, or indices on functions of fields
(like, perhaps, an index on the ending of the url to quickly filter jpg/png
requests vs. html/php requests vs. requests on directories, ending by '/')
might speed up your queries dramatically.
cheers
-- vbi
--
Even though I use zsh, holding down the TAB key just doesn’t write my
code for me, so I have to use a text editor.
-- Scott James Remnant
signature.asc
Description: This is a digitally signed message part.
