Jason Tackaberry wrote:
> Dirk Meyer wrote:
>> Yes. For kaa.vfs (and also kaa.thumb) I was thinking about a process
>> and using your IPC.
>
> Putting kaa.vfs in a separate process makes sense given that there are
> database locking issues. But for thumbnail generation, this could
> easily happen
Dirk Meyer wrote:
> And since Freevo uses threads for blocking IO (e.g. rom drive
> checking), we have GIL but most of the time we have no thread.
Which is fine.
> Yes. For kaa.vfs (and also kaa.thumb) I was thinking about a process
> and using your IPC.
Putting kaa.vfs in a separate process m
Jason Tackaberry wrote:
> Dirk Meyer wrote:
>> see that threading doesn't slow down things. Second (very strange):
>> when no thread is running, but threading itself is active, it is
>> slower. But still, not much.
>
> The GIL isn't initialized until a thread is created (or a C module
> explicitly
Dirk Meyer wrote:
> see that threading doesn't slow down things. Second (very strange):
> when no thread is running, but threading itself is active, it is
> slower. But still, not much.
The GIL isn't initialized until a thread is created (or a C module
explicitly initializes it, as in the case of
Toni Alatalo wrote:
> Martijn Voncken wrote:
>
>>>IPC can't be faster than threading because you need to convert
>>>
>>>
>>python has a GIL , and multicore is the future.
>>ipc will scale better on multicore processsors.
>>
>>
> also, threads may be slower due to internal locking they introdu
Martijn Voncken wrote:
IPC can't be faster than threading because you need to convert
python has a GIL , and multicore is the future.
ipc will scale better on multicore processsors.
also, threads may be slower due to internal locking they introduce to
the interpreter. i don't know if t
Jason Tackaberry wrote:
> Yes, adding a dir in a dir isn't going to work as it is. Probably the
> dirs table should have a dir_id column. Directory names should be
> decomposed into parts and new entries created for each part, with dir_id
> pointing to its parent.
>
> add_directory("/home/freev
On Sun, 2005-08-21 at 20:13 +0200, Dirk Meyer wrote:
> me. Two questions: How do I add a subdir? As dir in a dir? Or as file
> in a dir? Second: how to handle files in files (e.g. titles on a dvd)?
> Just create a dir entry with the filename?
Yes, adding a dir in a dir isn't going to work as it is
Martijn Voncken wrote:
> On 8/21/05, Jason Tackaberry <[EMAIL PROTECTED]> wrote:
>> On Sun, 2005-08-21 at 11:36 -0400, Jason Tackaberry wrote:
>> > Assuming indexed queries, it's the number of rows returned that matters,
>> > not the number of selects. A select which uses an indexed column which
>
On Sun, 2005-08-21 at 21:05 +0200, Dirk Meyer wrote:
> We we also add a generic key/value table, we should make sure we
> select this data for a File only when needed.
I think this generic key/value table would only be used in cases where
an existing file has a user-added attribute name (i.e. some
On Sun, 2005-08-21 at 20:13 +0200, Martijn Voncken wrote:
> The downside is that only advanced users can add
> ATTR_SEARCHABLE,ATTR_INDEXED or ATTR_KEYWORDS columns.
> I do like the idea of the ATTR_KEYWORDS column.
This is a problem that can be solved in the UI. If the UI wants to let
the user
Jason Tackaberry wrote:
> What we _can_ do is delay processing the row data into a more
> manageable form (i.e. unpickling the pickle column and moving the
> row data into a dict that's easier to work with) until after a
> simple file list is displayed in the UI. I think most of the big
> wins wil
On Sun, 2005-08-21 at 20:33 +0200, Dirk Meyer wrote:
> You both do different stuff.
That's what I thought too. But I wasn't sure if I misunderstood his
test. :)
> Question is: what do we want? In 90% of all cases we want a directory
> listing, not just one file. This means Tacks wins. One the o
On Sun, 2005-08-21 at 19:03 +0200, Dirk Meyer wrote:
> I should put a fxd file to images? Fxd files are known to Freevo, but
> I don't want to register a special table for it.
You can register the attributes that would be stored in the fxd files as
ATTR_SIMPLE for image file types. That way there
On Sun, 2005-08-21 at 19:53 +0200, Martijn Voncken wrote:
> i did a little test on that,most of the time is in python.
We have to count that time too. What we're experimenting with now is
what implementation is going to be faster.
> your-test: all static info from this directory.
> my-test:all d
On 8/21/05, Jason Tackaberry <[EMAIL PROTECTED]> wrote:
> For my prototype, attributes can be one of the following:
>
> 1. ATTR_SIMPLE: an attribute of arbitrary type that cannot be
> searched on. This is stored as a item in a pickled dict that is
> stored with the row.
>
Dirk Meyer wrote:
> So I guess I won't put non index files in the db, moving won't be a
> problem. And a generic table will do the trick.
After some testing (and adding unicode support), it looks good to
me. Two questions: How do I add a subdir? As dir in a dir? Or as file
in a dir? Second: how to
On 8/21/05, Jason Tackaberry <[EMAIL PROTECTED]> wrote:
> On Sun, 2005-08-21 at 11:36 -0400, Jason Tackaberry wrote:
> > Assuming indexed queries, it's the number of rows returned that matters,
> > not the number of selects. A select which uses an indexed column which
> > returns 0 rows is very, v
On 8/21/05, Jason Tackaberry <[EMAIL PROTECTED]> wrote:
> On Sun, 2005-08-21 at 17:30 +0200, Martijn Voncken wrote:
> > maybe i said something stupid here ;), i forgot about parsing the query
> etc..
> > but my point is that it's mosty the number of selects/sub selects that
> > makes things fast or
Jason Tackaberry wrote:
>> Still, what happens to bad files kaa.metadata can't parse? Where
>> should I add them? A guess a 'generic' table needs to be added. But
>> again: what happens when a file is identified later and is moved from
>> 'generic' to 'image'?
>
> Or add them to the image (or what
On Sun, 2005-08-21 at 12:27 -0400, Jason Tackaberry wrote:
> Attached is some (very ugly) test code which supports this statement.
I'm not sure what happened to the file attachment. Let's try that
again ...
from pysqlite2 import dbapi2 as sqlite
import os, time, sys
MULTI_CREATE_SCHEMA = """
On Sun, 2005-08-21 at 18:25 +0200, Dirk Meyer wrote:
> and...
>
> Jason Tackaberry wrote:
> > def dir(self, dir_id):
> > # XXX: this isn't how you'd want to do this, just testing ...
> > results = []
> > for type_name, (type_id, type_attrs) in self._file_types.items():
On Sun, 2005-08-21 at 11:36 -0400, Jason Tackaberry wrote:
> Assuming indexed queries, it's the number of rows returned that matters,
> not the number of selects. A select which uses an indexed column which
> returns 0 rows is very, very fast. It's practically a no-op.
Attached is some (very ug
and...
Jason Tackaberry wrote:
> def dir(self, dir_id):
> # XXX: this isn't how you'd want to do this, just testing ...
> results = []
> for type_name, (type_id, type_attrs) in self._file_types.items():
> [...]
> return results
This also doesn't lo
Jason Tackaberry wrote:
> My thinking is very much in line with dischi's. Have a table for each
> file type. These tables will have columns for important metadata (i.e.
> metadata that you'll want to search on later). These tables will also
> have a pickle column to store attributes we're not in
On Sun, 2005-08-21 at 17:30 +0200, Martijn Voncken wrote:
> maybe i said something stupid here ;), i forgot about parsing the query etc..
> but my point is that it's mosty the number of selects/sub selects that
> makes things fast or slow (assuming indexed query's)
Assuming indexed queries, it's t
Hi guys,
I had planned on writing something about this on Monday. But then I got
busy, and now I have family in town visiting for a while, and it's all
very annoying that I don't have lots of time for hacking. :)
As I said on IRC, I worked a bit on prototyping some ideas which I'll
explain below
On 8/21/05, Martijn Voncken <[EMAIL PROTECTED]> wrote:
> -the performance is often in traversing the index to find the starting
> point of your select.
maybe i said something stupid here ;), i forgot about parsing the query etc..
but my point is that it's mosty the number of selects/sub selects tha
On 8/21/05, Dirk Meyer <[EMAIL PROTECTED]> wrote:
> Martijn Voncken wrote:
[snip]
> IPC can't be faster than threading because you need to convert
> data. But maybe it still works very fast.
>
I prefer ipc above threading .
It is personal but threading is verry tricky.
I only use it as a blockin
Martijn Voncken wrote:
>> It is similar to the normal os operations. You have a listdir and you
>> can open files. But there are major differences:
>
> listdir->ok.
> an abstraction layer opening files can be tricky, please explain.
You try to open a file for writing (you can't open a file for rea
On 8/21/05, Dirk Meyer <[EMAIL PROTECTED]> wrote:
> Hi,
>
> The last few days I was thinking about a new mediadb. We did several
> rewrites in the past, so let us do this right this time. Martijn tried
> to put sqlite in the freevo mediadb, but he wanted to keep the old
> interface which turned ou
31 matches
Mail list logo
