Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[zoltan.kis]

Hello,

> From: Shalamov Alexander
> Sent: 15 March, 2011 09:37
> On 03/14/2011 10:20 PM, ext Patrick Ohly wrote:
> > I've said before and I say it again here, I consider performance
> > comparisons pointless at this time. Doing them badly just reduces any
> > good will that might be left from the people one is trying to
> convince.
> It is not only about performance, strangely nobody mentioned why we
> have
> "multi-purpose" storage.
> Right now we have call history, SMS, IM, MMS, contacts and other types
> of data stored in tracker.
> 
> Therefore its very easy to write query like "give me full communication
> history for contact A".
> Our libcommhistory and commhistory daemon components rely on contacts
> that stored in tracker.
> 
> By moving contacts to EDS we will need to change libcommhistory and
> commhistory-daemon components
> so they will do "cross-domain" queries and we will pay with performance
> for that.
> 
> BR,
> Alex

Thanks for bringing this up, I tried to ask it earlier in this thread, but
it didn't catch fire and I got tired:
"what are the test cases both Tracker and EDF (or any other storage) should
be measured against? Including horizontal use cases of apps (e.g. building
conversation history etc), with constraints on system load, use time, etc."

These horizontal use cases were the primary reason Tracker was used for
commhistory. Patrick you might remember I explained this when I shared our
experiments with a proposed application common database (sqlite vs
postgresql), where I also listed our technical requirements towards storages
and some example queries. They are mainly covered in the tracker test cases
shared by Philip earlier. 

The conclusion was that contacts,  IM [and call history] should share the
same database because of the nature of the queries. Since contacts was
stubbornly bound to tracker, commhistory had to be in Tracker too. It took a
good while to make the performance acceptable with tracker, but once it's
done, it looks like it's ditched and someone will have to start that work
again. Perfect.

Well, I understand that 
- after 2/11, Nokia's role in MeeGo became somewhat undefined, 
- MeeGo releases need to roll (thanks Intel)
- Intel has EDS competence, but does not have control over tracker
- there are performance question marks for tracker in certain use cases
= therefore an architecture decision favoring EDS was made in order to be
able to move forward in a controllable way.

This would be OK, if the decision was made according to the published
governance model, if the tracker team was asked about the issues, if there
were clear requirements and verification criteria, and if it was clearly
outlined how EDS is going to solve the problems Tracker allegedly did not. 

Questions regarding this have not been answered. That suggests the questions
were not even considered (bad), or Intel doesn't even take the burden to
answer (bad, suggesting "you pull off, we do our way, bye"). Good luck.

I don't know what is the MeeGo governance model today
(http://meego.com/about/governance looks to be outdated). What I know is
that for any architecture decision it is a MUST to
- understand the problem and have clear requirements (this is missing, or
not communicated)
- understand all dependencies (this is partially missing)
- define the target application space (e.g. handset, netbook, automotive,
etc) (this was missing, handset scope clearly has different priorities)
- have clear verification criteria (this is missing)
- consult all stakeholders (this was missing)
- present alternative options (this is missing, i.e. how EDS solves the
problems (what problems, actually?))
- select one option and stick to it for the given time span (this was done).

Please tell me what is the chance this is a good decision? It is not true
that the above necessarily takes too long, and the people able to drive it
are known. In matters of days a right decision can be made.
Could it be that for certain applications EDS is better, and for others
Tracker is better? Which and when? If a uniform solution is to be chosen
(over a wide range of devices), then which one is chosen? When?

As said last time, you made the choice, fine, go and do it.
Please make sure you define the missing things from the list above and next
time play by clear rules.
And someone needs to fork/rewrite/adapt libcommhistory (provided you want a
Qt UI), or ditch it too and use something else. 

Regards,
Zoltan



smime.p7s
Description: S/MIME cryptographic signature
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Sivan Greenberg]

On Tue, Mar 15, 2011 at 9:10 AM, Ville M. Vainio  wrote:
> On Mon, Mar 14, 2011 at 9:20 PM, Patrick Ohly  wrote:
>
>> I've said before and I say it again here, I consider performance
>> comparisons pointless at this time.
>
> Considering that e-d-s has a much more modest feature set than tracker
> (tracker in general being a much more ambitious project), I would have
> expected it to to trounce tracker in performance, which doesn't seem
> to be the case.

I am wondering if EDS was tested using flat file storage ? (sorry for
my ignorance in reading the test code and setup)

>
> This evidence might prompt to re-evaluate this part of the
> architectural plans. Or at least leave the door open to transitioning
> back to tracker when it's feasible.

Isn't this the situation already? I remember Arjan saying "this
round". I at least hope that what we are aiming for is to first
release something out the door already, usable to some degree, and
then refactor/ improve when technologies get more baked and their
interfaces.

-Sivan
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Ross Burton]

On Thu, 2011-03-10 at 12:45 +0100, Philip Van Hoof wrote:
> I also wonder what version of EDS is being proposed for MeeGo. The
> eds-fremantle or GNOME's EDS. And how much of the changes in for the
> Fremantle version are now in GNOME's upstream EDS? 

Upstream, obviously.  At least if I have any say in it, which I do.

(redacted comments on fremantle's EDS)

Ross
-- 
Intel Open Source Technology Centre
http://oss.intel.com/
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Adrien Bustany]

On Tue, 15 Mar 2011 10:21:12 +0100, Mathias Hasselmann wrote:

Am Dienstag, den 15.03.2011, 09:33 +0200 schrieb Adrien Bustany:

On Tue, 15 Mar 2011 08:10:18 +0100, Ville M. Vainio wrote:
> On Mon, Mar 14, 2011 at 9:20 PM, Patrick Ohly
>  wrote:
>
>> I've said before and I say it again here, I consider performance
>> comparisons pointless at this time.
>
> Considering that e-d-s has a much more modest feature set than
> tracker
> (tracker in general being a much more ambitious project), I would
> have
> expected it to to trounce tracker in performance, which doesn't 
seem

> to be the case.
>
> This evidence might prompt to re-evaluate this part of the
> architectural plans. Or at least leave the door open to 
transitioning

> back to tracker when it's feasible.

 If you're interested in the saving performance of both solutions, I
 answered the thread on the Tracker ML (didn't want to cross-spam
 Meego-Dev). If you abstract the fact that EDS has no batching API 
(and
 therefore seems to issue a fsync after saving each contact) by 
running

 it over libeatmydata, EDS is approximately twice faster than
 qtcontacts-tracker (though that area is being optimized currently). 
I

 haven't done any contact fetching benchmarks.


Could you post an archive link here for convenience?

Thank you,
Mathias


http://mail.gnome.org/archives/tracker-list/2011-March/msg00035.html

Cheers

Adrien
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Dienstag, den 15.03.2011, 09:33 +0200 schrieb Adrien Bustany:
> On Tue, 15 Mar 2011 08:10:18 +0100, Ville M. Vainio wrote:
> > On Mon, Mar 14, 2011 at 9:20 PM, Patrick Ohly 
> >  wrote:
> >
> >> I've said before and I say it again here, I consider performance
> >> comparisons pointless at this time.
> >
> > Considering that e-d-s has a much more modest feature set than 
> > tracker
> > (tracker in general being a much more ambitious project), I would 
> > have
> > expected it to to trounce tracker in performance, which doesn't seem
> > to be the case.
> >
> > This evidence might prompt to re-evaluate this part of the
> > architectural plans. Or at least leave the door open to transitioning
> > back to tracker when it's feasible.
> 
>  If you're interested in the saving performance of both solutions, I
>  answered the thread on the Tracker ML (didn't want to cross-spam
>  Meego-Dev). If you abstract the fact that EDS has no batching API (and
>  therefore seems to issue a fsync after saving each contact) by running
>  it over libeatmydata, EDS is approximately twice faster than
>  qtcontacts-tracker (though that area is being optimized currently). I
>  haven't done any contact fetching benchmarks.

Could you post an archive link here for convenience?

Thank you,
Mathias

-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Tue, 2011-03-15 at 09:33 +0200, Adrien Bustany wrote:
> On Tue, 15 Mar 2011 08:10:18 +0100, Ville M. Vainio wrote:

Hey Adrien, and Hey Ville M. Vainio,

> > On Mon, Mar 14, 2011 at 9:20 PM, Patrick Ohly 
> >  wrote:

> >> I've said before and I say it again here, I consider performance
> >> comparisons pointless at this time.
> >
> > Considering that e-d-s has a much more modest feature set than 
> > tracker (tracker in general being a much more ambitious project), I would 
> > have expected it to to trounce tracker in performance, which doesn't seem
> > to be the case.

I share the exact same opinion. I'd also like to point out, in defense
of "Evolution UI was running, this creates load on the E-D-S store" that
Tracker's RDF store has a direct-access method for opening the database
by clients due to its use of SQLite's WAL journaling mode.

This means that when a direct-access - enabled Tracker client reads from
the database, it doesn't disturb the tracker-store process directly
(only indirectly due to scheduling, but with process priorities in Linux
also this can be completely avoided).

GraphUpdated can be used in combination with Tracker's direct-access
mode. You get GraphUpdated about data deltas post the transaction
commit, so with SQLite WAL it's then available everywhere.

On the subject process priorities: SCHED_IDLE vs. SCHED_FIFO; with
direct-access you'll have it and you wont have to worry about the
process priority of the service tracker-store.

This of course depends on your process *having* access to meta.db (as I
mentioned in another E-mail can Aegis on Harmattan prevent you from
having access to meta.db, meaning that you must go over a IPC with
tracker-store to do read queries too). This IPC then uses FD passing
over D-Bus.

> > This evidence might prompt to re-evaluate this part of the
> > architectural plans. Or at least leave the door open to transitioning
> > back to tracker when it's feasible.
> 
>  If you're interested in the saving performance of both solutions, I
>  answered the thread on the Tracker ML (didn't want to cross-spam
>  Meego-Dev). If you abstract the fact that EDS has no batching API (and
>  therefore seems to issue a fsync after saving each contact) by running
>  it over libeatmydata, EDS is approximately twice faster than
>  qtcontacts-tracker (though that area is being optimized currently). I
>  haven't done any contact fetching benchmarks.

Note that it looks like you have e_book_add_contacts, commit_contacts
and remove_contacts. Those look like batch APIs to me. However, in the
example that I made for Tracker, I also didn't use Tracker's batch APIs
BatchSparqlUpdate() nor did I use its UpdateArray().

BatchSpaqlUpdate() is mostly about prioritization, UpdateArray() is
mostly about reducing D-Bus traffic.

I think it's interesting to explain Tracker's fsync() behaviour of its
RDF store:

Because we believe that fsync() is only necessary to avoid data loss in
case of an unclean shutdown of the system (when the filesystem isn't
unmounted but the power is cut nonetheless), we only do fsync() and only
on our journal file in case an application calls the Resources.Sync()
D-Bus call. We also open our SQLite db in a specific way.

Our SQLite database is always opened using:

PRAGMA synchronous = OFF
PRAGMA count_changes = 0
PRAGMA temp_store = FILE
PRAGMA auto_vacuum = 0
PRAGMA encoding = "UTF-8"
PRAGMA journal_mode = WAL

http://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-db-manager.c#n214

With synchronous = OFF wont SQLite do any such fsync() call. This is OK
because we do our own persistent journaling:

http://git.gnome.org/browse/tracker/tree/src/libtracker-data/tracker-db-journal.c


Doing an fsync() on a high-latency device like many nand and flash disks
are, is a very big contributor to ie. reduced UI responsiveness.

A monitor application that sees that the battery-bay is being opened can
for example call the Resources.Sync() D-Bus call as soon as possible. 

For devices that do a clean emergency shutdown on critical battery, but
where the battery can't be touched unless you void warranty of the
device, this isn't needed (at least theoretically: a crasher bug being
the exception here).

Cheers,

Philip

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Alexander Shalamov]

Hi Patrick,

On 03/14/2011 10:20 PM, ext Patrick Ohly wrote:
> I've said before and I say it again here, I consider performance
> comparisons pointless at this time. Doing them badly just reduces any
> good will that might be left from the people one is trying to convince.
It is not only about performance, strangely nobody mentioned why we have
"multi-purpose" storage.
Right now we have call history, SMS, IM, MMS, contacts and other types
of data stored in tracker.

Therefore its very easy to write query like "give me full communication
history for contact A".
Our libcommhistory and commhistory daemon components rely on contacts
that stored in tracker.

By moving contacts to EDS we will need to change libcommhistory and
commhistory-daemon components
so they will do "cross-domain" queries and we will pay with performance
for that.

BR,
Alex
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Adrien Bustany]

On Tue, 15 Mar 2011 08:10:18 +0100, Ville M. Vainio wrote:
On Mon, Mar 14, 2011 at 9:20 PM, Patrick Ohly 
 wrote:



I've said before and I say it again here, I consider performance
comparisons pointless at this time.


Considering that e-d-s has a much more modest feature set than 
tracker
(tracker in general being a much more ambitious project), I would 
have

expected it to to trounce tracker in performance, which doesn't seem
to be the case.

This evidence might prompt to re-evaluate this part of the
architectural plans. Or at least leave the door open to transitioning
back to tracker when it's feasible.


If you're interested in the saving performance of both solutions, I
answered the thread on the Tracker ML (didn't want to cross-spam
Meego-Dev). If you abstract the fact that EDS has no batching API (and
therefore seems to issue a fsync after saving each contact) by running
it over libeatmydata, EDS is approximately twice faster than
qtcontacts-tracker (though that area is being optimized currently). I
haven't done any contact fetching benchmarks.

Cheers

Adrien

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Ville M. Vainio]

On Mon, Mar 14, 2011 at 9:20 PM, Patrick Ohly  wrote:

> I've said before and I say it again here, I consider performance
> comparisons pointless at this time.

Considering that e-d-s has a much more modest feature set than tracker
(tracker in general being a much more ambitious project), I would have
expected it to to trounce tracker in performance, which doesn't seem
to be the case.

This evidence might prompt to re-evaluate this part of the
architectural plans. Or at least leave the door open to transitioning
back to tracker when it's feasible.
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Mo, 2011-03-14 at 19:03 +, Philip Van Hoof wrote:


Hi,
  
> http://mail.gnome.org/archives/tracker-list/2011-March/msg00033.html

Okay, I bite.

> The comparison is favoring Tracker in a number of ways:
>  * Having the Evolution UI running while inserting contacts into
>EDS slows down the whole process because all data is also
>getting sent back to the UI. This is more complex than just
>inserting the data into Tracker, which has nothing reading that
>data during that time.

Only seconds after the application finished did the Evo UI became 
unresponsive. The second run took 11s instead of 10s, I can
imagine that this 1s difference is the impact. That's of course
just speculation. Somebody would need to investigate this to be
sure.

However, on favoring:

Note that the E-D-S test appended contacts. It didn't replace
contacts. The Tracker test replaces contacts.

If I append contacts with Tracker, the time to add 1000 contacts
is under 5s instead of between 12s and 15s.

So I gave a 'huge' favor to E-D-S, to be honest. I should for the
E-D-S test lookup each contact and delete it, before adding
the new one and committing it.

 > * You use the low-level Tracker API to create contacts in Tracker.
 >   QtContacts should have been used instead, because that is the
 >   API that is relevant for MeeGo and that corresponds to the one
 >   you used for EDS.

There is no lower level API for E-D-S as far as I know ..

Afaik will the qct people repeat the test. Note, however,
that the query that we used comes from QtContacts project.

So it wont be much different.

QtContacts doesn't add a lot of overhead on top of the queries
that it produces. Not in performance.

> Of course you have a point about the system as a whole, but
> that still doesn't make the comparison any better.

Can Evolution's UI be running on MeeGo?

> I could speculate whether this comparison was skewed
> intentionally or unintentionally, but let's not go
> there, okay?

I prefer to stick to the numbers, yes.

> I've said before and I say it again here, I consider
> performance comparisons pointless at this time.

Why are they pointless?

> Doing them badly just reduces any good will that might
> be left from the people one is trying to convince.

More tests, including ones with QtContacts, will follow.

Cheers,

Philip

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.


___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines


___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Patrick Ohly]

On Mo, 2011-03-14 at 19:03 +, Philip Van Hoof wrote:
> On Mon, 2011-03-07 at 08:09 -0800, Arjan van de Ven wrote:
> 
> Hi Arjan,
> 
> > PIM storage
> > ===
> > The Address book, Calendar data and Email are currently stored in a 
> > tracker database, and accessed (officially) via a QtMobility API set.
> > There are a range of issues with this implementation, starting with the 
> > complexity of adding privacy controls, the performance and
> > scalability as well as the completeness for doing a proper syncml sync.
> 
> 
> http://mail.gnome.org/archives/tracker-list/2011-March/msg00033.html

Okay, I bite.

The comparison is favoring Tracker in a number of ways:
  * Having the Evolution UI running while inserting contacts into
EDS slows down the whole process because all data is also
getting sent back to the UI. This is more complex than just
inserting the data into Tracker, which has nothing reading that
data during that time.
  * You use the low-level Tracker API to create contacts in Tracker.
QtContacts should have been used instead, because that is the
API that is relevant for MeeGo and that corresponds to the one
you used for EDS.

Of course you have a point about the system as a whole, but that still
doesn't make the comparison any better. I could speculate whether this
comparison was skewed intentionally or unintentionally, but let's not go
there, okay?

I've said before and I say it again here, I consider performance
comparisons pointless at this time. Doing them badly just reduces any
good will that might be left from the people one is trying to convince.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.


___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Mon, 2011-03-07 at 08:09 -0800, Arjan van de Ven wrote:

Hi Arjan,

> PIM storage
> ===
> The Address book, Calendar data and Email are currently stored in a 
> tracker database, and accessed (officially) via a QtMobility API set.
> There are a range of issues with this implementation, starting with the 
> complexity of adding privacy controls, the performance and
> scalability as well as the completeness for doing a proper syncml sync.


http://mail.gnome.org/archives/tracker-list/2011-March/msg00033.html

Cheers,

Philip


-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Thu, 2011-03-10 at 12:39 +0100, Mathias Hasselmann wrote:
> Am Donnerstag, den 10.03.2011, 12:02 +0100 schrieb Philip Van Hoof:
> > On Wed, 2011-03-09 at 23:24 +0100, Mathias Hasselmann wrote:
> > > PS: To get some more numbers I ran few quick searches on bugs.gnome.org:
> > > 
> > >   ":evolution-data-server :contacts" finds 70 bugs
> > >   ":evolution-data-server" even finds 354 bugs
> > 
> > Nod. And it's not that several years ago at Nokia, during Fremantle,
> > people where 'satisfied' with EDS. I haven't seen huge improvements
> > happening to EDS since that period.
> 
> Fremantle's EDS version differs greatly from GNOME EDS. To get a
> realistic idea of the effort needed for turning libebook into something
> that's somewhat production ready, take a look at at the long commit
> history of Fremantle EDS: http://maemo.gitorious.org/eds-fremantle/

And still the teams using it were not satisfied in the end :)

At least that's what I remember.

I also wonder what version of EDS is being proposed for MeeGo. The
eds-fremantle or GNOME's EDS. And how much of the changes in for the
Fremantle version are now in GNOME's upstream EDS?

> Yes, we should have tried harder to get into upstream EDS development.
> First our version of libebook-dbus was in horrible. Later the delta
> became too large to handle. Too little time. Too many secrets around the
> N900. Naive planing. :-/

Ah, yes. Good old times Mathias. Good old times. We'll recover from that
when we're old. At some conference while complaining about those young
guys, with a good beer. You know. :-)


Cheers,

Philip


-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Donnerstag, den 10.03.2011, 12:02 +0100 schrieb Philip Van Hoof:
> On Wed, 2011-03-09 at 23:24 +0100, Mathias Hasselmann wrote:
> > PS: To get some more numbers I ran few quick searches on bugs.gnome.org:
> > 
> >   ":evolution-data-server :contacts" finds 70 bugs
> >   ":evolution-data-server" even finds 354 bugs
> 
> Nod. And it's not that several years ago at Nokia, during Fremantle,
> people where 'satisfied' with EDS. I haven't seen huge improvements
> happening to EDS since that period.

Fremantle's EDS version differs greatly from GNOME EDS. To get a
realistic idea of the effort needed for turning libebook into something
that's somewhat production ready, take a look at at the long commit
history of Fremantle EDS: http://maemo.gitorious.org/eds-fremantle/

Yes, we should have tried harder to get into upstream EDS development.
First our version of libebook-dbus was in horrible. Later the delta
became too large to handle. Too little time. Too many secrets around the
N900. Naive planing. :-/

Ciao,
Mathias

-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Wed, 2011-03-09 at 23:24 +0100, Mathias Hasselmann wrote:
> Am Mittwoch, den 09.03.2011, 07:30 -0800 schrieb Arjan van de Ven:

> So speaking about qtcontacts-tracker: Can you point me to bug reports,
> to or broken promises?
> 
> A quick search for "qtcontacts-tracker" on bugs.meego.com finds 19 bugs.
> Not a single show stopper judging from summaries. Nokia's internal bug
> tracker lists 17 bugs for qtcontacts-tracker right now. Our test suite
> (about 10k lines of code) reports 139 of 139 passed.
> 
> So what actually are the show stoppers?

We are now adding support for REPLACE in Tracker's SPARQL Update engine.
We know that qtcontacts-tracker had to use a lot of DELETE-WHERE-INSERT
constructions and it's likely that the DELETE-WHERE part and old values
check of INSERT is a major contributor to the Update performance for
qtcontacts-tracker's use-cases.

I think Adrien Bustany has started experimenting with this branch today.
So if Update performance is a show-stopper we will have new numbers this
week.

I made an article explaining how the current syntax works:

http://pvanhoof.be/blog/index.php/2011/03/09/a-replace-extension-for-trackers-sparqls-update

Note that qtcontacts-tracker can also, on top of the REPLACE that we're
adding this week, use UpdateArray (it's not yet doing that, afaik).

UpdateArray can be instrumental in reducing D-Bus traffic and calls (as
you pack multiple queries per call and yet you get per-query errors).

Note that we already use FD passing to avoid most of D-Bus's performance
problems.

> Where is the email telling us qtcontacts-tracker developers that there
> are show stopping issues? At least Patrick knows very well how to
> contact us.

Yes, we did see "some" E-mails on Tracker's upstream mailing list by
people from Intel. We also received a few patches and thanks to that we
gave priority to our UpdateArray API, then reworked their patch to fit
this.

But never any performance figures, requirements, big problems, etc.

It's very hard to claim that we are unresponsive towards Intel. There's
plenty of mailing list material to back that up.

I also do wonder where these 'discussions' that Arjan talked about took
place. Respected people, like David Neary, have asked the same question
in this meego-dev ML thread:

On Tue, 2011-03-08 at 12:01 +0100, Dave Neary wrote: 
> Hi,
> 
> Arjan van de Ven wrote:
> > Time has come and gone for this to be a discussion; this is a decision.
> 
> Out of interest, when was the time for this to be a discussion? And
> where did that discussion happen?
> 
> Thanks,
> Dave.

But I see a pattern here: all our questions are left unanswered or have
so far been answered by emotional responses.

But hey, fine. Been there, done that (Fremantle -> Harmattan).

> PS: To get some more numbers I ran few quick searches on bugs.gnome.org:
> 
>   ":evolution-data-server :contacts" finds 70 bugs
>   ":evolution-data-server" even finds 354 bugs

Nod. And it's not that several years ago at Nokia, during Fremantle,
people where 'satisfied' with EDS. I haven't seen huge improvements
happening to EDS since that period.


Cheers,

Philip


-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Martyn Russell]

On 09/03/11 19:23, Niels Mayer wrote:

On Mon, Mar 7, 2011 at 8:09 AM, Arjan van de
Ven  wrote:

To be clear, this does not mean that "tracker" is completely
removed; tracker is still being used (together with tumbler) for
indexing media on the device. At this point we are seeing serious
issues (performance/stability) with this solution, but the first
attempt will be to fix the deficiencies rather than a replacement.


Actually, where exactly are tracker's results used in the Netbook UX
for indexing media? I thought banshee was independently indexing
media as well... sometimes they even get into a fight about it:
http://lists.meego.com/pipermail/meego-community/2011-February/003597.html


There is no evidence on that link that proves Banshee and Tracker fight
with each other, the syslog has no relation as far as I can see to
Tracker. What are you basing this claim on?


At least on the 1.2 Netbook release, tracker does cause problems --
I've disabled its media scanning  to prevent issues ongoing issues:
http://lists.meego.com/pipermail/meego-community/2011-February/003605.html


What version is being used in this release?

This link suggests people really don't know what Tracker is doing. There
is no churn when downloading a file, we only index it once it is written
with CLOSE_WRITE.

Clearly Banshee is the problem there. What makes sense is to have a
plugin for Banshee to Tracker to avoid competing for processor time so
they can work together.

Also, if files are being copied around, it is possible to tell Tracker
to ignore certain directories, this may improve things too.


For Maemo issues with tracker -- and potential customizations  to
prevent performance issues:
http://lists.meego.com/pipermail/meego-community/2011-February/003598.html


The above link references another link from talk.maemo.org where they
discuss version 0.6.95.x of Tracker. We're not at version 0.10.1. These
versions are worlds apart.

Performance wasn't great for that version and it is supremely better now.

If people turn off Tracker, what happens to the other applications using
it? Do they just suffer?


 There appears to be duplicated functionality between tracker and
banshee's media indexing. If other tracker functionality has been
removed for 1.2, and tracker's media indexing results seem to be
unused by banshee, what about removing tracker entirely from the 1.2
Netbook UX?


Because you're taking a step backwards. The data is then not shared or 
useful between applications on the device and certainly not for 3rd 
party applications.


We've already seen this sort of approach before with Maemo, each group 
decides to sit in their corner writing their own DB for their own 
solution and then one day, someone wants information from another 
component in the system and wonders how to share it. This seems to be 
what Banshee have done.



there's a lot of impact from this currently, both in terms of just
raw CPU cycles to power impact to stability (we're seeing quite
some crashes, which worries me from a security pov)


I'm glad this is being considered and I concur. One other issue for
indexing: if a gstreamer codec needs to be invoked first, e..g. for
indexing or thumbnailing media, then is that potentially untrusted
or closed-source codec sandboxed, when executing? What of the
security of the gstreamer pipeline against media-based attack
vector?


Tracker (like applications) go through GStreamer for codec support, 
that's nothing to do with us. If it is inherently insecure, that's not 
Tracker's fault. Do you have proof or evidence this is the case?


--

All in all, I am quite concerned. I see a lot of random comments about 
Tracker not being good enough, generally based on old data or old 
threads or without evidence. That combined absolutely NO communication 
with core maintainers and channels upstream is not a recipe for success. 
Note: We're handling bug reports and patches from 4 bug trackers, 
watching 2 IRC channels and 4 mailing lists all the time.


--
Regards,
Martyn
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Mittwoch, den 09.03.2011, 07:30 -0800 schrieb Arjan van de Ven:
> On 3/9/2011 2:44 AM, Jeremiah Foster wrote:
> > On Mon, Mar 7, 2011 at 5:09 PM, Arjan van de Ven  
> > wrote:
> >> Hi,
> >>
> >> given the events of the last few weeks, the MeeGo architects have, and 
> >> still
> >> are, revisiting various parts of the MeeGo architecture.
> > Aside from this thread, where does this discussion happen? How do others 
> > engage?
> 
> this discussion has mostly been happening with the architects of the 
> handset and tablet (UI and OS) as well as consulting with a few of the 
> other vertical architects.
> 
> >> Having said that, three items are currently clear enough to make a final
> >> decision on:
> >>
> >> 1) MSSF / MeeGo security framework
> >> 2) Buteo Sync
> >> 3) PIM storage (currently stored in the tracker database)
> > Reading between the lines, it appears that the MeeGo architects are
> > re-evaluating Nokia contributions.
> 
> No we are re-evaluating those areas where there is an issue now. Areas 
> that are very complete and functional because the responsible
> folks made sure worked well in MeeGo aren't even close to being up for 
> discussion, Nokia or otherwise. This is about an evaluation of real gaps 
> in MeeGo
> functionality for whatever reason (never got implemented, the developers 
> focused on Harmattan(Maemo) instead of MeeGo, something stayed binary
> and never got open sourced, the area never got resourced, whatever) are 
> severely lacking completeness/functionality in MeeGo.

So speaking about qtcontacts-tracker: Can you point me to bug reports,
to or broken promises?

A quick search for "qtcontacts-tracker" on bugs.meego.com finds 19 bugs.
Not a single show stopper judging from summaries. Nokia's internal bug
tracker lists 17 bugs for qtcontacts-tracker right now. Our test suite
(about 10k lines of code) reports 139 of 139 passed.

So what actually are the show stoppers?

Where is the email telling us qtcontacts-tracker developers that there
are show stopping issues? At least Patrick knows very well how to
contact us.

Ciao,
Mathias

PS: To get some more numbers I ran few quick searches on bugs.gnome.org:

  ":evolution-data-server :contacts" finds 70 bugs
  ":evolution-data-server" even finds 354 bugs

-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Niels Mayer]

On Mon, Mar 7, 2011 at 8:09 AM, Arjan van de Ven  wrote:
> To be clear, this does not mean that "tracker" is completely removed;
> tracker is still being used (together with tumbler) for indexing media
> on the device. At this point we are seeing serious issues
> (performance/stability) with this solution, but the first attempt will be to
> fix the deficiencies rather than a replacement.

Actually, where exactly are tracker's results used in the Netbook UX
for indexing media? I thought banshee was independently indexing media
as well... sometimes they even get into a fight about it:
http://lists.meego.com/pipermail/meego-community/2011-February/003597.html

At least on the 1.2 Netbook release, tracker does cause problems --
I've disabled its media scanning  to prevent issues ongoing issues:
http://lists.meego.com/pipermail/meego-community/2011-February/003605.html

For Maemo issues with tracker -- and potential customizations  to
prevent performance issues:
http://lists.meego.com/pipermail/meego-community/2011-February/003598.html

There appears to be duplicated functionality between tracker and
banshee's media indexing. If other tracker functionality has been
removed for 1.2, and tracker's media indexing results seem to be
unused by banshee, what about removing tracker entirely from the 1.2
Netbook UX?

> there's a lot of impact from this currently, both in terms of just raw CPU 
> cycles to power impact to stability (we're seeing quite some crashes, which 
> worries me from a security pov)

I'm glad this is being considered and I concur. One other issue for
indexing: if a gstreamer codec needs to be invoked first, e..g. for
indexing or thumbnailing media, then is that potentially untrusted or
closed-source codec sandboxed, when executing? What of the security of
the gstreamer pipeline against media-based attack vector?

-- Niels
http://nielsmayer.com
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Vitaly Repin]

Hello,

On Tue, 08 Mar 2011, ext David Woodhouse wrote:

> At this point it seems like it'll *so* much be quicker to reimplement
> it,

Mmm. Then good luck in re-implementing it. I'm not sarcastic here.

> Which part isn't obvious? That we need ActiveSync support, to
> communicate with legacy crap systems that can't talk standard protocols?

Yes. It is not obvious for everyone. Unfortunately.

> Or that a closed source implementation is a complete non-starter, since
> it can't be debugged and well-integrated into the system?

This seems to be also not obvious for everyone.

> Can we focus on the bit about the DBus API first? There's no *licensing*
> issue there, right? You could share that without having to consult the
> lawyers? Or just implement DBus introspection in the next release... :)

Unfortunately I need to consult lawyers about opensourcing anything.
Even if it's "hello, world" app made by me in working hours.

I'll try anyway.

> I've love to be proven wrong in my assumption that Nokia will never
> manage to pull their finger out and fix the MfE licensing in a
> reasonable amount of time. By all means, please put me in touch with the
> right people and we'll see if we can make it happen.

I'll send you private mail about this.
-- 
WBR & WBW,
Vitaly Repin,
Productivity, Nokia-D
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Arjan van de Ven]

On 3/9/2011 2:44 AM, Jeremiah Foster wrote:

On Mon, Mar 7, 2011 at 5:09 PM, Arjan van de Ven  wrote:

Hi,

given the events of the last few weeks, the MeeGo architects have, and still
are, revisiting various parts of the MeeGo architecture.

Aside from this thread, where does this discussion happen? How do others engage?


this discussion has mostly been happening with the architects of the 
handset and tablet (UI and OS) as well as consulting with a few of the 
other vertical architects.



Having said that, three items are currently clear enough to make a final
decision on:

1) MSSF / MeeGo security framework
2) Buteo Sync
3) PIM storage (currently stored in the tracker database)

Reading between the lines, it appears that the MeeGo architects are
re-evaluating Nokia contributions.


No we are re-evaluating those areas where there is an issue now. Areas 
that are very complete and functional because the responsible
folks made sure worked well in MeeGo aren't even close to being up for 
discussion, Nokia or otherwise. This is about an evaluation of real gaps 
in MeeGo
functionality for whatever reason (never got implemented, the developers 
focused on Harmattan(Maemo) instead of MeeGo, something stayed binary
and never got open sourced, the area never got resourced, whatever) are 
severely lacking completeness/functionality in MeeGo.


This also has nothing to do with ARM optimization or not. I don't even 
understand how you would think it would.



___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Jeremiah Foster]

On Mon, Mar 7, 2011 at 5:09 PM, Arjan van de Ven  wrote:
> Hi,
>
> given the events of the last few weeks, the MeeGo architects have, and still
> are, revisiting various parts of the MeeGo architecture.

Aside from this thread, where does this discussion happen? How do others engage?

> Having said that, three items are currently clear enough to make a final
> decision on:
>
> 1) MSSF / MeeGo security framework
> 2) Buteo Sync
> 3) PIM storage (currently stored in the tracker database)

Reading between the lines, it appears that the MeeGo architects are
re-evaluating Nokia contributions. This is well and good, but the
question that you do not address Arjan is who is going to be
responsible for assuring that this re-evaluation is done with careful
consideration of both MeeGo's stated target architectures? If you rip
out libraries and software that is optimized or designed for the ARM
platform for example that may lead to the perception that MeeGo is
sliding back in Moblin.

Careful assessment of the current architecture requirements needs to
be done from a cross-platform perspective and done in the open. It
would be good if MeeGo had more ARM experts at the architects level to
ensure parity in at least discourse on design. Right now it is in
danger of sinking into a back room monolog.

Regards,

Jeremiah
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Graham Cobb]

On Wednesday 09 March 2011 00:59:28 Peter Robinson wrote:
> No idea if its suitable but evolution-mapi at least works with
> exchange not sure of the difference between mapi and active sync
> support.

Unfortunately they are very different: MAPI is a very different sort of 
protocol 
which basically provides direct access to the Exchange server, which no 
enterprise will allow outside their intranet.  ActiveSync is designed for 
sync, is limited in what it can do (deliberately) and is designed to work 
through firewalls/gateways.  And a bunch of other stuff (like administrators 
can 
limit sync to devices which report that they provide a remote-wipe feature, 
etc).

Graham
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Martyn Russell]

On 09/03/11 07:47, Martyn Russell wrote:

On 09/03/11 01:00, Peter Robinson wrote:

On Tue, Mar 8, 2011 at 11:58 AM, wrote:

Hi,

From: ext Martyn Russell [mar...@lanedo.com]
Sent: 08 March 2011 13:53


As you can imagine, if you want to be able to query, insert and merge
120k of contracts, you really can't have your cake and eat it so to
speak without some sacrifices one way or another.


That's not true.

Nothing prevents from having 2 DB, where the first one is the usual
(could be considered as cache)
and the other is much larger and used only in case the first search
fails.
Maybe with the explicit consent from the user to proceed with such
expensive operation.

Obviously there are problems of keeping the DBs in synch, but they do
not
seem impossible to solve, for example deferring the work to a later
time when
the device is idle and connected to a power source.


I thought tracker already supported indexing a e-d-s data store anyway.


It does.



Let me expand on that. We have a plugin for Evolution, so it will index 
data about emails by pushing them to Tracker. We do not talk to e-d-s 
directly.


--
Regards,
Martyn
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Martyn Russell]

On 09/03/11 01:00, Peter Robinson wrote:

On Tue, Mar 8, 2011 at 11:58 AM,  wrote:

Hi,

From: ext Martyn Russell [mar...@lanedo.com]
Sent: 08 March 2011 13:53


As you can imagine, if you want to be able to query, insert and merge
120k of contracts, you really can't have your cake and eat it so to
speak without some sacrifices one way or another.


That's not true.

Nothing prevents from having 2 DB, where the first one is the usual (could be 
considered as cache)
and the other is much larger and used only in case the first search fails.
Maybe with the explicit consent from the user to proceed with such
expensive operation.

Obviously there are problems of keeping the DBs in synch, but they do not
seem impossible to solve, for example deferring the work to a later time when
the device is idle and connected to a power source.


I thought tracker already supported indexing a e-d-s data store anyway.


It does.

--
Regards,
Martyn
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Peter Robinson]

On Tue, Mar 8, 2011 at 11:58 AM,   wrote:
> Hi,
>
> From: ext Martyn Russell [mar...@lanedo.com]
> Sent: 08 March 2011 13:53
>
>> As you can imagine, if you want to be able to query, insert and merge
>> 120k of contracts, you really can't have your cake and eat it so to
>> speak without some sacrifices one way or another.
>
> That's not true.
>
> Nothing prevents from having 2 DB, where the first one is the usual (could be 
> considered as cache)
> and the other is much larger and used only in case the first search fails.
> Maybe with the explicit consent from the user to proceed with such
> expensive operation.
>
> Obviously there are problems of keeping the DBs in synch, but they do not
> seem impossible to solve, for example deferring the work to a later time when
> the device is idle and connected to a power source.

I thought tracker already supported indexing a e-d-s data store anyway.

Peter
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Peter Robinson]

On Tue, Mar 8, 2011 at 1:37 PM, David Woodhouse  wrote:
> On Tue, 2011-03-08 at 12:41 +0200, Vitaly Repin wrote:
>> Good point. It is closed-sourced. This was a business decision. Has any
>> MeeGo guy except the MfE developers ever tried to talk about open-sourcing
>> of the MfE solution?
>>
>> Rhetorical question.
>
> I've been looking at ActiveSync support recently, so I can answer it
> even if it was intended to be rhetorical.
>
> To be honest, I've found the existing MfE solution on Maemo to be so
> horridly broken and unreliable that I didn't even explore the option of
> using it. It loses sync for some reason almost every week, and gives me
> *no* sensible error messages about what went wrong.
>
> (I appreciate that coherent error handling is out of fashion in MeeGo,
> and baffling the users with silent failure seems to be the norm now, but
> I'm old-fashioned and I prefer to buck the trend here.)
>
> The solution we're looking at for ActiveSync is probably going be a
> dæmon which implements the protocol itself, can run in the background
> and handle 'push' from the server, etc. The dæmon would be under a BSD
> or Apache licence.
>
> Then we have plugins for SyncEvolution and $MAILER which talk to that
> dæmon over DBus.
>
> This way, the plugins for sync and mail frameworks can be properly
> integrated with the code that they plug in to; it's only the ActiveSync
> dæmon which has special licensing conditions in some parts of the world.

No idea if its suitable but evolution-mapi at least works with
exchange not sure of the difference between mapi and active sync
support.

Peter
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Peter Robinson]

On Tue, Mar 8, 2011 at 11:20 AM, Felipe Contreras
 wrote:
> On Mon, Mar 7, 2011 at 6:09 PM, Arjan van de Ven  
> wrote:
>> PIM storage
>> ===
>> The Address book, Calendar data and Email are currently stored in a tracker
>> database, and accessed (officially) via a QtMobility API set.
>> There are a range of issues with this implementation, starting with the
>> complexity of adding privacy controls, the performance and
>> scalability as well as the completeness for doing a proper syncml sync.
>>
>> Because of all these items and the available expertise, we have decided to
>> start replacing PIM storage with the Evolution Data Server.
>> This change will land together with the SyncEvolution change (due to the
>> intimate relationship between the storage and sync of PIM data).
>> This change should largely be invisible to applications since applications
>> are supposed to access this data via the appropriate QtMobility
>> APIs. But to avoid setting precedents, the lower level components will be
>> removed from the architecture diagrams effective immediately.
>>
>> To be clear, this does not mean that "tracker" is completely removed;
>> tracker is still being used (together with tumbler) for indexing media
>> on the device. At this point we are seeing serious issues
>> (performance/stability) with this solution, but the first attempt will be to
>> fix the
>> deficiencies rather than a replacement.
>
> Isn't libfolks superseding e-d-s?

No, its on top of e-d-s. Its similar to anerley (not sure if its used
outside of the netbook spin).

Peter
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Tue, 2011-03-08 at 09:53 -0700, Clark, Joel wrote:

Hey Clark Joel,

> > There are real-world scenarios (think mirroring sensitive enterprise or
> > social web data) were an additional address book is needed, because data
> > must not be mixed with the "normal" contacts.
> 
> And systems that have multiple user profiles 

Preferably you use a Tracker per UNIX user. But also here can graphs be
used if necessary. I think EDS is similar in this regard.

We don't support system-wide Tracker instances; we lack a use-case for
that (and it sounds like a silly idea to us too).

> > You mentioned plans to add access control. Can that be implemented while
> retaining the direct read capability and the 
> 
> Also essential for systems with multiple user profiles and concurrent 
> multi-user access.

So as mentioned in earlier reply to Ohly Patrick, is user access to
meta.db regulated using UNIX file permissions using the
GID::metadata-users credential (which grants your process group-id
permissions).

When this read access isn't available then libtracker-sparql falls back
to FD passing over D-Bus automatically.

Unfortunately needs WAL also writable access to meta.db's directory (for
the journal files that it writes) but said GID:: credential should only
be granted to certified applications (if such security is of importance
for the integrator).


Cheers,

Philip

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Tue, 2011-03-08 at 08:43 -0800, Patrick Ohly wrote:
> On Di, 2011-03-08 at 10:31 +, Philip Van Hoof wrote:
> > Can somebody from Intel answer One of the questions about EDS?
> 
> All you are asking for is performance. To me, that wasn't the main issue
> with Tracker. The key drawbacks of Tracker as storage for contacts are
> these:
>  1. only one address book

You can use either a graph or things like a nie:DataSource and the
property nie:dataSource to have multiple address books.

>  2. no support for controlling access

This is unfortunately as far as I know no different in EDS.

Tracker's plans for per-resource access control is to use its (limited)
support for graphs for that.

Access to metadata is then granted per graph. (not implemented, and I'm
not trying to make it sound that this is easy)

Combined with 1. that would mean that you could grant access to one
address book to application A, but not to application B, although B has
access to another address book.

So for example

INSERT { GRAPH  {  a nco:Contact ... } }

Application X would get access to urn:protected-xxx in some way. It does
a query for all nco:Contact, and it gets .

Application Y would not get access to this. It does a query for all
nco:Contact, and it doesn't get .

And explained lower: Application Y gets no read access to meta.db, so it
must use tracker-store who'd regulate this access. Nor can Application Y
use sqlite3_open on its own, as it has no read access other than via
tracker-store (which uses FD passing D-Bus as IPC).

> There are real-world scenarios (think mirroring sensitive enterprise or
> social web data) were an additional address book is needed, because data
> must not be mixed with the "normal" contacts.

Sure, ok

> You mentioned plans to add access control. Can that be implemented while
> retaining the direct read capability and the performance improvement
> that you mentioned when announcing that feature? Doesn't look possible
> to me.

I have to be careful now what I write because only after Aegis became
public were NDA things lifted a bit on this stuff ... (not our fault)

What I reply is available in gitorious publicly anyway, so hopefully
it's all fine ;-)

Check the files tracker.postinst and tracker.aegis, *.aegis:
http://meego.gitorious.org/tracker/tracker/trees/harmattan/debian

With Aegis we give certified applications read access to meta.db using
the .

The directory $HOME/.cache/tracker is protected to be readable only by
processes in that got this GID granted through Aegis. The library
libtracker-sparql, which enables SPARQL query capability, detects
whether or not the process has access and selects either D-Bus (over fd
passing) or direct-access to meta.db based on the UNIX file permissions.

It is true that once a process has permission to read meta.db, that you
can't hide it the data that is in meta.db. We had no plans to do in-db
encryption as this would just make things horribly slow (decrypting
cells in custom SQLite functions, no thanks).

On Harmattan a plan was to only give certified applications the right to
add the GID::metadata-users credential. Meaning that the source code is
known to the vendor, and known not to abuse meta.db.

We investigated whether storing meta.db on a AegisFS was an option. But
because AegisFS uses FUSE, it is notoriously slow on each syscall.

Would meta.db be stored on a AegisFS, then the data would also be
protected using encryption (to survive reboot to root-mode).

> The other question of course is about the schedule. Access control is
> needed for products *now*, not in some distant future. EDS doesn't have
> access control either, but I consider it much easier to add (already
> based on a daemon concept, much more limited scope).

The same "daemon concept" applies to Tracker's tracker-store. As access
to meta.db is regulated through Aegis's GID:: credential.


Cheers,

Philip

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Robin Burchell]

On 08/03/11 16:53, Clark, Joel wrote:

From: Patrick Ohly



There are real-world scenarios (think mirroring sensitive enterprise or
social web data) were an additional address book is needed, because data
must not be mixed with the "normal" contacts.


And systems that have multiple user profiles


Then hopefully these multiple user profiles would have separate home 
directories, which would each have a per-user tracker database, like on 
any other system tracker is used on.


Best regards,

--
Robin Burchell
http://rburchell.com
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Clark, Joel]

> From: Patrick Ohly

> There are real-world scenarios (think mirroring sensitive enterprise or
> social web data) were an additional address book is needed, because data
> must not be mixed with the "normal" contacts.

And systems that have multiple user profiles 

> You mentioned plans to add access control. Can that be implemented while
retaining the direct read capability and the 

Also essential for systems with multiple user profiles and concurrent 
multi-user access.

> Best Regards, Patrick Ohly

regards, Joel
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Patrick Ohly]

On Di, 2011-03-08 at 10:31 +, Philip Van Hoof wrote:
> Can somebody from Intel answer One of the questions about EDS?

All you are asking for is performance. To me, that wasn't the main issue
with Tracker. The key drawbacks of Tracker as storage for contacts are
these:
 1. only one address book
 2. no support for controlling access

There are real-world scenarios (think mirroring sensitive enterprise or
social web data) were an additional address book is needed, because data
must not be mixed with the "normal" contacts.

You mentioned plans to add access control. Can that be implemented while
retaining the direct read capability and the performance improvement
that you mentioned when announcing that feature? Doesn't look possible
to me.

The other question of course is about the schedule. Access control is
needed for products *now*, not in some distant future. EDS doesn't have
access control either, but I consider it much easier to add (already
based on a daemon concept, much more limited scope).

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.


___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Tue, 2011-03-08 at 15:23 +, Graham Cobb wrote:
> On Tuesday 08 March 2011 12:55:45 Philip Van Hoof wrote:
> > But I have the feeling that we won't see *any* numbers whatsoever.

Hi Graham, thanks for your reply

> I realise the question was directed at the MeeGo architects but I would hope 
> that the Nokia product managers for the PIM/tracker developments had 
> quantitative requirements for performance and scalability.  What were they? 
> Did you meet them?

Yes, there are such requirements. I can't make them available publicly
myself unfortunately.

> If I had been the product manager I would have tried to get requirements in 
> three ways:
> 
> 1) Put an engineer for a couple of days on testing market leading enterprise 
> devices (hint, Blackberry).  Determine any practical scalability limits 
> (numbers of events, contacts, emails, etc) as well as performance with large 
> databases (update, lookup, etc).

It's similar.

> 2) Ask my own (Nokia) sales people how they actually use their handheld 
> devices: how many events, contacts, emails, etc they have on their device?  
> How many different folders they have them divided into?  How and why do they 
> look people up? Do they use a card scanning app on their phone to import 
> business cards?  Do they consider the phone or Exchange to be the master? 
> What 
> would happen if you stole their phone now and refused to give it back?  Etc.  
> I am sure the answers would be very different to the way the engineers or 
> even 
> product managers use their devices.

sure

> 3) Ask the network operators what their requirements are: many of them have 
> pretty clear requirements.

Yes, such things take place. Of course.

> Nokia used to be the best at understanding and interpreting market 
> requirements.  In the old days, that was not only for the consumer segment 
> but 
> even for business (there is a reason the 6310i is still available on eBay 
> today!).  

> Are there really no quantitative requirements you can tell us about, Philip?

I'm asking for numbers and measurements on EDS in comparison with
Tracker. Not only requirements (other people are asking for our
requirements, and some requirements can probably be made public -- sure,
just not up to me)

- The word was that Tracker doesn't perform well enough and therefore it
is being replaced by EDS. I'm guessing that means that EDS *does*
perform well on identical datasets then?

  Are there numbers on that conclusion? We are into numbers and
technical arguments. Are the methods to reproduce available?

  We have some performance tests in Tracker's repo publicly available,
by the way.

  For example (just one example, I don't feel like flooding this ML):
http://git.gnome.org/browse/tracker/tree/tests/functional-tests/performance-tc.py

  That test also illustrates some of the levels of query flexibility and
domain scalability we need for certain applications. How does EDS
compare to those?

- The word also was that EDS is more scalable. What does scalable mean?
Does it scale over more application domains and use-cases than Tracker?
How does that work? Does EDS scale better at query flexibility (can you
do more flexible, more powerful queries with EDS)? Does it scale better
with large amounts of data? And also in combination with aforementioned
query flexibility?

- The word also was that EDS must have privacy controls, since for the
lack of having privacy controls is Tracker being replaced with EDS.
Where is that feature documented? How does that work?

Arjan gave legitimate reasons for replacing a technology. But How does
the replacement improve things?

I think those are reasonable questions to ask given the impact of the
decision. I was witness of the Fremantle -> Harmattan days, and what I'm
seeing here feels like the exact same mistakes Nokia made being repeated
'in exactly the same way', by Intel.


Cheers,

Philip

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[David Woodhouse]

On Tue, 2011-03-08 at 16:23 +0200, Vitaly Repin wrote:
> - It does not work due to the fact that (sometimes) google responds to
>   our queries in the way which is not specified by the EAS protocol 
>   specs.
> - When Google was contacted, they rejected to answer any technical
>   questions of ours. My personal (!) opinion on the matter is that they
>   don't want to promote EAS, they want to promote the google data API.

Well the Google data API is *standard* protocols, isn't it?

So *I* want to promote the "google data API" too.

In fact, I was very disappointed that the N900 didn't ship with standard
sync protocols supported "out of the box". I played with SyncEvolution
briefly for Google sync, but there was too much work to be done on the
UI side and the automatically-trigger-a-sync-when-we-get-a-network side
and a bunch of other things that were already *done* for MfE but for
some reason even *those* parts didn't seem to be open.

(Or perhaps I just didn't look in the right places; I concede I didn't
look *that* hard. I do have a day job...)

> >  - Not open source so it can't be debugged and fixed by those for whom
> >it *would* be a priority.
> 
> This is a request to legal and business people. I totally agree with
> this point.
 ...
> Could you or guys from Intel / Linux Foundation try to push the open-sourcing 
> of MfE? To
> bring this question on board? To make any kind of noise about it?

Is it really going to make any difference? Especially given that we have
internal deadlines which are *very* tight, for getting this working.

At this point it seems like it'll *so* much be quicker to reimplement
it, than it would be even to get a coherent discussion started with the
right folks in Nokia.

I'm painfully aware that even at the best of times, and even when there
haven't been legal hurdles to cope with, Nokia have been *very* slow to
open things up.

> > That sounds interesting; I'd be very interested to see how much we can
> > re-use. I assume that you'll never get anywhere with your lawyers on the
> > topic of open sourcing it? 
> 
> If we don't have support from outside - for sure. They need to see the
> business needs. Practically, the demand.

Which part isn't obvious? That we need ActiveSync support, to
communicate with legacy crap systems that can't talk standard protocols?

Or that a closed source implementation is a complete non-starter, since
it can't be debugged and well-integrated into the system?

> > But perhaps we could design the DBus API to
> > be similar to the one you're already using, and make the transition a
> > whole lot easier? Being able to use existing plugins to QMF and
> > libcamel, perhaps with some modification, would be *really* useful — is
> > there any better prospect of opening *those*?
> 
> Interesting point. I'll bring it on board internally. Sounds reasonable.

Can we focus on the bit about the DBus API first? There's no *licensing*
issue there, right? You could share that without having to consult the
lawyers? Or just implement DBus introspection in the next release... :)

> But after Feb 11 I personally  started to be really pessimistic about 
> open-sourcing
> of anything made by Nokia.

Pessimism is my base state. It's one of the reasons I *love* being
proven wrong occasionally :)

I've love to be proven wrong in my assumption that Nokia will never
manage to pull their finger out and fix the MfE licensing in a
reasonable amount of time. By all means, please put me in touch with the
right people and we'll see if we can make it happen.

-- 
dwmw2

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Graham Cobb]

On Tuesday 08 March 2011 12:55:45 Philip Van Hoof wrote:
> But I have the feeling that we won't see *any* numbers whatsoever.

I realise the question was directed at the MeeGo architects but I would hope 
that the Nokia product managers for the PIM/tracker developments had 
quantitative requirements for performance and scalability.  What were they? 
Did you meet them?

If I had been the product manager I would have tried to get requirements in 
three ways:

1) Put an engineer for a couple of days on testing market leading enterprise 
devices (hint, Blackberry).  Determine any practical scalability limits 
(numbers of events, contacts, emails, etc) as well as performance with large 
databases (update, lookup, etc).

2) Ask my own (Nokia) sales people how they actually use their handheld 
devices: how many events, contacts, emails, etc they have on their device?  
How many different folders they have them divided into?  How and why do they 
look people up? Do they use a card scanning app on their phone to import 
business cards?  Do they consider the phone or Exchange to be the master? What 
would happen if you stole their phone now and refused to give it back?  Etc.  
I am sure the answers would be very different to the way the engineers or even 
product managers use their devices.

3) Ask the network operators what their requirements are: many of them have 
pretty clear requirements.

Nokia used to be the best at understanding and interpreting market 
requirements.  In the old days, that was not only for the consumer segment but 
even for business (there is a reason the 6310i is still available on eBay 
today!).  

Are there really no quantitative requirements you can tell us about, Philip?

Graham
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Graham Cobb]

On Tuesday 08 March 2011 13:44:44 Vitaly Repin wrote:
> On Tue, 08 Mar 2011, ext David Woodhouse wrote:
> > To be honest, I've found the existing MfE solution on Maemo to be so
> > horridly broken and unreliable that I didn't even explore the option of
> > using it. It loses sync for some reason almost every week, and gives me
> > *no* sensible error messages about what went wrong.
> 
> What solution have you tried? Do you refer to N900 or something else
> (MeeGo y.x)?

Much as I hate MfE for being closed source, the losing sync problem (with no 
error messages) may not be MfE's fault.  I have a personal app which uses OWA 
to synchronise my contacts & events (in kontact) with Exchange.  This also 
loses sync about every week and re-downloads everything.  In this case, as it 
is my code, I can see that OWA is not giving any errors -- it is just telling 
me that everything has changed since my last sync and everything has to be 
downloaded again.  That is not true but the problem, whatever it is, is at 
Microsoft's end.

Graham
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Graham Cobb]

On Tuesday 08 March 2011 13:37:15 David Woodhouse wrote:
> He'll be talking about the GAL, which in our case is an 18MiB download.
> I think our "personal" contacts folders in Outlook are typically
> relatively small.

I respectfully suggest you are not in Sales.  While 80K seems unusual, most 
sales people in my experience have several thousand and some tens of 
thousands.  And most of them prefer to have their main database in their own 
lists, not in some corporate database (whatever company policy may say: those 
contacts are their "tools of their trade").

PIM sizing requirements are ALWAYS completely underestimated by engineers who, 
in this particular area, are not the power users (or even typical)!

Graham
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Vitaly Repin]

Hello,

On Tue, 08 Mar 2011, ext David Woodhouse wrote:

> N900, using it against Google.

Test it against real Exchange server. May be, you'll reconsider your
opinion about maemo MfE solution quality.

Regarding google - I confirm that it just does not work from end-user
PoV. Google is marked as unsupported, however.

> I know it's "unsupported", but that's a
> chicken-and-egg situation. It's only "unsupported" because it's:

>  - Not working and not a high priority for Nokia.

- It does not work due to the fact that (sometimes) google responds to our 
queries in
  the way which is not specified by the EAS protocol specs.
- When Google was contacted, they rejected to answer any technical
  questions of ours. My personal (!) opinion on the matter is that they
  don't want to promote EAS, they want to promote the google data API.

>  - Not open source so it can't be debugged and fixed by those for whom
>it *would* be a priority.

This is a request to legal and business people. I totally agree with
this point.

> > This is EXACTLY how it is done in MeeGo/Maemo with the only difference -
> > licensing part.

> It's a sensible design. But closed source is a complete showstopper for
> us.

Could you or guys from Intel / Linux Foundation try to push the open-sourcing 
of MfE? To
bring this question on board? To make any kind of noise about it?

> That sounds interesting; I'd be very interested to see how much we can
> re-use. I assume that you'll never get anywhere with your lawyers on the
> topic of open sourcing it? 

If we don't have support from outside - for sure. They need to see the
business needs. Practically, the demand.

> But perhaps we could design the DBus API to
> be similar to the one you're already using, and make the transition a
> whole lot easier? Being able to use existing plugins to QMF and
> libcamel, perhaps with some modification, would be *really* useful — is
> there any better prospect of opening *those*?

Interesting point. I'll bring it on board internally. Sounds reasonable.
But after Feb 11 I personally  started to be really pessimistic about 
open-sourcing
of anything made by Nokia.

> I'd love to work with your team on this.

The same.
-- 
WBR & WBW,
Vitaly Repin,
Productivity, Nokia-D
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[David Woodhouse]

On Tue, 2011-03-08 at 15:44 +0200, Vitaly Repin wrote:
> Hello,
> 
> On Tue, 08 Mar 2011, ext David Woodhouse wrote:
> 
> > I've been looking at ActiveSync support recently, so I can answer it
> > even if it was intended to be rhetorical.
> 
> Thanks a lot!
> 
> > To be honest, I've found the existing MfE solution on Maemo to be so
> > horridly broken and unreliable that I didn't even explore the option of
> > using it. It loses sync for some reason almost every week, and gives me
> > *no* sensible error messages about what went wrong.
> 
> What solution have you tried? Do you refer to N900 or something else
> (MeeGo y.x)?

N900, using it against Google. I know it's "unsupported", but that's a
chicken-and-egg situation. It's only "unsupported" because it's:

 - Not working and not a high priority for Nokia.
 - Not open source so it can't be debugged and fixed by those for whom
   it *would* be a priority.

I see *those* two things as problems, and "Google is unsupported" merely
as a symptom.

> > The solution we're looking at for ActiveSync is probably going be a
> > dæmon which implements the protocol itself, can run in the background
> > and handle 'push' from the server, etc. The dæmon would be under a BSD
> > or Apache licence.
> 
> This is EXACTLY how it is done in MeeGo/Maemo with the only difference -
> licensing part.

It's a sensible design. But closed source is a complete showstopper for
us.

> > Then we have plugins for SyncEvolution and $MAILER which talk to that
> > dæmon over DBus.
> 
> Now we have the plugins to QMF which talk to daemon over D-BUS. 
> In N900 times we had the plugin to modest and libcamel provider to talk
> with the daemon.

That sounds interesting; I'd be very interested to see how much we can
re-use. I assume that you'll never get anywhere with your lawyers on the
topic of open sourcing it? But perhaps we could design the DBus API to
be similar to the one you're already using, and make the transition a
whole lot easier? Being able to use existing plugins to QMF and
libcamel, perhaps with some modification, would be *really* useful — is
there any better prospect of opening *those*?

I'd love to work with your team on this.

-- 
dwmw2

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Vitaly Repin]

Hello,

On Tue, 08 Mar 2011, ext David Woodhouse wrote:

> I've been looking at ActiveSync support recently, so I can answer it
> even if it was intended to be rhetorical.

Thanks a lot!

> To be honest, I've found the existing MfE solution on Maemo to be so
> horridly broken and unreliable that I didn't even explore the option of
> using it. It loses sync for some reason almost every week, and gives me
> *no* sensible error messages about what went wrong.

What solution have you tried? Do you refer to N900 or something else
(MeeGo y.x)?

> The solution we're looking at for ActiveSync is probably going be a
> dæmon which implements the protocol itself, can run in the background
> and handle 'push' from the server, etc. The dæmon would be under a BSD
> or Apache licence.

This is EXACTLY how it is done in MeeGo/Maemo with the only difference -
licensing part.

> Then we have plugins for SyncEvolution and $MAILER which talk to that
> dæmon over DBus.

Now we have the plugins to QMF which talk to daemon over D-BUS. 
In N900 times we had the plugin to modest and libcamel provider to talk
with the daemon.

-- 
WBR & WBW,
Vitaly Repin,
Productivity, Nokia-D
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Marius Vollmer]

ext Philip Van Hoof  writes:

>>> Nothing prevents from having 2 DB, [...]
>>
>> Also, we've had > 1 DB before in Tracker [...]
>
> Some technical chitchat on this:

I think Igor is just proposing that we do the indexing lazily (not
counting it towards the import operation itself), and somehow notify the
user that indexing is still ongoing or give him/her the option to do a
simple grep on the non-indexed part.

Anyone up for writing a SPARQL query engine that swallows TTL files?  We
could then do batch imports by giving TTL files to Tracker, index them
lazily, and while there still is a non-indexed TTL file around, we run
the slow SPARQL TTL query engine on them.

Does that make any sense?
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[David Woodhouse]

On Tue, 2011-03-08 at 12:41 +0200, Vitaly Repin wrote:
> Good point. It is closed-sourced. This was a business decision. Has any
> MeeGo guy except the MfE developers ever tried to talk about open-sourcing
> of the MfE solution?
> 
> Rhetorical question.

I've been looking at ActiveSync support recently, so I can answer it
even if it was intended to be rhetorical.

To be honest, I've found the existing MfE solution on Maemo to be so
horridly broken and unreliable that I didn't even explore the option of
using it. It loses sync for some reason almost every week, and gives me
*no* sensible error messages about what went wrong.

(I appreciate that coherent error handling is out of fashion in MeeGo,
and baffling the users with silent failure seems to be the norm now, but
I'm old-fashioned and I prefer to buck the trend here.)

The solution we're looking at for ActiveSync is probably going be a
dæmon which implements the protocol itself, can run in the background
and handle 'push' from the server, etc. The dæmon would be under a BSD
or Apache licence.

Then we have plugins for SyncEvolution and $MAILER which talk to that
dæmon over DBus.

This way, the plugins for sync and mail frameworks can be properly
integrated with the code that they plug in to; it's only the ActiveSync
dæmon which has special licensing conditions in some parts of the world.

It also means that we are very agnostic about *what* we have to plug in
to. SyncEvolution was a simple enough choice because that can be used
from within Buteo too. For the mail side, we have just been saying
"$MAILER" so far. But I'm very happy for that to be EDS.

> > Oh and my exchange contact database contains between 80k and 120k 
> > people. that'll take a while.
> 
> Just to clarify - do you refer to your "Contacts" folder or to the
> complete Global Address List of your company which is accessible
> through Exchange typically?  They are different. 

He'll be talking about the GAL, which in our case is an 18MiB download.
I think our "personal" contacts folders in Outlook are typically
relatively small.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Igor.Stoppa]

Hi,

From: ext Philip Van Hoof [phi...@codeminded.be]
Sent: 08 March 2011 14:55

[snip]

> Some technical chitchat on this:

[snip]

sorry, I couldn't decide what to leave

I believe I'm taking a _very_ different angle

My proposal was:

step 1:

refer to the normal DB, as it is now

step 2:

should that fail, use the huge one


The integrity/synch problem should be easy to address:

* If the data in the first DB is more recent, it will be found 
  during 1)
* whenever the large DB is re-loaded, the smaller one can
 be refreshed (the time should be anyway acceptable _and_ 
 it can be done when the device is unused, so it doesn't 
 have to be lightning fast)


Sorry if I'm blindly ignoring all the discussion about UNION etc,
but I have the feeling that it is based on some requirement that
has not been formulated addressing what are acceptable
compromises from overall usability perspective.

I do not know if there are other reasons why what I describe cannot be
realistically implemented, but I would be quite happy if I could have
something like that as an option.

After all when I consult the physical white/yellow pages I do accept that
they are not always up-to-date.

cheers, igor
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Andre Klapper]

On Mon, 2011-03-07 at 14:56 -0800, Arjan van de Ven wrote:
> I'm sorry that your technology did not win. Actually, I'll take that 
> back. I'm not sorry. Your technology did not win
> this round. It won the last round and then NOTHING good got put into 
> MeeGo.

No need for malice towards developers of tracker because of decisions
pushed by others (managers of Nokia).
Wrong audience for expressing disappointment, I'd say.

andre
-- 
Andre Klapper (maemo.org bugmaster)
http://www.openismus.com

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Tue, 2011-03-08 at 12:12 +, Martyn Russell wrote:
> On 08/03/11 11:58, igor.sto...@nokia.com wrote:
> > Hi,
> 
> Hi Igor,
> 
> > From: ext Martyn Russell [mar...@lanedo.com]
> > Sent: 08 March 2011 13:53
> >
> >> As you can imagine, if you want to be able to query, insert and merge
> >> 120k of contracts, you really can't have your cake and eat it so to
> >> speak without some sacrifices one way or another.
> >
> > That's not true.
> >
> > Nothing prevents from having 2 DB, where the first one is the usual
> > (could be considered as cache) and the other is much larger and used
> > only in case the first search fails. Maybe with the explicit consent
> > from the user to proceed with such expensive operation.

[cut]

> Also, we've had > 1 DB before in Tracker and it was a nightmare. There 
> are problems with joins, security, speed, etc. and we've (as a team) 
> discussed this so much already. We have also done testing with n 
> databases vs 1 database to make sure that our approach makes the most 
> sense with SQLite as it also depends on which database you use.

Some technical chitchat on this:

One of the big problems with 'two or more databases' for one class of
data (in a decomposed schema, where each class gets its own table) is
that each (sub)query done on the tables must become a UNION with the two
tables.

UNION in SQLite internally means ~ executing two queries; twice as
slow. Mind that SQLite does relatively few magic behind the scenes. As a
DB engine it's usually / relatively straightforward.

And in the end is or should splitting up a table yield similar results
as adding an index on a single table, at least for querying. So just add
an index?

Having a lot of tables with some of the tables having a lot of rows
should not slow down queries on the smaller tables in a sqlite's .db. So
for that there is also no real reason to have multiple .db files. If
SQLite does get slowed down (a lot) because of that, then that sounds to
me like a (serious) bug in SQLite.

As for read access without IPC, and the claim that with multiple .db
files you can avoid some locks when multiple processes access the .db
file, can you open SQLite databases in WAL journaling mode and avoid
most of any such lock altogether (for reading). It's not yet true MVCC
but for just reading it's quite useful that way.

For updating this doesn't work, so we have fd-passing over DBus to
tracker-store for updating (for that reason).

Also note that in SQLite a transaction locks each .db that is ATTACHed
to the connection where the transaction started.

ps. I'm not saying that SQLite is the best DBMS on all architectures and
platforms. Especially not when you have a Gig of RAM (just a few more
years and phone-sized devices have that, you'll see). Over the years we
learned to deal with what it is. It has quite a few problems idd.

Most of those popular DBMS's do use too much RAM, though. And if I can
use gigabytes of RAM and waste massive amounts of I/O and battery then
no, it's not difficult to make things perform well. Also if I can choose
the storage hardware, would help a lot. Etc.

For all the numbers that have (not) been posted here: I wonder about all
those. What RAM usage? What storage hardware? What amount of data? What
kind of data? How flexible can it be queried afterward? And how fast is
that query post data entry?

But I have the feeling that we won't see *any* numbers whatsoever.


Cheers,

Philip

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Martyn Russell]

On 08/03/11 11:58, igor.sto...@nokia.com wrote:

Hi,


Hi Igor,


From: ext Martyn Russell [mar...@lanedo.com]
Sent: 08 March 2011 13:53


As you can imagine, if you want to be able to query, insert and merge
120k of contracts, you really can't have your cake and eat it so to
speak without some sacrifices one way or another.


That's not true.

Nothing prevents from having 2 DB, where the first one is the usual (could be 
considered as cache)
and the other is much larger and used only in case the first search fails.
Maybe with the explicit consent from the user to proceed with such
expensive operation.


We have also considered this or "views" and it is something we've got in 
mind to investigate more thoroughly, but NOT as separate databases).


But as you say, you then have a sync issue to deal with and this syncing 
is EXACTLY the problem application developers report to us and don't 
want to see. For example, you create an image and want the image to be 
in the database instantly so you can query on it, but you're still 
writing the file to disk at the time. The latencies and race conditions 
here are nasty for application developers and your double DB solution 
really doesn't help there.


Also, we've had > 1 DB before in Tracker and it was a nightmare. There 
are problems with joins, security, speed, etc. and we've (as a team) 
discussed this so much already. We have also done testing with n 
databases vs 1 database to make sure that our approach makes the most 
sense with SQLite as it also depends on which database you use.


--
Regards,
Martyn
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Igor.Stoppa]

Hi,

From: ext Martyn Russell [mar...@lanedo.com]
Sent: 08 March 2011 13:53

> As you can imagine, if you want to be able to query, insert and merge
> 120k of contracts, you really can't have your cake and eat it so to
> speak without some sacrifices one way or another.

That's not true.

Nothing prevents from having 2 DB, where the first one is the usual (could be 
considered as cache)
and the other is much larger and used only in case the first search fails.
Maybe with the explicit consent from the user to proceed with such
expensive operation.

Obviously there are problems of keeping the DBs in synch, but they do not
seem impossible to solve, for example deferring the work to a later time when 
the device is idle and connected to a power source.


cheers, igor
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Martyn Russell]

On 08/03/11 10:37, igor.sto...@nokia.com wrote:

Hi,

From: meego-dev-boun...@meego.com [meego-dev-boun...@meego.com] on
behalf of ext Clark, Joel [joel.cl...@intel.com] Sent: 08 March 2011
12:29


Why would you think MeeGo only runs on a phone?


And even if it was a phone, considering that nowadays there are tens
or hundreds of GB of storage, as business user I'd rather play it
safe and have the whole directory stored locally.

Typically one ends up in a foreign country with roaming problems,
barely compatible/reliable phone connection, no WiFi and need to
call/SMS someone.

All of this without taking into account roaming costs that might
actually be prohibitive for some.


Right, but disk space is not the issue here. As said before, querying is 
what we optimise for primarily, if you want to insert or merge 120k 
contacts, that's ok, but don't expect it to be super fast.


Before we had SPARQL, we had equivalent speed for updating and querying 
and the speed for querying was not good enough. So we focused on 
querying because that's the primary use case for every target that 
Tracker is used on. The frequency of syncing or updating data in general 
is much less common than querying it.


To make querying faster, it is a careful balance between data redundancy 
in tables (to avoid joining with other tables) and indexes on columns 
(to make lookups faster and avoiding full table scans). Adding this data 
and these indexes increases disk space and indexing times but increases 
query times as a direct result.


As you can imagine, if you want to be able to query, insert and merge 
120k of contracts, you really can't have your cake and eat it so to 
speak without some sacrifices one way or another.


As Philip says, there are still some areas to improve on and we're 
working on that.


--
Regards,
Martyn
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Riku Voipio]

On 03/08/2011 12:29 PM, ext Clark, Joel wrote:
>> From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On 
>> Behalf Of Martyn Russell
>> My first thought here is, why are you trying to save your entire exchange 
>> contact 
>> database onto your phone? 

> Why would you think MeeGo only runs on a phone?

Quite offtopic since I have no knowledge of tracker/e-d-s internals but:

Even on desktop people using outlook don't synchronize the entire
corporate directory to a local database. You have "personal contacts"
synchronized and the corporate directory is used via online LDAP
searches. I would expect phones, tablets, etc to work the same way.

Arjans 80K users in phonebook is still challenge that the PIM database
should be able to handle gracefully (if not necessarily fast). Even if
it makes the phone book UI inconvenient to use, the UI should not croak
under the big dataset either.
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Felipe Contreras]

On Mon, Mar 7, 2011 at 6:09 PM, Arjan van de Ven  wrote:
> PIM storage
> ===
> The Address book, Calendar data and Email are currently stored in a tracker
> database, and accessed (officially) via a QtMobility API set.
> There are a range of issues with this implementation, starting with the
> complexity of adding privacy controls, the performance and
> scalability as well as the completeness for doing a proper syncml sync.
>
> Because of all these items and the available expertise, we have decided to
> start replacing PIM storage with the Evolution Data Server.
> This change will land together with the SyncEvolution change (due to the
> intimate relationship between the storage and sync of PIM data).
> This change should largely be invisible to applications since applications
> are supposed to access this data via the appropriate QtMobility
> APIs. But to avoid setting precedents, the lower level components will be
> removed from the architecture diagrams effective immediately.
>
> To be clear, this does not mean that "tracker" is completely removed;
> tracker is still being used (together with tumbler) for indexing media
> on the device. At this point we are seeing serious issues
> (performance/stability) with this solution, but the first attempt will be to
> fix the
> deficiencies rather than a replacement.

Isn't libfolks superseding e-d-s?

-- 
Felipe Contreras
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Adrien Bustany]

On Tue, 8 Mar 2011 12:31:56 +0200, Vitaly Repin wrote:

Hello,

On Tue, 08 Mar 2011, ext Adrien Bustany wrote:


Synchronization is a solved problem, mind you, our mail for exchange
plugin does a pretty good job at saving contacts in a slow way. 
Using

APIs the right way, it could sync 500 contacts in 80 seconds, as
mentioned above.


I am very sorry but I have to state the following items here:

1) Exchange implementation has taken into use ALL your and Contacts 
team

guidelines/directions to improve the contacts synchronization speed.
2) We are not aware of any "improper" API usage at this point of 
time.

3) I would be really happy to hear the new suggestions from you or
Contacts team to improve the situation. In form of bug reports, 
coming

to my desk in Ruoholahti, emails, phone calls. Anything!

\begin{"take it personally" mode}
- Exchange team was the first team which has shown the problem with
Contacts sync speed. Yes, our guy, not somebody else, have filed
that famous bug about the contacts speed.
- The first response was - don't even expect the contacts speed to be 
better

  than we have now (and it was really slow that days. REALLY.)
- We have spent a lot of energy / time to convince the teams in Nokia
MeeGo to fix
  their stuff, changed the way we used their APIs etc. The end result 
-
  the contacts sync. speed has improved a lot. Not only for us but 
for

  everybody.
- And now we are getting the very "positive" feedback in public
mailing list from
our colleagues just because we are closed-sourced (thanks to the
managers!) and
the public could not check your words about crappy MfE.
Well done! Blame these closed-sources suckers, they can't strike 
back!

\end{"take it personally" mode}


I admit my way of putting it was blunt, and probably not exact. As you
rightfully pointed out, you were the first to raise the speed issue, 
heck,

I think our save benchmark still uses some code of yours!

What I meant to say was: currently, syncing 300 or so contacts from my 
MFE
account takes more than 80 seconds. So something can be improved 
somewhere.
I have been meaning to spend some time on looking into this, but 
haven't
done it so far. Maybe it is just the server that is slow, maybe we do 
things
wrong, maybe you can batch more contacts together... Who knows! And in 
any

case, we'll never be able to store 120k contacts in a reasonable time.

And about the "we're closed source": I hardly see how I could hold your 
team
accountable for this, since this decision most likely belongs to Nokia 
legal.


Cheers

Adrien
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Dave Neary]

Hi,

Arjan van de Ven wrote:
> Time has come and gone for this to be a discussion; this is a decision.

Out of interest, when was the time for this to be a discussion? And
where did that discussion happen?

Thanks,
Dave.

-- 
Email: dne...@maemo.org
Jabber: bo...@jabber.org

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Martyn Russell]

On 08/03/11 10:29, Clark, Joel wrote:

From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On 
Behalf Of Martyn Russell
My first thought here is, why are you trying to save your entire exchange 
contact
database onto your phone?


Why would you think MeeGo only runs on a phone?


It's just an example and you're avoiding the question.

A phone is just a device which (I perceive) to have the least resources 
able to cope with such a requirement.


--
Regards,
Martyn
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Vitaly Repin]

Hello,

On Mon, 07 Mar 2011, ext Arjan van de Ven wrote:
> can you point at the source code in MeeGo where this "mail for exchange" 
> plugin lives?
> (or anywhere for that matter).

Good point. It is closed-sourced. This was a business decision. Has any
MeeGo guy except the MfE developers ever tried to talk about open-sourcing
of the MfE solution?

Rhetorical question.

But I need to accept that your sarcasm is very valid here.

> Oh and my exchange contact database contains between 80k and 120k 
> people. that'll take a while.

Just to clarify - do you refer to your "Contacts" folder or to the
complete Global Address List of your company which is accessible through
Exchange typically?  They are different.

> you seem to be proud of 80 seconds for 500 people I would absolutely 
> not be proud about such
> a abysmal number.

Completely agree. The contacts sync speed should be increased.

-- 
WBR & WBW,
Vitaly Repin,
Productivity, Nokia-D
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Igor.Stoppa]

Hi,

From: meego-dev-boun...@meego.com [meego-dev-boun...@meego.com] on behalf of 
ext Clark, Joel [joel.cl...@intel.com]
Sent: 08 March 2011 12:29

> Why would you think MeeGo only runs on a phone?

And even if it was a phone, considering that nowadays there are tens or 
hundreds of GB of storage,
as business user I'd rather play it safe and have the whole directory stored 
locally.

Typically one ends up in a foreign country with roaming problems, barely 
compatible/reliable phone connection, no WiFi
and need to call/SMS someone.

All of this without taking into account roaming costs that might actually be 
prohibitive for some.

cheers, igor
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Vitaly Repin]

Hello,

On Tue, 08 Mar 2011, ext Adrien Bustany wrote:

> Synchronization is a solved problem, mind you, our mail for exchange
> plugin does a pretty good job at saving contacts in a slow way. Using
> APIs the right way, it could sync 500 contacts in 80 seconds, as
> mentioned above.

I am very sorry but I have to state the following items here:

1) Exchange implementation has taken into use ALL your and Contacts team
guidelines/directions to improve the contacts synchronization speed.
2) We are not aware of any "improper" API usage at this point of time.
3) I would be really happy to hear the new suggestions from you or
Contacts team to improve the situation. In form of bug reports, coming
to my desk in Ruoholahti, emails, phone calls. Anything!

\begin{"take it personally" mode}
- Exchange team was the first team which has shown the problem with
Contacts sync speed. Yes, our guy, not somebody else, have filed
that famous bug about the contacts speed.
- The first response was - don't even expect the contacts speed to be better
  than we have now (and it was really slow that days. REALLY.)
- We have spent a lot of energy / time to convince the teams in Nokia MeeGo to 
fix
  their stuff, changed the way we used their APIs etc. The end result -
  the contacts sync. speed has improved a lot. Not only for us but for
  everybody.
- And now we are getting the very "positive" feedback in public mailing list 
from
our colleagues just because we are closed-sourced (thanks to the managers!) and
the public could not check your words about crappy MfE.
Well done! Blame these closed-sources suckers, they can't strike back!
\end{"take it personally" mode}

-- 
WBR & WBW,
Vitaly Repin,
Productivity, Nokia-D
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

Interesting thread ... (not).

Congratulations for guiding it with insightful and non-emotional
contributions, Arjan.

Can somebody from Intel answer One of the questions about EDS?

On Mon, 2011-03-07 at 22:40 +0100, Philip Van Hoof wrote:

Question one:

> Can you explain us, how does EDS improve privacy control compared to
> Tracker's RDF store? Does EDS have a way to represent different graphs
> (part of Tracker's future plans to do per-resource privacy control)?
> 
>   How in EDS do you say that data (about) x isn't to be made
>   available to queries made by application y? It's the first time
>   that I hear that EDS has such a privacy related capabilities.

Question two:

> Can you also go a bit deeper into how EDS's performance compares to
> Tracker's RDF store for comparable data entry operations? Have you guys
> done measurements on this? Can the results of these measurements and the
> method to reproduce be publicized? Are the queries you used available
> somewhere? Where?

Question three:

> What do you mean with scalability? Scalable over multiple application
> domains other than PIM? Or scalable to more data?

Question four:

> And what kind of data? I wonder how EDS is or can be more scalable
> when the underlying database technology (SQLite, apparently) is identical.
> 
> In Tracker is each class stored in a decomposed table. This means that
> contact and E-mail data are each in separate tables. Although decomposed
> typically makes INSERT a bit slower, SELECT is a lot faster.

Question five:

> How will or does EDS make SELECT equally 'scalable' with equally complex
> queries?


> Also note that the Tracker team optimizes both INSERT and SELECT
> use-cases on a use-case per use-case basis, using its domainindexes and
> indexes.

Question seven:

> Have you contacted the Tracker team about the scalability
> problems (if any)?

Cheers,

Philip


-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Clark, Joel]

> From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On 
> Behalf Of Martyn Russell
> My first thought here is, why are you trying to save your entire exchange 
> contact 
> database onto your phone? 
> -- 
> Regards,
> Martyn

Why would you think MeeGo only runs on a phone?

regards
Joel

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Vitaly Repin]

Hello,

On Tue, 08 Mar 2011, Marius Vollmer wrote:
> > On Mon, 07 Mar 2011, ext Arjan van de Ven wrote:
> >> The Address book, Calendar data and Email are currently stored in a 
> >> tracker database, and accessed (officially) via a QtMobility API set.
> > Just to correct you a little bit - emails are exported to tracker
> > database.
> I like to formulate this as "Tracker indexes emails and stores their
> meta data, similar to media files".

Let me provide the brief explanations how tracker and e-mail are
integrated in Harmattan now (if somebody does not care about Harmattan,
he/she could go directly to the last lines of my mail). I believe that
it will be helpful to clearly understand that E-Mail and Tracker are 
decoupled and completely independent.

To manipulate (add/remove/edit) the data in QMailStore, QMF API is used.
Anybody can call this API.  When this API is used, it not only adds
the data into the QMF database but also exports parts of it (meta data needed
by Tracker) to Tracker through its API. Currently we have a dedicated process
to do it, msgindexer. If you kill this process, the e-mail are not visible
in tracker but e-mail subsystem works perfectly well. Pay attention - I
include MUA into "e-mail subsystem" here as MUA uses QMF API to access mails.
So, tracker and e-mail are decoupled.

Another interesting point - in current MeeGo distribution msgindexer is
not included and there is no integration of E-Mail subsystem and Tracker.

Looks like this bug should be WONTFIXed now: 
https://bugs.meego.com/show_bug.cgi?id=6598
I don't see any point to include the email/tracker integration to MeeGo
just after announcing the decision about serious change in the role of Tracker 
in
MeeGo.
-- 
WBR & WBW,
Vitaly Repin,
Productivity, Nokia-D
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Martyn Russell]

On 07/03/11 22:56, Arjan van de Ven wrote:

On 3/7/2011 2:31 PM, Adrien Bustany wrote:

Synchronization is a solved problem, mind you, our mail for exchange
plugin does a pretty good job at saving contacts in a slow way. Using
APIs the right way, it could sync 500 contacts in 80 seconds, as
mentioned above.


can you point at the source code in MeeGo where this "mail for exchange"
plugin lives?
(or anywhere for that matter).

Oh and my exchange contact database contains between 80k and 120k
people. that'll take a while.


My first thought here is, why are you trying to save your entire 
exchange contact database onto your phone? Surely it makes sense to only 
save contacts from the database you work with or contacted at some stage 
or by email affiliation. Storing information about people you likely 
will never be in contact with doesn't sound like a sane approach.



you seem to be proud of 80 seconds for 500 people I would absolutely
not be proud about such
a abysmal number.


What *wouldn't* be an abysmal number?
Would you be prepared to sacrifice query speed for indexing speed?

NOTE: Tracker is optimised for querying NOT for updating.

--

All in all, after reading this thread, I think Philip's questions are 
quite pertinent. I think Ivan's points are also highly relevant. Just to 
reiterate:


 - How can we fix problems if you don't communicate them to us? We have 
so many communication channels it's silly (IRC, mailing lists, email, etc).


 - What are your targets? We will don't have any numbers other than 
80-120k contacts. What about times, etc.


 - What are you basing your move to EDS on? What targets/numbers?

--
Regards,
Martyn
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Michael Meeks]

On Tue, 2011-03-08 at 08:18 +0100, Mathias Hasselmann wrote:
> Am Montag, den 07.03.2011, 14:56 -0800 schrieb Arjan van de Ven:
> > Oh and my exchange contact database contains between 80k and 120k 
> > people. that'll take a while.
..
>   * Do we really want to fully sync such large datasets? Shall we
> sync only limited datasets, e.g. title and email address?

It seems clear that the ability to search a corporate contacts database
quickly and efficiently is rather a useful one. As I understand it (and
I'm not involved with this decision, but tend to think it is a sensible,
pragmatic one), Tracker necessarily must have all the 120k corporate
contacts in its local RDF store - to then be able to use sparql /
native-sql-faster-shortcut to query them. Perhaps not in memory per-se,
but on some nearby flash / spinning media, and of course synchronised
regularly vs. the authoritative 120k contacts in the corporate database.

For all the limits of the e-d-s searching language (and certainly it is
used for querying only contacts stores), it can interact with and cache
essentially infinite (LDAP, Exchange, Groupwise etc.) remote databases.
That contrasts with tracker's (much more powerful of course), wonderful
RDF magic - which AFAIR requires all the latest data to be local.

Of course, that is only one difference; but to me important. Clearly
tracker can be duct-taped / abstracted to support this in some way -
nothing is impossible with software, with the application of infinite
resources to a given problem.

Or perhaps, as normal I'm just mistaken ...

All the best,

Michael.

-- 
 michael.me...@novell.com  <><, Pseudo Engineer, itinerant idiot


___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Dienstag, den 08.03.2011, 08:25 +0100 schrieb Carsten Munk:
> 2011/3/8 Mathias Hasselmann :
> > Am Dienstag, den 08.03.2011, 08:10 +0100 schrieb Carsten Munk:
> >> 2011/3/8 Mathias Hasselmann :
> >> > Am Montag, den 07.03.2011, 14:56 -0800 schrieb Arjan van de Ven:
> >> >> I'm sorry that your technology did not win. Actually, I'll take that
> >> >> back. I'm not sorry. Your technology did not win
> >> >> this round. It won the last round and then NOTHING good got put into
> >> >> MeeGo. If you really want to try to sell
> >> >> the same thing again, sorry, too little too late. I have a really hard
> >> >> time feeling sympathy for this.
> >> >
> >> > This entire paragraph is childish. Do you really have to resort to such
> >> > infantile behavior to back your point?
> >>
> >> Hi,
> >>
> >> Please adhere to http://wiki.meego.com/Mailing_list_guidelines - let's
> >> not resort to personal attacks or abuse, it's not a suitable or
> >> constructive way of discussing important issues.
> >>
> >> BR
> >> Carsten Munk
> >
> > Carsten,
> >
> > please don't shoot the messenger.
> 
> Same goes for me, don't shoot me for pointing out that we do have
> mailing list guidelines and we need to act accordingly here - all of
> us.

ACK.

Ciao,
Mathias

-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Dienstag, den 08.03.2011, 09:03 +0200 schrieb Marius Vollmer:
> ext Arjan van de Ven  writes:
> 
> > I'm sorry that your technology did not win. Actually, I'll take that
> > back. I'm not sorry. Your technology did not win this round. It won
> > the last round and then NOTHING good got put into MeeGo. If you really
> > want to try to sell the same thing again, sorry, too little too
> > late. I have a really hard time feeling sympathy for this.
> 
> Fair enough, but let's try to not make this a "us vs them" thing.
> 
> Let's give Tracker a fair chance: I know you do, and I also respect your
> decision that you can't ship with it as the PIM backend right now, but I
> still believe the 'Tracker approach' has a lot of good things to offer.
> 
> We just need to make it work well enough, and I am certain we can.

Well said. Thank you.

Little side note at the end: I've spent two years with EDS when working
on Fremantle's contact's application. Therefore I believe to have some
insight into what EDS can achieve and what not.

For simple sync tasks like store and forget, it is almost perfect. Hard
to beat the performance it's trivial architecture there.

If you'd like to do select contacts by filters things become less
optimal: EDS' query language is quite limited. For fast lookup you must
create indexes, which are expensive to create. Actually in Fremantle's
implementation of EDS most time for saving contacts is spent for
updating indexes. Have to admit to not have reliable numbers right now,
but as far as I remember time for updating indexes was in the scale of
80..90%.

If you now also want to aggregate data, e.g. online presence, then EDS
fails flat. You cannot easily remember origin of information. Automatic
unmerge of contacts almost impossible. It's query language is too
limited for doing those aggregations. In Fremantle we ended with keeping
__all__ contacts in memory so that we could update presence information.
Not exactly a scaling solution. 80k contacts surly will kill the EDS
based N900.

Ciao,
Mathias

PS: 
-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Ville M. Vainio]

On Mon, Mar 7, 2011 at 6:09 PM, Arjan van de Ven  wrote:

> Because of all these items and the available expertise, we have decided to
> start replacing PIM storage with the Evolution Data Server.

Can you expand on what this means for email? Will we still see
sqlite-backed QMF as the email solution, or are you switching to some
other existing email solution?
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Carsten Munk]

2011/3/8 Mathias Hasselmann :
> Am Dienstag, den 08.03.2011, 08:10 +0100 schrieb Carsten Munk:
>> 2011/3/8 Mathias Hasselmann :
>> > Am Montag, den 07.03.2011, 14:56 -0800 schrieb Arjan van de Ven:
>> >> I'm sorry that your technology did not win. Actually, I'll take that
>> >> back. I'm not sorry. Your technology did not win
>> >> this round. It won the last round and then NOTHING good got put into
>> >> MeeGo. If you really want to try to sell
>> >> the same thing again, sorry, too little too late. I have a really hard
>> >> time feeling sympathy for this.
>> >
>> > This entire paragraph is childish. Do you really have to resort to such
>> > infantile behavior to back your point?
>>
>> Hi,
>>
>> Please adhere to http://wiki.meego.com/Mailing_list_guidelines - let's
>> not resort to personal attacks or abuse, it's not a suitable or
>> constructive way of discussing important issues.
>>
>> BR
>> Carsten Munk
>
> Carsten,
>
> please don't shoot the messenger.

Same goes for me, don't shoot me for pointing out that we do have
mailing list guidelines and we need to act accordingly here - all of
us.

BR
Carsten Munk

>
> Ciao,
> Mathias
> --
> Mathias Hasselmann 
> http://openismus.com/
>
>
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Dienstag, den 08.03.2011, 08:10 +0100 schrieb Carsten Munk:
> 2011/3/8 Mathias Hasselmann :
> > Am Montag, den 07.03.2011, 14:56 -0800 schrieb Arjan van de Ven:
> >> I'm sorry that your technology did not win. Actually, I'll take that
> >> back. I'm not sorry. Your technology did not win
> >> this round. It won the last round and then NOTHING good got put into
> >> MeeGo. If you really want to try to sell
> >> the same thing again, sorry, too little too late. I have a really hard
> >> time feeling sympathy for this.
> >
> > This entire paragraph is childish. Do you really have to resort to such
> > infantile behavior to back your point?
> 
> Hi,
> 
> Please adhere to http://wiki.meego.com/Mailing_list_guidelines - let's
> not resort to personal attacks or abuse, it's not a suitable or
> constructive way of discussing important issues.
> 
> BR
> Carsten Munk

Carsten, 

please don't shoot the messenger.

Ciao,
Mathias
-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Montag, den 07.03.2011, 14:56 -0800 schrieb Arjan van de Ven:

> Oh and my exchange contact database contains between 80k and 120k 
> people. that'll take a while.

Ok, first share of numbers is there. Now missing:

  * How long do competing systems need to __fully__ store __and__
index them? What time is acceptable? 
  * Do we really want to fully sync such large datasets? Shall we
sync only limited datasets, e.g. title and email address?
  * Must contacts be fully searchable right after sync already, or
is it sufficient to only store them for the moment, make them
available per contact id and index them in background later?

> you seem to be proud of 80 seconds for 500 people I would absolutely 
> not be proud about such
> a abysmal number.

I don't see how you concluded Adrien would be proud of this numbers.
We know ourself that they are not optimal.

Still he did something you didn't do: He provided facts, instead of
anecdotes or pipe dreams: "let's switch change technology and suddenly
everything will become better". Actually Nokia did exactly this
schoolbook mistake several times and we all now see the consequences of
such management practice right now. Actually you whine about those
consequences right now.

Ciao
Mathias
-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Carsten Munk]

2011/3/8 Mathias Hasselmann :
> Am Montag, den 07.03.2011, 14:56 -0800 schrieb Arjan van de Ven:
>> I'm sorry that your technology did not win. Actually, I'll take that
>> back. I'm not sorry. Your technology did not win
>> this round. It won the last round and then NOTHING good got put into
>> MeeGo. If you really want to try to sell
>> the same thing again, sorry, too little too late. I have a really hard
>> time feeling sympathy for this.
>
> This entire paragraph is childish. Do you really have to resort to such
> infantile behavior to back your point?

Hi,

Please adhere to http://wiki.meego.com/Mailing_list_guidelines - let's
not resort to personal attacks or abuse, it's not a suitable or
constructive way of discussing important issues.

BR
Carsten Munk

>
> Ciao,
> Mathias
> --
> Mathias Hasselmann 
> http://openismus.com/
>
> ___
> MeeGo-dev mailing list
> MeeGo-dev@meego.com
> http://lists.meego.com/listinfo/meego-dev
> http://wiki.meego.com/Mailing_list_guidelines
>
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Montag, den 07.03.2011, 14:56 -0800 schrieb Arjan van de Ven:
> I'm sorry that your technology did not win. Actually, I'll take that 
> back. I'm not sorry. Your technology did not win
> this round. It won the last round and then NOTHING good got put into 
> MeeGo. If you really want to try to sell
> the same thing again, sorry, too little too late. I have a really hard 
> time feeling sympathy for this.

This entire paragraph is childish. Do you really have to resort to such
infantile behavior to back your point?

Ciao,
Mathias
-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Marius Vollmer]

ext Arjan van de Ven  writes:

> I'm sorry that your technology did not win. Actually, I'll take that
> back. I'm not sorry. Your technology did not win this round. It won
> the last round and then NOTHING good got put into MeeGo. If you really
> want to try to sell the same thing again, sorry, too little too
> late. I have a really hard time feeling sympathy for this.

Fair enough, but let's try to not make this a "us vs them" thing.

Let's give Tracker a fair chance: I know you do, and I also respect your
decision that you can't ship with it as the PIM backend right now, but I
still believe the 'Tracker approach' has a lot of good things to offer.

We just need to make it work well enough, and I am certain we can.
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Mathias Hasselmann]

Am Montag, den 07.03.2011, 15:56 -0800 schrieb Arjan van de Ven:
> On 3/7/2011 3:33 PM, zoltan@nokia.com wrote:
> >
> > Let's define what we want, create a Meego-standardized test bench, make
> 
> we can do many things that delay MeeGo moving forward.
> We're not going to do that.

You won't move MeeGo forward by making arbitrary decision out of your
guts. Without having numbers. Without discussion. With spreading food.
With insulting people.

Others tried that.
They failed.

It's an architecture's task to define architecture goals.
Based on research. Based on numbers.

You didn't present any numbers so far.
You didn't define performance goals yes.

Without such information it is impossible to meet your performance
goals. We don't have mind reading machines yet.

> Time has come and gone for this to be a discussion; this is a decision.

The big question is: How could it come that far?
Was this a lack of communication? A lack of leadership?
Even more: What will be done to prevent such problems in the future?
Actually: What makes you sure that EDS will solve this problems?

WHERE ARE THE NUMBERS backing your decision?
Did you roll dices? Or how did your decision process work?

Ciao,
Mathias
-- 
Mathias Hasselmann 
http://openismus.com/

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[zoltan.kis]

> On 3/7/2011 3:33 PM, zoltan@nokia.com wrote:
> >
> > Let's define what we want, create a Meego-standardized test bench,
>
> we can do many things that delay MeeGo moving forward.
> We're not going to do that.

If EDF fails as well a year later because you did not set proper requirements 
and it won't match the problem, then you delay Meego even more. Without tests 
and facts it is just another bet.

> Time has come and gone for this to be a discussion; this is a decision.

True that it has been too long discussion.

I don't know how decisions are made in Meego nowadays, but this has all chances 
to become a poor one (not because EDF).

You can move forward with EDF for the next release(s), but please establish 
proper criteria. I still want to see the test cases and data sets against which 
we will measure EDF or anything else, otherwise we will eternally make the same 
mistakes.

This would also give tracker a chance to improve in the background, against now 
clear requirements.

Regards, Zoltan
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Marius Vollmer]

ext Vitaly Repin  writes:

> Hello,
>
> On Mon, 07 Mar 2011, ext Arjan van de Ven wrote:
>
>> The Address book, Calendar data and Email are currently stored in a 
>> tracker database, and accessed (officially) via a QtMobility API set.
>
> Just to correct you a little bit - emails are exported to tracker
> database.

I like to formulate this as "Tracker indexes emails and stores their
meta data, similar to media files".
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Adrien Bustany]

Le Mon, 07 Mar 2011 14:56:59 -0800,
Arjan van de Ven  a écrit :

> On 3/7/2011 2:31 PM, Adrien Bustany wrote:
> > Synchronization is a solved problem, mind you, our mail for exchange
> > plugin does a pretty good job at saving contacts in a slow way.
> > Using APIs the right way, it could sync 500 contacts in 80 seconds,
> > as mentioned above.
> 
> can you point at the source code in MeeGo where this "mail for
> exchange" plugin lives?
> (or anywhere for that matter).

Aha good point, it's a closed component, and that sucks.

> 
> Oh and my exchange contact database contains between 80k and 120k 
> people. that'll take a while.
> you seem to be proud of 80 seconds for 500 people I would
> absolutely not be proud about such
> a abysmal number.
Hmm, quoting myself "That is not lightning fast" and "this mail is not a
plea for Tracker in MeeGo". So, mind you, even if saving 500 contact
was taking 1ms I wouldn't be proud of it, since I have absolutely no
credit in Tracker's saving performance :) And when technical decisions
are at stake, I tend to put things like "honor", "personal tastes" etc.
away and stick to technical facts. And as you rightly point out, if you
want to store 80k contacts, Tracker most likely does not cut it.

> 
> > message of this thread. To me it sounds like "the Intel architects
> > decided that MeeGo will be that way". I don't exactly how this kind
> > of decision is supposed to be taken, and while I understand you
> > can't ask everybody's opinion for all decisions, I see little
> > transparency there. Hopefully somebody will enlighten me!
> > I also hope in the future MeeGo architects will ask the right people
> > when they have questions on a software piece.
> 
> We'll make sure to ask the people whom we can be sure of that they
> have a commitment
> or charter to support MeeGo so that MeeGo can succeed.
> 
> I am really not interested in discussing hypothetical of what could
> have been. I've had more than my
> share of that kind of discussion in the last year+.
> At this point I'm more interested in getting something to work with 
> technology and people that work
> on MeeGo to make MeeGo succeed.
> 
> I'm sorry that your technology did not win. Actually, I'll take that 
> back. I'm not sorry. Your technology did not win
> this round. It won the last round and then NOTHING good got put into 
> MeeGo. If you really want to try to sell
> the same thing again, sorry, too little too late. I have a really
> hard time feeling sympathy for this.

I must admit I have a hard time understanding this part :) Tracker is
not really "my technology", I happen to use it in my daily work and as
such know what you can do with it and what you can't... I am no Tracker
salesman, nor I do have "Tracker" tatooed on my left shoulder ;)

Again, this message *was not meant to contradict your decision*, since I
have myself no personal grief against EDS, and also tend to look for
what's best *given clear requirements*. As Zoltan said, we never saw
those requirements (we as in we people who investigated time using
Tracker, which is *a* technology, not *our* technology), and I don't
really care about who we should blame, who we should pity etc.

Cheers

Adrien
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Arjan van de Ven]

On 3/7/2011 3:33 PM, zoltan@nokia.com wrote:


Let's define what we want, create a Meego-standardized test bench, make


we can do many things that delay MeeGo moving forward.
We're not going to do that.

Time has come and gone for this to be a discussion; this is a decision.


___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[zoltan.kis]

> Oh and my exchange contact database contains between 80k and 120k people.
that'll take a while...

This extreme example brings up the question: what are the data sets, for
netbook/handset/car-set etc, both for typical average usage and extremes for
the given category? How should we handle exceptions/out of range situations?

And what are the test cases both Tracker and EDF (or any other storage)
should be measured against? Including horizontal use cases of apps (e.g.
building conversation history etc), with constraints on system load, use
time, etc. 

Without setting these first, the data storage story will continue to be a
series of bets, trials and failures.
Now we just have another new-old bet (EDF), without setting what we want.
Not that with Tracker we did :).

Then, Philip's questions from below have not been answered yet.

Let's define what we want, create a Meego-standardized test bench, make
measurements, and decide only then about solutions using
EDF/tracker/sqlite/postgresql/graphDB's/whatever. Otherwise decisions alone
will not take us anywhere because we don't know where we want to get.
Stating that tracker sucks doesn't mean EDF won't. It depends on what you're
after. Perhaps you or someone else know, but then please share :).

Regards,
Zoltan


> -Original Message-
> From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com]
> On Behalf Of ext Philip Van Hoof
> Sent: 07 March, 2011 23:40
> Can you explain us, how does EDS improve privacy control compared to
> Tracker's RDF store? Does EDS have a way to represent different graphs
> (part of Tracker's future plans to do per-resource privacy control)?
> 
>   How in EDS do you say that data (about) x isn't to be made
>   available to queries made by application y? It's the first time
>   that I hear that EDS has such a privacy related capabilities.
> 
> Can you also go a bit deeper into how EDS's performance compares to
> Tracker's RDF store for comparable data entry operations? Have you guys
> done measurements on this? Can the results of these measurements and
> the
> method to reproduce be publicized? Are the queries you used available
> somewhere? Where?
> 
> What do you mean with scalability? Scalable over multiple application
> domains other than PIM? Or scalable to more data? And what kind of
> data?
> I wonder how EDS is or can be more scalable when the underlying
> database
> technology (SQLite, apparently) is identical.
> 
> In Tracker is each class stored in a decomposed table. This means that
> contact and E-mail data are each in separate tables. Although
> decomposed
> typically makes INSERT a bit slower, SELECT is a lot faster. How will
> or
> does EDS make SELECT equally 'scalable' with equally complex queries?
> 
> Also note that the Tracker team optimizes both INSERT and SELECT
> use-cases on a use-case per use-case basis, using its domainindexes and
> indexes. Have you contacted the Tracker team about the scalability
> problems (if any)?
> 
> Or is EDS only more scalable during data entry?
> 
> > This change will land together with the SyncEvolution change (due to
> the
> > intimate relationship between the storage and sync of PIM data).
> > This change should largely be invisible to applications since
> > applications are supposed to access this data via the appropriate
> QtMobility
> > APIs.
> 
> That's not entirely true. Many applications are nowadays accessing said
> data through the QSparql API of Harmattan. QtMobility isn't always the
> ideal access-layer. Especially not for more complex use-cases.
> 


smime.p7s
Description: S/MIME cryptographic signature
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Arjan van de Ven]

On 3/7/2011 2:31 PM, Adrien Bustany wrote:

Synchronization is a solved problem, mind you, our mail for exchange
plugin does a pretty good job at saving contacts in a slow way. Using
APIs the right way, it could sync 500 contacts in 80 seconds, as
mentioned above.


can you point at the source code in MeeGo where this "mail for exchange" 
plugin lives?

(or anywhere for that matter).

Oh and my exchange contact database contains between 80k and 120k 
people. that'll take a while.
you seem to be proud of 80 seconds for 500 people I would absolutely 
not be proud about such

a abysmal number.


message of this thread. To me it sounds like "the Intel architects
decided that MeeGo will be that way". I don't exactly how this kind of
decision is supposed to be taken, and while I understand you can't
ask everybody's opinion for all decisions, I see little transparency
there. Hopefully somebody will enlighten me!
I also hope in the future MeeGo architects will ask the right people
when they have questions on a software piece.


We'll make sure to ask the people whom we can be sure of that they have 
a commitment

or charter to support MeeGo so that MeeGo can succeed.

I am really not interested in discussing hypothetical of what could have 
been. I've had more than my

share of that kind of discussion in the last year+.
At this point I'm more interested in getting something to work with 
technology and people that work

on MeeGo to make MeeGo succeed.

I'm sorry that your technology did not win. Actually, I'll take that 
back. I'm not sorry. Your technology did not win
this round. It won the last round and then NOTHING good got put into 
MeeGo. If you really want to try to sell
the same thing again, sorry, too little too late. I have a really hard 
time feeling sympathy for this.





___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Adrien Bustany]

Le Tue, 8 Mar 2011 00:31:42 +0200,
Adrien Bustany  a écrit :

> Le Mon, 07 Mar 2011 22:40:10 +0100,
> Philip Van Hoof  a écrit :
> 
> > On Mon, 2011-03-07 at 08:09 -0800, Arjan van de Ven wrote:
> > 
> > Hi Arjan,
> > 
> > > PIM storage
> > > ===
> > > The Address book, Calendar data and Email are currently stored in
> > > a tracker database, and accessed (officially) via a QtMobility API
> > > set. There are a range of issues with this implementation,
> > > starting with the complexity of adding privacy controls, the
> > > performance and scalability as well as the completeness for doing
> > > a proper syncml sync.
> > 
> > That's great!
> > 
> > Can you explain us, how does EDS improve privacy control compared to
> > Tracker's RDF store? Does EDS have a way to represent different
> > graphs (part of Tracker's future plans to do per-resource privacy
> > control)?
> > 
> > How in EDS do you say that data (about) x isn't to be made
> > available to queries made by application y? It's the first
> > time that I hear that EDS has such a privacy related capabilities.
> > 
> > Can you also go a bit deeper into how EDS's performance compares to
> > Tracker's RDF store for comparable data entry operations? Have you
> > guys done measurements on this? Can the results of these
> > measurements and the method to reproduce be publicized? Are the
> > queries you used available somewhere? Where?
> > 
> > What do you mean with scalability? Scalable over multiple
> > application domains other than PIM? Or scalable to more data? And
> > what kind of data? I wonder how EDS is or can be more scalable when
> > the underlying database technology (SQLite, apparently) is
> > identical.
> > 
> > In Tracker is each class stored in a decomposed table. This means
> > that contact and E-mail data are each in separate tables. Although
> > decomposed typically makes INSERT a bit slower, SELECT is a lot
> > faster. How will or does EDS make SELECT equally 'scalable' with
> > equally complex queries?
> > 
> > Also note that the Tracker team optimizes both INSERT and SELECT
> > use-cases on a use-case per use-case basis, using its domainindexes
> > and indexes. Have you contacted the Tracker team about the
> > scalability problems (if any)?
> > 
> > Or is EDS only more scalable during data entry?
> > 
> > > Because of all these items and the available expertise, we have
> > > decided to start replacing PIM storage with the Evolution Data
> > > Server.
> > 
> > Ah, I see.
> > 
> > > This change will land together with the SyncEvolution change (due
> > > to the intimate relationship between the storage and sync of PIM
> > > data). This change should largely be invisible to applications
> > > since applications are supposed to access this data via the
> > > appropriate QtMobility APIs.
> > 
> > That's not entirely true. Many applications are nowadays accessing
> > said data through the QSparql API of Harmattan. QtMobility isn't
> > always the ideal access-layer. Especially not for more complex
> > use-cases.
> > 
> > Are you planning to support or implement a QSparql backend for EDS?
> > 
> > I'd be thrilled to have a query language like or as powerful as
> > SPARQL in EDS! In what timeframe can we expect this? Are you
> > planning to use existing SPARQL endpoint technology for this? Which?
> > 
> > > But to avoid setting precedents, the lower level components will 
> > > be removed from the architecture diagrams effective immediately.
> > > 
> > > To be clear, this does not mean that "tracker" is completely
> > > removed; tracker is still being used (together with tumbler) for
> > > indexing media on the device. At this point we are seeing serious
> > > issues (performance/stability) with this solution, but the first
> > > attempt will be to fix the
> > > deficiencies rather than a replacement.
> > 
> > At this moment is Tracker (via QSparql and earlier but deprecated
> > libqttracker) used on Harmattan within quite a wide variety of
> > (horizontal) applications.
> > 
> > Are the plans to only use Tracker for indexing? That's a fairly
> > small use-case of Tracker on Harmattan nowadays. Fremantle's
> > metadata solution of course had an almost exclusive focus on file
> > metadata (indexing). But that's not the case on Harmattan. What's
> > the plan for those apps?
> > 
> > 
> > 
> > Cheers!
> > 
> > Philip
> > 
> 
> Just to complement Philip's message, here's my take on the subject:
> 
> Comparing Tracker and EDS is a bit like comparing a hitch-hiker's
> backpack and a purse. The backpack can definitely store your credit
> card, and the purse too. It is probably easier to put the card in your
> purse than to take off the backpack, open the good pocket etc.
> Now the question is, do you need a purse or a backpack?
> 
> If we go on the technical field, Tracker as a solution for contacts is
> still a solution that is evolving a lot (you might argue that then
> it's not stable/mature, I won't contradict you if you say it's stil

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Adrien Bustany]

Le Mon, 07 Mar 2011 22:40:10 +0100,
Philip Van Hoof  a écrit :

> On Mon, 2011-03-07 at 08:09 -0800, Arjan van de Ven wrote:
> 
> Hi Arjan,
> 
> > PIM storage
> > ===
> > The Address book, Calendar data and Email are currently stored in a 
> > tracker database, and accessed (officially) via a QtMobility API
> > set. There are a range of issues with this implementation, starting
> > with the complexity of adding privacy controls, the performance and
> > scalability as well as the completeness for doing a proper syncml
> > sync.
> 
> That's great!
> 
> Can you explain us, how does EDS improve privacy control compared to
> Tracker's RDF store? Does EDS have a way to represent different graphs
> (part of Tracker's future plans to do per-resource privacy control)?
> 
>   How in EDS do you say that data (about) x isn't to be made
>   available to queries made by application y? It's the first
> time that I hear that EDS has such a privacy related capabilities.
> 
> Can you also go a bit deeper into how EDS's performance compares to
> Tracker's RDF store for comparable data entry operations? Have you
> guys done measurements on this? Can the results of these measurements
> and the method to reproduce be publicized? Are the queries you used
> available somewhere? Where?
> 
> What do you mean with scalability? Scalable over multiple application
> domains other than PIM? Or scalable to more data? And what kind of
> data? I wonder how EDS is or can be more scalable when the underlying
> database technology (SQLite, apparently) is identical.
> 
> In Tracker is each class stored in a decomposed table. This means that
> contact and E-mail data are each in separate tables. Although
> decomposed typically makes INSERT a bit slower, SELECT is a lot
> faster. How will or does EDS make SELECT equally 'scalable' with
> equally complex queries?
> 
> Also note that the Tracker team optimizes both INSERT and SELECT
> use-cases on a use-case per use-case basis, using its domainindexes
> and indexes. Have you contacted the Tracker team about the scalability
> problems (if any)?
> 
> Or is EDS only more scalable during data entry?
> 
> > Because of all these items and the available expertise, we have
> > decided to start replacing PIM storage with the Evolution Data
> > Server.
> 
> Ah, I see.
> 
> > This change will land together with the SyncEvolution change (due
> > to the intimate relationship between the storage and sync of PIM
> > data). This change should largely be invisible to applications
> > since applications are supposed to access this data via the
> > appropriate QtMobility APIs.
> 
> That's not entirely true. Many applications are nowadays accessing
> said data through the QSparql API of Harmattan. QtMobility isn't
> always the ideal access-layer. Especially not for more complex
> use-cases.
> 
> Are you planning to support or implement a QSparql backend for EDS?
> 
> I'd be thrilled to have a query language like or as powerful as SPARQL
> in EDS! In what timeframe can we expect this? Are you planning to use
> existing SPARQL endpoint technology for this? Which?
> 
> > But to avoid setting precedents, the lower level components will 
> > be removed from the architecture diagrams effective immediately.
> > 
> > To be clear, this does not mean that "tracker" is completely
> > removed; tracker is still being used (together with tumbler) for
> > indexing media on the device. At this point we are seeing serious
> > issues (performance/stability) with this solution, but the first
> > attempt will be to fix the
> > deficiencies rather than a replacement.
> 
> At this moment is Tracker (via QSparql and earlier but deprecated
> libqttracker) used on Harmattan within quite a wide variety of
> (horizontal) applications.
> 
> Are the plans to only use Tracker for indexing? That's a fairly small
> use-case of Tracker on Harmattan nowadays. Fremantle's metadata
> solution of course had an almost exclusive focus on file metadata
> (indexing). But that's not the case on Harmattan. What's the plan for
> those apps?
> 
> 
> 
> Cheers!
> 
> Philip
> 

Just to complement Philip's message, here's my take on the subject:

Comparing Tracker and EDS is a bit like comparing a hitch-hiker's
backpack and a purse. The backpack can definitely store your credit
card, and the purse too. It is probably easier to put the card in your
purse than to take off the backpack, open the good pocket etc.
Now the question is, do you need a purse or a backpack?

If we go on the technical field, Tracker as a solution for contacts is
still a solution that is evolving a lot (you might argue that then it's
not stable/mature, I won't contradict you if you say it's still under
development). With our current versions, we fetch 5000 contacts (when I
say contact, I mean a full blown contact, many phone numbers, mail
addresses etc.) in around 15 seconds, and store them in around 80
seconds. That is not lightning fast. You can do better with a SQLite
datab

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Arjan van de Ven]

On 3/7/2011 1:40 PM, Philip Van Hoof wrote:



Because of all these items and the available expertise, we have decided
to start replacing PIM storage with the Evolution Data Server.

Ah, I see.


good

This change will land together with the SyncEvolution change (due to the
intimate relationship between the storage and sync of PIM data).
This change should largely be invisible to applications since
applications are supposed to access this data via the appropriate QtMobility
APIs.

That's not entirely true. Many applications are nowadays accessing said
data through the QSparql API of Harmattan. QtMobility isn't always the
ideal access-layer. Especially not for more complex use-cases.


MeeGo isn't "Harmattan".  Harmattan isn't MeeGo. (It's Maemo)

I couldn't care less if Harmattan / Apps do or don't so something; that 
has nothing to do with MeeGo.




Are you planning to support or implement a QSparql backend for EDS?


I suspect we'll never see QSparql in MeeGo the way things are going

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Philip Van Hoof]

On Mon, 2011-03-07 at 08:09 -0800, Arjan van de Ven wrote:

Hi Arjan,

> PIM storage
> ===
> The Address book, Calendar data and Email are currently stored in a 
> tracker database, and accessed (officially) via a QtMobility API set.
> There are a range of issues with this implementation, starting with the 
> complexity of adding privacy controls, the performance and
> scalability as well as the completeness for doing a proper syncml sync.

That's great!

Can you explain us, how does EDS improve privacy control compared to
Tracker's RDF store? Does EDS have a way to represent different graphs
(part of Tracker's future plans to do per-resource privacy control)?

How in EDS do you say that data (about) x isn't to be made
available to queries made by application y? It's the first time
that I hear that EDS has such a privacy related capabilities.

Can you also go a bit deeper into how EDS's performance compares to
Tracker's RDF store for comparable data entry operations? Have you guys
done measurements on this? Can the results of these measurements and the
method to reproduce be publicized? Are the queries you used available
somewhere? Where?

What do you mean with scalability? Scalable over multiple application
domains other than PIM? Or scalable to more data? And what kind of data?
I wonder how EDS is or can be more scalable when the underlying database
technology (SQLite, apparently) is identical.

In Tracker is each class stored in a decomposed table. This means that
contact and E-mail data are each in separate tables. Although decomposed
typically makes INSERT a bit slower, SELECT is a lot faster. How will or
does EDS make SELECT equally 'scalable' with equally complex queries?

Also note that the Tracker team optimizes both INSERT and SELECT
use-cases on a use-case per use-case basis, using its domainindexes and
indexes. Have you contacted the Tracker team about the scalability
problems (if any)?

Or is EDS only more scalable during data entry?

> Because of all these items and the available expertise, we have decided 
> to start replacing PIM storage with the Evolution Data Server.

Ah, I see.

> This change will land together with the SyncEvolution change (due to the 
> intimate relationship between the storage and sync of PIM data).
> This change should largely be invisible to applications since 
> applications are supposed to access this data via the appropriate QtMobility
> APIs.

That's not entirely true. Many applications are nowadays accessing said
data through the QSparql API of Harmattan. QtMobility isn't always the
ideal access-layer. Especially not for more complex use-cases.

Are you planning to support or implement a QSparql backend for EDS?

I'd be thrilled to have a query language like or as powerful as SPARQL
in EDS! In what timeframe can we expect this? Are you planning to use
existing SPARQL endpoint technology for this? Which?

> But to avoid setting precedents, the lower level components will 
> be removed from the architecture diagrams effective immediately.
> 
> To be clear, this does not mean that "tracker" is completely removed; 
> tracker is still being used (together with tumbler) for indexing media
> on the device. At this point we are seeing serious issues 
> (performance/stability) with this solution, but the first attempt will 
> be to fix the
> deficiencies rather than a replacement.

At this moment is Tracker (via QSparql and earlier but deprecated
libqttracker) used on Harmattan within quite a wide variety of
(horizontal) applications.

Are the plans to only use Tracker for indexing? That's a fairly small
use-case of Tracker on Harmattan nowadays. Fremantle's metadata solution
of course had an almost exclusive focus on file metadata (indexing). But
that's not the case on Harmattan. What's the plan for those apps?



Cheers!

Philip

-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Vitaly Repin]

Hello,

On Mon, 07 Mar 2011, ext Arjan van de Ven wrote:

> The Address book, Calendar data and Email are currently stored in a 
> tracker database, and accessed (officially) via a QtMobility API set.

Just to correct you a little bit - emails are exported to tracker
database. They are stored in QMF database. Exporting to tracker is
optional feature.

-- 
WBR & WBW,
Vitaly Repin,
Productivity, Nokia-D
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Ivan Frade]

Hi,

On Mon, Mar 7, 2011 at 6:09 PM, Arjan van de Ven  wrote:
> PIM storage
> ===
> To be clear, this does not mean that "tracker" is completely removed;
> tracker is still being used (together with tumbler) for indexing media
> on the device. At this point we are seeing serious issues
> (performance/stability) with this solution, but the first attempt will be to
> fix the
> deficiencies rather than a replacement.

 The whole tracker team is working in the upstream repo [1], bugzilla
[2] and mailing list [3] and so far we haven't receive any feedback
about those performance issues. Lets use upstream as a communication
channel in the benefit of everybody.

 Before taking any more drastic decisions, please keep us in the loop.
It would be nice to have the chance to solve the problems and improve
our code (active developers are working on it 8 hours per day!),
instead of receiving the decisions out of the blue without any option
to defend ourselves. Not that the reasoning was wrong, but comes too
late when we cannot do anything.

 Regards,

Ivan

[1] http://git.gnome.org/browse/tracker
[2] https://bugzilla.gnome.org/buglist.cgi?quicksearch=product%3Atracker
[3] http://mail.gnome.org/mailman/listinfo/tracker-list
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

Re: [MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Robin Burchell]

Hi Arjan,

On Mon, Mar 7, 2011 at 4:09 PM, Arjan van de Ven  wrote:
> Because of all these items and the available expertise, we have decided to
> start replacing PIM storage with the Evolution Data Server.

Worth noting that Maemo5 (Fremantle) had an eds QtContacts backend of
sorts (see: 
http://www.qt.gitorious.org/qt-mobility/contacts/trees/master/plugins/contacts/maemo5).

I'm not sure what sort of state it's in as to usability with regular
eds, but it could possibly be useful as a starting point..

Best regards,

Robin

--
Robin Burchell
http://rburchell.com
___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines

[MeeGo-dev] Some architecture changes (MSSF / Buteo / PIM storage)

[Arjan van de Ven]

Hi,

given the events of the last few weeks, the MeeGo architects have, and 
still are, revisiting various parts of the MeeGo architecture.
While I'd love to say that we have the whole situation clear, the 
reality is that there still is a very complex situation. In part because 
just not everything is clear yet
around "who" and "what", and in part because various parts of the MeeGo 
OS architecture are very tightly coupled...

it's not like MIkkado where you can pull out one stick at a time.

Having said that, three items are currently clear enough to make a final 
decision on:


1) MSSF / MeeGo security framework
2) Buteo Sync
3) PIM storage (currently stored in the tracker database)



Security
===
The security direction of MeeGo has been broken up into two different 
focuses: short-term and long-term.In the short term,
we want to complete the development of key portions of the Mobile 
Simplified Security Framework that allow us to have complete
solutions around the areas of Access Control, Integrity and Security 
Software Distribution.This will not entail *all* of the pieces
that have previously been discussed in these areas, but instead will 
include a minimum subset of features that allow a coherent
story across all key security areas. For MeeGo 1.2 specifically, this 
means that we're not planning on integrating the MSSF pieces
that invasive or incomplete at this point, such as the "upstart" 
integration that was communicated on this list previously.


In the long-term, we will re-evaluate the direction we are taking with 
MeeGo security with a new focus on *End-User Privacy*.
While we do not intend to immediately remove the security enabling 
technologies we have been including in MeeGo, all security
technologies will be re-examined with this new focus in mind.We will 
revisit technology choices made for MSSF (as well as non-MSSF
components that have security implications) and make appropriate changes 
in these areas given this direction change.



Buteo Sync
==
The Buteo Sync framework in MeeGo is currently very incomplete; various 
promised and needed components never materialized, and
are unlikely to materialize in the future. On the Intel side, we've 
found that we ended up glueing SyncEvolution into Buteo on a regular
basis to fulfill basic customer requirements (like 
sync-with-google-address-book).


Because of this, and the available expertise, we have decided to start 
replacing Buteo with SyncEvolution.
For MeeGo 1.2, it's not currently clear if the engineering work that 
this entails will be done in time, so 1.2 might still ship with Buteo.
However, Buteo is removed as architectural component effective 
immediately to avoid creating an API/ABI promise for a component

that we know is being replaced

SyncEvolution is an existing mature open source project with a history 
of functionality and compatibility, and we're confident that

the switch to this project will benefit Sync in MeeGo for years to come.

PIM storage
===
The Address book, Calendar data and Email are currently stored in a 
tracker database, and accessed (officially) via a QtMobility API set.
There are a range of issues with this implementation, starting with the 
complexity of adding privacy controls, the performance and

scalability as well as the completeness for doing a proper syncml sync.

Because of all these items and the available expertise, we have decided 
to start replacing PIM storage with the Evolution Data Server.
This change will land together with the SyncEvolution change (due to the 
intimate relationship between the storage and sync of PIM data).
This change should largely be invisible to applications since 
applications are supposed to access this data via the appropriate QtMobility
APIs. But to avoid setting precedents, the lower level components will 
be removed from the architecture diagrams effective immediately.


To be clear, this does not mean that "tracker" is completely removed; 
tracker is still being used (together with tumbler) for indexing media
on the device. At this point we are seeing serious issues 
(performance/stability) with this solution, but the first attempt will 
be to fix the

deficiencies rather than a replacement.

___
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines