Hi List, There has been a long time since the last status update and few related commits. You will find below information about recent work, issues encountered and roadmap update.
[0x0] Contents
========================================================
[0x4] Create the nspi_server mapiproxy module
[0x5] EMSABP provider merging
[0x6] Roadmap Update
========================================================
[0x4] Create the nspi_server mapiproxy module
=============================================
First, I've noticed that developing nspi_server as a mapiproxy module
wouldn't be worthwhile and would cause more troubles than benefits. It
will instead be static code within dcesrv_mapiproxy. The server will be
enabled through parametric options in smb.conf and I'm currently looking
how we can remove most of the parametric options we were using in the
past: no more NSPI guid in smb.conf but directly in samba4 AD etc.
Making the nspi server be part of mapiproxy also offers several
advantages:
1. mapiproxy modules (separated dso) using their own static
lists/structures, require developers to maintain a per-session
mechanism for multiple users support. This task has been
implemented up to 50% in mpm_cache module and its implementation
mechanism should be available to all modules. However if the
NSPI server is linked with dcesrv_mapiproxy, then we do not need
to care about this issue anymore and we can safely retrieve the
user context from the dcesrv_call_state structures received from
samba4.
2. If the NSPI server is static, then it means we can easily use
mapiproxy modules and control how NSPI server should behave.
I've been working on the overall design and implementation of this
solution and have identified all the possible technical issues, so there
wouldn't be any problem. Furthermore, I'll be able to reproduce this
behavior and develop emsmdb and rfr skeleton fairly easily.
[0x5] EMSABP provider merging
=============================
There is a task we omitted to tag as "blocker" and prevent from any
proper EMSABP provider merge and which is OpenChange provisioning.
I've started to work on this issue since Tuesday. So far I have
(locally) updated the python scripts and modules and I'm currently
working on updating the openchange ldif files so we can add new classes
into samba4 schema, modify existing ones and add records openchange
needs into the sam database.
However, we have not been touching the ldif files for a ... very very
long time and it appears that they require some changes - which I have
not finished to identify yet. Furthermore, I was basing the provisioning
task update over samba4-alpha5 release and there has been many changes
in samba4 git after the release got cut down.
This is yet another reason (see libmapi-0.8 release schedule post) why
we need to upgrade from alpha5 release to recent git revision.
When this task is completed, I'll be able to push a preliminary commit
and have the underlying data required to test emsabp step by step and
fix the implementation.
[0x6] Roadmap update
====================
The issues mentioned above shouldn't impact on the schedule planned for
the merging process. However, I'll probably won't be able to add support
for new NSPI functions (other than the ones already supported) within
next week.
Finally, NSPI server development, merging etc. goes with a mapitest
module development designed to test and compare behavior between
Exchange and OpenChange NSPI server and guarantee we do the things
properly.
Cheers,
Julien.
On Sat, 2008-09-06 at 22:18 +0200, Julien Kerihuel wrote:
> Hi List,
>
> Quick update: "NSPI protocol update" and "Completing NSPI protocol
> implementation in libmapi" tasks are done and committed on svn r726.
>
> This is an important update: most of the NSPI stack has been rewritten
> and both the NSPI stack implementation and IDL are complete. There's an
> exception regarding NspiModLinkAtt and NspiModProps which behavior isn't
> - imho - clear in MS docs and for which I have asked further
> information:
> http://forums.msdn.microsoft.com/en-US/os_exchangeprotocols/thread/7beab30a-fa15-48f8-977a-4f951a933b7a
>
> I have also implemented a preliminary mapitest module for the NSPI stack
> which I'll presumably improve and use to test the EMSABP server
> robustness and behavopr. You can run the test suite with:
>
> $ mapitest --mapi-calls=NSPI-ALL --dump-data -d10 --color
>
> Developers using libmapi trunk shouldn't notice much difference except
> if your application directly uses libmapi/nspi.c API. Developers relying
> on IProfAdmin.c API are not affected by this change.
>
> However there is one difference that may affect your application:
> the FlagList to SPropTagArray change.
> You should normally only have to rename FlagList to SPropTagArray to
> keep your existing application working. If it doesn't make the job,
> please have a look at openchange tools and mapitest for implementation
> examples.
>
> The next step will probably have more direct impact to your application
> but is intended to make openchange MAPI structures closer to thoses
> described in MS-* specifications. "[0x3] Renaming MAPI structures"
>
> FYI, the whole NSPI implementation was committed as one big chunk so we
> can temporary revert in case something goes wrong.
>
> Cheers,
> Julien.
>
> On Fri, 2008-09-05 at 15:54 +0200, Julien Kerihuel wrote:
> > Hi List,
> >
> > Here is a status update of the work in progress and a more detailed
> > roadmap of the NSPI development part.
> >
> > [0x0] Contents
> > ==============================================================
> > [0x1] NSPI protocol update
> > [0x2] Completing NSPI protocol implementation in libmapi
> > [0x3] Renaming MAPI structures
> > [0x4] Create the nspi_server mapiproxy module
> > [0x5] EMSABP provider merging
> > [0x6] EMSABP provider update
> > [0x7] RFR and NSPI wireshark dissector
> > ==============================================================
> >
> >
> > [0x1] NSPI protocol update
> > ==========================
> >
> > I've started to work on a complete update of the NSPI protocol, closely
> > following MS-NSPI.pdf specifications. This task is mostly about renaming
> > IDL function and fields, add new set of structures and modify existing
> > ones to match the IDL with more accuracy. This preliminary task is
> > almost over:
> > - NSPI profiles work properly again
> > - existing EMSABP provider, dcesrv_exchange and mapiproxy have
> > been updated to reflect these changes. There is still a bit of
> > inconsistencies in dcesrv_exchange, but this file is scheduled
> > for deletion and will be replaced in further commits by a brand
> > new and working mapiproxy module.
> >
> > I'll however be waiting for [0x2] to be completed before I push the
> > first update.
> >
> > Note that this task shouldn't normally affect external applications if
> > you rely on the IProfAdmin interface. If you directly call API functions
> > from libmapi/nspi.c, then you'll have a bit of renaming to do in the
> > future.
> >
> >
> >
> > [0x2] Completing NSPI protocol implementation in libmapi
> > ========================================================
> >
> > I've also started this task and have begun to implement some new NSPI
> > function (IDL + libmapi implementation) such as NspiQueryColumns or
> > NspiSeekEntries. I have also started to write a new mapitest module
> > designed to test the NSPI protocol functions. This task will be
> > completed when the full protocol is implemented, NSPI mapitest module is
> > complete and pass the full NSPI tests.
> >
> > When this task is over, (scheduled for the end of week), I'll push the
> > first update on trunk.
> >
> >
> >
> > [0x3] Renaming MAPI structures
> > ==============================
> >
> > Ticket #002 on http://trac.openchange.org is inconsistent regarding the
> > scope of the planned update and will be updated to reflect the intended
> > modifications. Basically, this modification is about making libmapi data
> > structures naming more conventional and remove the mapi_ prefix to MAPI
> > structures in the EMSMDB IDL. NSPI structures will be suffixed with _r
> > as specified in the specs.
> >
> > This may require a major update for developers using openchange, but
> > this will be a one shot effort. Furthermore, some of the structures
> > names won't be changed as it would be more inconvenient than worthwhile.
> >
> >
> >
> > [0x4] Create the nspi_server mapiproxy module
> > =============================================
> >
> > I have already implemented a preliminary skeleton of the nspi_server
> > mapiproxy module designed to dispatch NSPI calls to the correct
> > function. I'm also in the process of merging server/dcesrv_exchange.c
> > NSPI related code into this module and brings the necessary updates:
> > sanity checks etc.
> >
> > One of the task (which shouldn't be too long), will be to add some
> > parametric options to smb.conf and let the user choose whether it wants
> > mapiproxy to act as a real Address Book provider (nspi_server mode) or
> > transparently relay the traffic. FYI, there is already a
> > dcesrv_mapiproxy_nspi.c file into mapiproxy designed to modify some NSPI
> > traffic on the fly. mapiproxy needs to use either this file in case it
> > relays the traffic or the forthcoming nspi_server is it acts as a
> > server.
> >
> >
> >
> > [0x5] EMSABP provider merging
> > =============================
> >
> > The next step is to merge the existing EMSABP provider into mapiproxy.
> > I've also started to work on this task and I am rewriting the provider
> > from scratch so it includes more sanity checks and gets closer to
> > MS-NSPI specifications (expected server behavior). This task also
> > includes the merge of provisioning scripts back to trunk, so we can
> > extend Samba4 AD schemas, attributes and users entries with Exchange
> > ones again. I am yet unsure how long this merge/update process will
> > require.
> >
> >
> >
> > [0x6] EMSABP provider update
> > ============================
> >
> > The next step is to improve the EMSABP provider so it can handle a more
> > consequent set of NSPI calls. This will probably lead in an update of
> > the EMSABP provider, Exchange schemas, attributes and MAPITAGS to x500
> > mapping table.
> >
> >
> >
> > [0x7] RFR and NSPI wireshark dissector
> > ======================================
> >
> > Last but not least, I thought you might be interested to know I have
> > supplied a patch to wireshark to add RFR protocol dissector and will
> > send an update for the NSPI dissector as soon as task [0x2] is over.
> >
> > https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=2850
> >
> >
> > That's all for the moment.
> >
> > Cheers,
> > Julien.
> >
> >
> > On Wed, 2008-09-03 at 17:04 +0200, Julien Kerihuel wrote:
> > > Hi List,
> > >
> > > You will find below information about the roadmap on openchange server,
> > > how we plan to update the existing code and preliminary information on
> > > the EMSMDB store development roadmap.
> > >
> > >
> > > [0x0] Contents
> > > =======================================================
> > > [0x1] Introduction
> > > [0x2] Hybrid design: merging server and mapiproxy
> > > [0x3] Next step
> > > [0x4] References
> > > =======================================================
> > >
> > >
> > >
> > > [0x1] Introduction
> > > ===================
> > >
> > > OpenChange has in the past implemented half an Exchange server (EMSABP
> > > provider - AddressBook), but due to the efforts on libmapi, work on the
> > > EMSMDB provider - the core of openchange server - has critically been
> > > slowed down. Furthermore the EMSABP is at the moment outdated and not
> > > usable in trunk.
> > >
> > > In the meantime, we have put a lot of design, development and
> > > documentation efforts in mapiproxy [1]. This hybrid proxy server starts
> > > to have nice features such as the cache module which is able to replace
> > > a real Exchange server for some operations such as attachments or large
> > > message contents streaming. It also offers read-ahead and norelay
> > > options so mapiproxy modules can really control how they discuss both
> > > with MAPI clients and remote Exchange server.
> > >
> > > Last but not least, there is an issue between openchange server and
> > > mapiproxy. Both of them registers the same endpoints and you can't have
> > > both shared libraries installed at the same time.
> > >
> > >
> > >
> > > [0x2] Hybrid design: merging server and mapiproxy
> > > ==================================================
> > >
> > > The new approach will be to merge openchange server into mapiproxy.
> > >
> > > The main objectives are to fix the incompatibility mentioned above, put
> > > openchange server on good rails again and finally improve mapiproxy with
> > > important features.
> > >
> > > This will let users configure mapiproxy to act as a transparent gateway
> > > and/or progressively and seamlessly replace an Exchange server with
> > > openchange.
> > >
> > > Some of the forthcoming modules we can plan is to progressively populate
> > > Samba4 AD when data is not available in the SAM database. For example:
> > > 1. Outlook wants to resolve a name (NSPI - EMSABP provider)
> > > 2. mapiproxy asks openchange server for the name
> > > 3. If openchange server has the information, it is returned
> > > directly to Outlook
> > > 4. If openchange server doesn't have the information, mapiproxy
> > > relays the information to the real Exchange server, update the
> > > SAM database with the information fetched and return the info
> > > back to the client.
> > > 5. When Outlook requests for the same username again, it is read
> > > from openchange server and requests are not forwarded to
> > > Exchange anymore.
> > >
> > > For this integration to be successful, Jelmer's work on python's
> > > provisioning script will be merged back to trunk, providers be moved
> > > into mapiproxy and a new module will be developed to call emsabp
> > > provider. I'm willing to finish the initial merging task within the next
> > > 2 weeks. The update module designed above will next be developed
> > > depending on expressed needs.
> > >
> > > I also plan to start the development of a exchange_ds_rfr provider and
> > > mapiproxy module. Finally I plan to implement a skeleton draft of a
> > > emsmdb provider.
> > >
> > >
> > >
> > > [0x3] Next Step
> > > ================
> > >
> > > When the work above above is completed, I'm willing to schedule a
> > > meeting with people interested in mapidb development (as described in
> > > [2]) and define a preliminary functional draft and API for the
> > > abstraction storage backend.
> > >
> > >
> > >
> > > [0x4] References
> > > =================
> > > [1] http://mapiproxy.openchange.org
> > > [2] http://wiki.openchange.org/index.php/EMSMDB
> > >
> > >
> > > Cheers,
> > > Julien.
>
> _______________________________________________
> devel mailing list
> [email protected]
> http://mailman.openchange.org/listinfo/devel
--
Julien Kerihuel
[EMAIL PROTECTED]
OpenChange Project Manager
GPG Fingerprint: 0B55 783D A781 6329 108A B609 7EF6 FE11 A35F 1F79
signature.asc
Description: This is a digitally signed message part
_______________________________________________ devel mailing list [email protected] http://mailman.openchange.org/listinfo/devel
