On 7 Dec 2007, at 10:09, Syed Haider wrote:
Hi Michael,
On Fri, 2007-12-07 at 09:38 +0000, Michael Wicks wrote:
Dear Biomart,
I have been investigating the use of Biomart with our database
(http://www.eurepxress.org), following on from a recently departed
colleague (Chuang-Kee Ong, who some of you may have dealt with
recently). and I have a couple of questions:
1. Once a Mart has been created, how do you refresh the contents of
the
Mart if the database on which it is based changes?
You have to schedule upgrades of database into mart. This process can
be
done use MartBuilder's sql dump. Otherwise if you are confident about
your DB -> Mart mappings, just keep a little script and schedule it to
run automatically and upgrade your Mart. Richard would be able to
comment on this if there is a better way.
Hi Michael,
it depends what you mean by 'database changes' :) if you mean
the source relational schema changes e.g adding new tables, changing
defintion
of existing ones etc .... then yes we do have Mbuilder 'update'
procedure
to pick up the changes and put the two in sync as Syed describes here.
If you just mean 'data in the db changes' then as I said
previously you drop your mart and you re-run mart build again (using
the same
xml transformation config - if you are using MBuilder ). We have been
so far relying on the fact
that the release schedule typically is less frequent then the time
required to re-build
mart.
The new procedure for incremental data updates (without necessity to
re-run it from scratch)
in cases of very frequent data updates
is being considered at the moment but there is no firm implementation
deadline as yet
a.
Our database grows daily with new data, and without some procedures or
processes, preferably non-manual, put in place, the Mart will almost
immediately become inconsistent with our base data. I can see how to
do
this with the tools supplied but only manually. Also I don't really
want to start building something myself, as it occurred to me that
this
must have been an issue before with other Marts, and that there might
be
a pre-existing solution to refreshing Marts already.
Have you come across this issue before, and if so, can you suggest a
solution please?
2. Documentation. I can't find anywhere in the documentation any
explanation of how to go about implementing Importables and Exportable
fields. Our data has a number of potential linking fields and we
would
like to implement these links when we put our Mart live.
We will additional information about this and couple of example.
Usually, deployers tend to learn from marts which are already
configured
and publicly available. For instance, the best way to find what goes in
which field is to see Ensembl Mart 'gene_ensembl' dataset XML available
on host: martdb.ensembl.org port: 3316, user: anonymous, no password,
database and schema: ensembl_mart_47, db driver: mysql. Using these
credentials, connect using MartEditor. See how XML is configured and at
the same time visit www.biomart.org/biomart/martview to see how the
Ensembl Mart looks like. Would be very useful.
Do you have a document that explains this in more detail please (I
have
been looking at a document called "Biomart 0.6 Documentation" so far)?
(A minor documentation issue: when configuring a Mart, the Perl script
configure.pl has to be run. I'm still not sure of the difference
between the "Y" and "N" options that I'm presented with when this
script
runs, even after looking at the documentation. So I run the script
twice! Can you explain what is going on here please?)
Many user do not wish to install MartView (web interface) and on
install
API dependencies. So they need to say 'n'.
Many thanks for your time.
Yours
Mike Wicks
--
======================================
Syed Haider.
EMBL-European Bioinformatics Institute
Wellcome Trust Genome Campus, Hinxton,
Cambridge CB10 1SD, UK.
======================================
------------------------------------------------------------------------
-------
Arek Kasprzyk
EMBL-European Bioinformatics Institute.
Wellcome Trust Genome Campus, Hinxton,
Cambridge CB10 1SD, UK.
Tel: +44-(0)1223-494606
Fax: +44-(0)1223-494468
------------------------------------------------------------------------
-------
