Hi Everyone, As Dominic pointed out, we are working anyway on an import into BASE2 using tab2mage format in the short time (see ticket 338), and once MAGE-TAB specifications are stable move on to support this format too. There is a MAGE-TAB jamboree happening in Philadelphia next week, so I will count on Helen to update us on the outcome of this meeting so we can possibly expedite the process. The tab2mage import will prove no doubt extremely useful to exchange data and to get data from repositories as pointed out So everything is in the pipeline and we jsut need a little bit a patience, days have only 24 hours
cheers Philippe -- **************************************************************** Philippe Rocca-Serra (Ph.D) Nutri/Toxicogenomics Internal Coordinator EBI- The European Bioinformatics Institute EMBL Outstation Hinxton Wellcome Trust Genome Campus CB10 1SD Cambridge, UK email: [EMAIL PROTECTED] Tel: +44 (0) 1223 492 553 Fax: +44 (0) 1223 494 468 http://www.ebi.ac.uk/microarray/Projects/tox-nutri/index.html **************************************************************** Helen Parkinson wrote: > Hi, > > this is an interesting problem and one we have been dealing with at > ArrayExpress as well. For really large experiments the only thing that > that has worked for submissions is a spreadsheet - currently we use > tab2mage for this purpose. *tab2mage*.sourceforge.net/ > > > You might think about an importer that takes a spreadsheet, or partial > spreadsheet plus files in an archive as input, e.g. MAGETAB, this would > allow people to pull in other people's data downloaded from public > repositories, and allow biologists to format up their own data easily. > And as spreadsheet export from BASE is planned the import would also be > nice, round tripping is also a good test of format and application > > cheers > > Helen > > Jari Häkkinen wrote: > >> There is a possibility to get access to the subversion repository and >> trac environment just for this kind of stuff. Check out >> http://baseplugins.thep.lu.se Trac environment allows for discussions >> about the importer through the ticketing system. >> >> >> Jari >> >> >> Micha Bayer wrote: >> >> >>> That sounds like a hell of a lot more work than what I have done so far.... >>> :-) >>> >>> My first shot at this has been quite basic and it only creates new >>> RawBioAssays and a single Experiment that these belong to. It could of >>> course be extended to do more stuff, but my time allowance for this is >>> limited and I don't think I'll be able to spend a lot more time on this >>> than I already have. >>> >>> I think the upshot of all this discussion has been: >>> >>> 1. people need a batch importer (desperately, in fact), and >>> 2. a plugin is much better and will (hopefully) be possible >>> >>> With that in mind, I'll go away and look into turning my stuff into a >>> plugin now, and when I have got the basic version of that working (i.e. >>> file upload, RawBioAssay and Experiment creation only, as above) I'll make >>> the code available to the community somehow and someone else can pick it up >>> from there if they want to make it fancier. I hope that's okay -- even with >>> no extra effort from someone else it'll still mean that a hell of a lot of >>> effort can be saved. >>> >>> Perhaps the dev team can point me in the right direction initially. My >>> colleague Iain here is just working on a file importer plugin that handles >>> Excel files and his code starts off like this: >>> >>> >>> public class ExcelFileImporter extends AbstractPlugin implements >>> InteractivePlugin, AutoDetectingImporter >>> { >>> private static final Set<GuiContext> guiContexts = >>> Collections.singleton(new GuiContext(Item.RAWBIOASSAY, >>> GuiContext.Type.ITEM)); >>> >>> >>> So am I right in thinking that if I change this to the following... >>> >>> >>> public class ZipFileImporter extends AbstractPlugin implements >>> InteractivePlugin, AutoDetectingImporter >>> { >>> private static final Set<GuiContext> guiContexts = >>> Collections.singleton(new GuiContext(Item.EXPERIMENT, >>> GuiContext.Type.ITEM)); >>> >>> >>> ....then I'll get an import tab created in the Experiment list view which >>> will let me import the zip file using my plugin? >>> >>> We're planning to make the Excel plugin available for everyone too BTW, >>> once it's finished. >>> >>> Cheers >>> Micha >>> >>> ================================== >>> Dr Micha M Bayer >>> Bioinformatics Specialist >>> Genetics Programme >>> The Scottish Crop Research Institute >>> Invergowrie >>> Dundee >>> DD2 5DA >>> Scotland, UK >>> Telephone +44(0)1382 562731 ext. 2309 >>> Fax +44(0)1382 562426 >>> http://www.scri.sari.ac.uk/MichaBayer.htm >>> ============================= >>> >>> >>> >>> >>>> -----Original Message----- >>>> From: [EMAIL PROTECTED] [mailto:basedb-users- >>>> [EMAIL PROTECTED] On Behalf Of Bob MacCallum >>>> Sent: 29 November 2006 18:36 >>>> To: BASE ML >>>> Subject: Re: [base] Offline files >>>> >>>> >>>> It looks like I let the cat out of the bag on this one... >>>> >>>> Some discussion among the community would be good to establish what we >>>> want >>>> from a bulk loader. For example, I would like all objects created and >>>> linked, >>>> from samples (with annotations) right up to raw bioassays (grouped into an >>>> experiment as Micha does, ideally). Obviously to do this you need to >>>> specify >>>> the relationships between all the samples, annotations, dyes, hybs, data >>>> files, but that could easily end up being as much work as manually loading >>>> the >>>> data ;-) >>>> >>>> Therefore you have to make compromises; the main one being that you treat >>>> every sample/extract/etc in exactly the same way (same protocols, same >>>> amounts, same labels, same array design). This would probably work for >>>> most >>>> people - but let's discuss it anyway... >>>> >>>> One issue with the bulk loader I wrote for BASE 1 is that if your >>>> experiment >>>> involves a dye swap for one biological replicate, the bulk loader will >>>> label >>>> each extract with both dyes for *all* replicates (leaving you with unused >>>> labelled extracts after linking to the hybs). >>>> >>>> One quite low cost way to define your experiment for a bulk loader is >>>> Tab2MAGE >>>> (soon to be MAGE-TAB, see >>>> http://www.biomedcentral.com/1471-2105/7/489/abstract ) - although it >>>> feels >>>> strange to say this because I have been *exporting* Tab2MAGE from BASE 1.2 >>>> rather than importing it. The input files for our bulk loader are >>>> specified >>>> here: >>>> http://wiki.vectorbase.org/vectorbase/index.php/MicroarrayExperiments >>>> (look for the example file links) but note that the BASE-specific >>>> parameters >>>> (protocols, raw file parsers, array designs, etc) are configured by the >>>> user >>>> in an interactive process before the upload. Our bulk loader is also >>>> 2-channel only... >>>> >>>> I have a feeling that the Tab2MAGE parser could be adapted into a BASE 2 >>>> bulk >>>> loader somehow, although I think all Tab2MAGE code is Perl. >>>> >>>> I think MAGE-TAB is something we could get the average bench scientist to >>>> provide (with a little help MGEDifying the annotations). So far every >>>> experiment I have loaded has come with some kind of spreadsheet (each one >>>> in a >>>> different format of course), so the leap to MAGE-TAB is not so great. >>>> >>>> Ok, enough from me... >>>> >>>> >>>> >>>> >>>> Micha Bayer writes: >>>> > >>>> > >>>> > > -----Original Message----- >>>> > > From: [EMAIL PROTECTED] [mailto:basedb- >>>> users- >>>> > > [EMAIL PROTECTED] On Behalf Of Nicklas Nordborg >>>> > > Sent: 29 November 2006 14:40 >>>> > > To: BASE ML >>>> > > Subject: Re: [base] Offline files >>>> > > >>>> > > Jari Häkkinen wrote: >>>> > > > Hi all, >>>> > > > >>>> > > > Just a thought, I think the best choice to trigger a large import >>>> like >>>> > > > this is to create a plug-in instead of adding extra tabs to the >>>> file >>>> > > > browser. The plug-in should start by requesting the zip >>>> file/directory >>>> > > > to import. If the zip file is not already uploaded the user will >>>> have an >>>> > > > opportunity to upload it. After choosing the zip file/directory the >>>> > > > plug-in could start its import and creation of necessary items. >>>> > > > >>>> > > > The above is fairly straightforward already today and the new code >>>> would >>>> > > > start at import/creation of necessary items. >>>> > > > >>>> > > > This will yield very little (or no) changes to the core and the >>>> plug-in >>>> > > > should probably be started from the experiment listing page. This >>>> will >>>> > > > of course change the experiment list jsp (a import tab is needed). >>>> > > >>>> > > No change is needed to the jsp. The new plugin will be detected and >>>> an >>>> > > "Import" button will appear. >>>> > > >>>> > > /Nicklas >>>> > >>>> > So what interface would I have to implement for this to happen, if I >>>> were to turn my code into a plugin? >>>> > >>>> > If it's not too much work I might consider doing this, that would be >>>> much better. >>>> > >>>> > Cheers >>>> > >>>> > Micha >>>> > >>>> > ================================== >>>> > Dr Micha M Bayer >>>> > Bioinformatics Specialist >>>> > Genetics Programme >>>> > The Scottish Crop Research Institute >>>> > Invergowrie >>>> > Dundee >>>> > DD2 5DA >>>> > Scotland, UK >>>> > Telephone +44(0)1382 562731 ext. 2309 >>>> > Fax +44(0)1382 562426 >>>> > http://www.scri.sari.ac.uk/MichaBayer.htm >>>> > ============================= >>>> > >>>> > >>>> > >>>> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ >>>> _ _ >>>> > >>>> > DISCLAIMER: >>>> > >>>> > This email is from the Scottish Crop Research Institute, but the views >>>> > expressed by the sender are not necessarily the views of SCRI and its >>>> > subsidiaries. This email and any files transmitted with it are >>>> confidential >>>> > to the intended recipient at the e-mail address to which it has been >>>> > addressed. It may not be disclosed or used by any other than that >>>> addressee. >>>> > If you are not the intended recipient you are requested to preserve >>>> this >>>> > confidentiality and you must not use, disclose, copy, print or rely on >>>> this >>>> > e-mail in any way. Please notify [EMAIL PROTECTED] quoting the >>>> > name of the sender and delete the email from your system. >>>> > >>>> > Although SCRI has taken reasonable precautions to ensure no viruses are >>>> > present in this email, neither the Institute nor the sender accepts any >>>> > responsibility for any viruses, and it is your responsibility to scan >>>> the email >>>> > and the attachments (if any). >>>> > >>>> > >>>> > ----------------------------------------------------------------------- >>>> -- >>>> > Take Surveys. Earn Cash. Influence the Future of IT >>>> > Join SourceForge.net's Techsay panel and you'll get the chance to share >>>> your >>>> > opinions on IT & business topics through brief surveys - and earn cash >>>> > >>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >>>> > _______________________________________________ >>>> > The BASE general discussion mailing list >>>> > basedb-users@lists.sourceforge.net >>>> > unsubscribe: send a mail with subject "unsubscribe" to >>>> > [EMAIL PROTECTED] >>>> >>>> -- >>>> Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups | >>>> Division of Cell and Molecular Biology | Imperial College London | >>>> Phone +442075941945 | Email [EMAIL PROTECTED] >>>> >>>> ------------------------------------------------------------------------- >>>> Take Surveys. Earn Cash. Influence the Future of IT >>>> Join SourceForge.net's Techsay panel and you'll get the chance to share >>>> your >>>> opinions on IT & business topics through brief surveys - and earn cash >>>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >>>> _______________________________________________ >>>> The BASE general discussion mailing list >>>> basedb-users@lists.sourceforge.net >>>> unsubscribe: send a mail with subject "unsubscribe" to >>>> [EMAIL PROTECTED] >>>> >>>> >>> ------------------------------------------------------------------------- >>> Take Surveys. Earn Cash. Influence the Future of IT >>> Join SourceForge.net's Techsay panel and you'll get the chance to share your >>> opinions on IT & business topics through brief surveys - and earn cash >>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >>> _______________________________________________ >>> The BASE general discussion mailing list >>> basedb-users@lists.sourceforge.net >>> unsubscribe: send a mail with subject "unsubscribe" to >>> [EMAIL PROTECTED] >>> >>> >>> >> >> > > ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject "unsubscribe" to [EMAIL PROTECTED]
