1) Regarding space issues - if you're on a reasonably recent Linux, I've become fond of the "bind mount" over symlinks. < My personal opinion of the week > E.g. In your case - I would do this Stop the server Copy all files from /var/opt/MarkLogic to wherever you have space e.g cp -pr /var/opt/MarkLogic /bigfilesystem/
then just mount over top (the old files will be untouched) mount -B /bigfilesystem/MarkLogic /var/opt/MarkLogic Put this in /etc/fstab to survive a reboot /bigfilesystem/MarkLogic /var/opt/MarkLogic none bind I do this myself to swap between configurations --- Re Modules TO put modules into the DB is the same as putting data into the DB. Configure your HTTP server application to refer to database for Modules. If you're the only developer, the pre-existing Modules DB is a good choice. Your choice of tools ... mlcp is good, I like xmlsh, there is RecordLoader etc. any tool that can copy a file to a DB. You can also load directly form XQuery in Query Console using xdmp:document-load() https://docs.marklogic.com/xdmp:document-load The official docs are here: https://docs.marklogic.com/5.0/guide/app-dev/loading The Dev site lists a dozen or so tools https://developer.marklogic.com/code I suggest the 'best' is whatever you are more familiar with ... Port 8000 on V8 is both a REST and XCC port so all tools should work fine with that. Copy your files to a subdirectory in Modules (say /myapp/ ) Copy them in *text* mode (some tools may assume its XML and complain). Then assign execute and read permissions to /myapp/ for your user. Assuming your using a 'plain' HTTP Server Set your App config to Modules Database: Modules (or whatever you pic) Modules Root: /myapp/ (the subdir in your Modules) Your good to go. ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation d...@marklogic.com Phone: +1 812-482-5224 Cell: +1 812-630-7622 www.marklogic.com -----Original Message----- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Ross A Sent: Saturday, April 11, 2015 3:05 PM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Error debugging HTTP app server through Oxygen David, I'm running MarkLogic 8. Sent from my iPhone > On Apr 11, 2015, at 12:50 PM, general-requ...@developer.marklogic.com wrote: > > Send General mailing list submissions to > general@developer.marklogic.com > > To subscribe or unsubscribe via the World Wide Web, visit > http://developer.marklogic.com/mailman/listinfo/general > or, via email, send a message with subject or body 'help' to > general-requ...@developer.marklogic.com > > You can reach the person managing the list at > general-ow...@developer.marklogic.com > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of General digest..." > > > Today's Topics: > > 1. Re: Adding DOCTYPE header to xml documents in MarkLogic > (Mary Holstege) > 2. Re: DISK space error (Shashidhar Rao) > 3. Re: DISK space error (Indrajeet Verma) > 4. Error debugging HTTP app server through Oxygen (Ross A) > 5. Re: Error debugging HTTP app server through Oxygen (David Lee) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 10 Apr 2015 12:39:40 -0700 > From: Mary Holstege <mary.holst...@marklogic.com> > Subject: Re: [MarkLogic Dev General] Adding DOCTYPE header to xml > documents in MarkLogic > To: MarkLogic Developer Discussion <general@developer.marklogic.com> > Message-ID: <op.xwv58e06fi7...@mary-z620.marklogic.com> > Content-Type: text/plain; charset="utf-8"; format=flowed; delsp=yes > > On Fri, 10 Apr 2015 12:05:58 -0700, Danny Sinang <d.sin...@gmail.com> > wrote: > >> Hi Mary, >> >> I have to output the DOCTYPE below. >> >> <!DOCTYPE Acme.Journal SYSTEM "../../../png.dtd"[]> >> >> >> I'm thinking I can do this inside my query : >> >> declare option xdmp:output "doctype-system="../../../png.dtd"; >> >> >> But how will "Acme.Journal" and the trailing "[]" get included in the >> DOCTYPE tag ? >> >> Regards, >> Danny > > The Acme.Journal will happen automatically if the root of your > document is Acme.Journal. > > The [] is redundant and there is no way to force it. No XML tool > should require it. > > //Mary > >> >> >> On Wed, Feb 13, 2013 at 11:03 AM, Mary Holstege >> <mary.holst...@marklogic.com >>> wrote: >> >>> On Wed, 13 Feb 2013 07:36:26 -0800, Geert Josten >>> <geert.jos...@dayon.nl> >>> wrote: >>> >>>> Hi Karthik, >>>> >>>> >>>> >>>> If you let MarkLogic parse such doctypes properly (using >>> document-load, >>>> document-get or unquote), you would see it gets stripped away. That >>>> is conform XML recommendation. The doctype would have no use inside >>>> MarkLogic, so I wouldn?t bother inserting it at storage within >>>> MarkLogic. >>> Instead, >>>> add >>>> the doctype on output. You can specify output options within a >>> specific >>>> XQuery file, or you can specify it in general for an App Server in >>>> the Admin interface. >>> >>> Just to expand a bit. The DOCTYPE header is not valid as a literal >>> in XQuery, so if you were trying to add it to your content via >>> XQuery, that is why you got the error. >>> >>> MarkLogic does not do DTD validation, so the presence or absence of >>> the header will not materially affect how the XML is processed. If >>> you want the header to appear when you output the content again, >>> this can be achieved through serialization parameters. You can >>> configure these for the appserver, or in your query header. >>> >>> e.g. >>> >>> declare option xdmp:output "doctype-public=-/NLM/DTD Test//En"; >>> declare option xdmp:output "doctype-system=Test.dtd"; >>> >>> //Mary >>> >>>> >>>> I have a requirement to add a DOCTYPE header to a xml document >>>> stored >>> in >>>> MarkLogic DB. >>>> >>>> <!DOCTYPE Test PUBLIC "-//NLM//DTD Test //EN" >>> "Test.dtd"> >>>> >>>> >>>> >>>> If I try to insert a document with the above DOCTYPE header, I am >>> getting >>>> ?Unexpected token syntax error, unexpected Junk_?. >>>> >>>> I didn?t face any issue when I try to insert documents with XSD >>> mapping. >>>> >>>> Is it possible to insert documents with DOCTYPE header? >>> _______________________________________________ >>> General mailing list >>> General@developer.marklogic.com >>> http://developer.marklogic.com/mailman/listinfo/general > > > -- > Using Opera's revolutionary email client: http://www.opera.com/mail/ > > > ------------------------------ > > Message: 2 > Date: Sat, 11 Apr 2015 12:46:24 +0530 > From: Shashidhar Rao <raoshashidhar...@gmail.com> > Subject: Re: [MarkLogic Dev General] DISK space error > To: general@developer.marklogic.com > Message-ID: > > <cafy8jifw4qe4gwtehi4bby5dput+pxd4cb6rr34-nuntfgo...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Thanks to all experts who have given me suggestions really appreciate > your help and have learnt so many things. > > I created a symbolic link to /var/opt/MarkLogic /home/Marklogic(where > there is lots of space) and created 4 forest(This number I am not > quite sure its the correct number) but the loading is going on without any > error. > By the way I am using mlcp to load. > > Thanks and thanks to everyone once again. > > On Sat, Apr 11, 2015 at 12:17 AM, Shashidhar Rao > <raoshashidhar...@gmail.com >> wrote: > >> Thanks Geert , I don't have any issue going over the steps again. As >> I want to start all over again. >> >> On Fri, Apr 10, 2015 at 11:41 PM, Shashidhar Rao < >> raoshashidhar...@gmail.com> wrote: >> >>> Thanks for all your suggestions >>> Marklogic keeps all its data in /var/opt/Marklogic , so now I want >>> to fix this issue. Below are the steps I would like to follow . >>> >>> /home has the highest disk space which is 1.2 TB So , I create a >>> folder /home/opt/Marklogic >>> ln -s /var/opt/Marklogic /home/opt/Marklogic >>> >>> There a couple of files inside in /var/opt/Marklogic from the >>> previous database and forest creation such as forest , database and other >>> xmls. >>> Should I delete all these files, >>> >>> Won't there be any problem if I create a new database. >>> >>> Thanks once again. >>> >>> On Fri, Apr 10, 2015 at 5:57 PM, Shashidhar Rao < >>> raoshashidhar...@gmail.com> wrote: >>> >>>> Hi Marklogic experts, >>>> >>>> We are running into space problem while trying to load data through >>>> mlcp >>>> >>>> 1. Total disk size 1.8 TB >>>> 2. Free space is 1.3 TB /home >>>> 3. we are loading 250 GB data. >>>> >>>> Error we are getting is below - >>>> >>>> SVC-FILWRT: File write error: write 'filename': No space left on >>>> device Error in merge of forest [Forest-Name]: XDMP-MERGESPACE: Not >>>> merging due to disk space limitations, need=584MB, have=260MB >>>> >>>> This I found somewhere-- >>>> Merges require at least twice as much free disk space as used by >>>> the forest data (3X rule). If a merge runs out of disk space, it will fail. >>>> >>>> We feel there is enough space as free space is 1.3 TB >>>> >>>> What configurations we can make in database merging page to avoid >>>> this error. >>>> >>>> Thanks > -------------- next part -------------- An HTML attachment was > scrubbed... > URL: > http://developer.marklogic.com/pipermail/general/attachments/20150411/ > 3ede75d7/attachment-0001.html > > ------------------------------ > > Message: 3 > Date: Sat, 11 Apr 2015 18:12:51 +0530 > From: Indrajeet Verma <indrajeet.ve...@gmail.com> > Subject: Re: [MarkLogic Dev General] DISK space error > To: MarkLogic Developer Discussion <general@developer.marklogic.com> > Message-ID: > > <CAKwHAehVtVja7_Hca24vE18=1xiwxuhbia8xnbrsxpk36us...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > To know more about Forests and other performance improvements, you can > refer > https://developer.marklogic.com/pubs/architecture/inside-marklogic-ser > ver-r7.pdf > > On Sat, Apr 11, 2015 at 12:46 PM, Shashidhar Rao > <raoshashidhar...@gmail.com >> wrote: > >> Thanks to all experts who have given me suggestions really appreciate >> your help and have learnt so many things. >> >> I created a symbolic link to /var/opt/MarkLogic >> /home/Marklogic(where there is lots of space) and created 4 >> forest(This number I am not quite sure its the correct number) but the >> loading is going on without any error. >> By the way I am using mlcp to load. >> >> Thanks and thanks to everyone once again. >> >> On Sat, Apr 11, 2015 at 12:17 AM, Shashidhar Rao < >> raoshashidhar...@gmail.com> wrote: >> >>> Thanks Geert , I don't have any issue going over the steps again. As >>> I want to start all over again. >>> >>> On Fri, Apr 10, 2015 at 11:41 PM, Shashidhar Rao < >>> raoshashidhar...@gmail.com> wrote: >>> >>>> Thanks for all your suggestions >>>> Marklogic keeps all its data in /var/opt/Marklogic , so now I want >>>> to fix this issue. Below are the steps I would like to follow . >>>> >>>> /home has the highest disk space which is 1.2 TB So , I create a >>>> folder /home/opt/Marklogic >>>> ln -s /var/opt/Marklogic /home/opt/Marklogic >>>> >>>> There a couple of files inside in /var/opt/Marklogic from the >>>> previous database and forest creation such as forest , database and other >>>> xmls. >>>> Should I delete all these files, >>>> >>>> Won't there be any problem if I create a new database. >>>> >>>> Thanks once again. >>>> >>>> On Fri, Apr 10, 2015 at 5:57 PM, Shashidhar Rao < >>>> raoshashidhar...@gmail.com> wrote: >>>> >>>>> Hi Marklogic experts, >>>>> >>>>> We are running into space problem while trying to load data >>>>> through mlcp >>>>> >>>>> 1. Total disk size 1.8 TB >>>>> 2. Free space is 1.3 TB /home >>>>> 3. we are loading 250 GB data. >>>>> >>>>> Error we are getting is below - >>>>> >>>>> SVC-FILWRT: File write error: write 'filename': No space left on >>>>> device Error in merge of forest [Forest-Name]: XDMP-MERGESPACE: >>>>> Not merging due to disk space limitations, need=584MB, have=260MB >>>>> >>>>> This I found somewhere-- >>>>> Merges require at least twice as much free disk space as used by >>>>> the forest data (3X rule). If a merge runs out of disk space, it will >>>>> fail. >>>>> >>>>> We feel there is enough space as free space is 1.3 TB >>>>> >>>>> What configurations we can make in database merging page to avoid >>>>> this error. >>>>> >>>>> Thanks >> >> _______________________________________________ >> General mailing list >> General@developer.marklogic.com >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general > -------------- next part -------------- An HTML attachment was > scrubbed... > URL: > http://developer.marklogic.com/pipermail/general/attachments/20150411/ > b3f4792b/attachment-0001.html > > ------------------------------ > > Message: 4 > Date: Sat, 11 Apr 2015 11:50:59 -0400 > From: Ross A <a.ro...@gmail.com> > Subject: [MarkLogic Dev General] Error debugging HTTP app server > through Oxygen > To: general@developer.marklogic.com > Message-ID: > > <CAA9kcfyRMt-GZUwaoEdLHek9kgjWUax9rtqxVjbOEKuoUGz=m...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > So I'm running couple of the MarkLogic getting started tutorials. I > set up my MarkLogic datasource/connection both in oxygen and eclipse. > That part looks great. > > The problem I'm having is, I can't seem to run the debugger. I keep > getting the following error below in both oxygen and eclipse: > > Description: > com.marklogic.xcc.exceptions.XQueryException.getStack()[Lcom/marklogic > /xcc/exceptions/XQueryStackFrame; > > Severity: > Error > > System ID: > oxygen://user1@localhost/MarkLogic DS$MarkLogic > Connection/fs/AppServer/$AppServer/dump.xqy$/dump.xqy > > I'm I missing anything? > > Thx > Ross > -------------- next part -------------- An HTML attachment was > scrubbed... > URL: > http://developer.marklogic.com/pipermail/general/attachments/20150411/ > e9f2e46f/attachment-0001.html > > ------------------------------ > > Message: 5 > Date: Sat, 11 Apr 2015 16:50:08 +0000 > From: David Lee <david....@marklogic.com> > Subject: Re: [MarkLogic Dev General] Error debugging HTTP app server > through Oxygen > To: MarkLogic Developer Discussion <general@developer.marklogic.com> > Message-ID: > > <6ad72d76c2d6f04d8be471b70d4b991e0bc72...@exchg10-be01.marklogic.com> > Content-Type: text/plain; charset="utf-8" > > Is your module in the filesystem or the database ? > I recall that Oxygen could only debug one or the other, I think only DB based > modules ( seeing the error: > Connection/fs/AppServer/$AppServer/dump.xqy$/dump.xqy > The "fs" part is a clue. > > What version of MarkLogic are you running ? > You should also contact Oxygen support/help - ML is not directly > involved in Oxygen software support (we don't have their code and don't know > exactly what it does - but we are on good terms.) They are very responsive > and if you can isolate the problem they will help you with it or fix it - or > contact us if they think its a bug in the server. > > > > > ---------------------------------------------------------------------- > ------- > David Lee > Lead Engineer > MarkLogic Corporation > d...@marklogic.com > Phone: +1 812-482-5224 > Cell: +1 812-630-7622 > www.marklogic.com<http://www.marklogic.com/> > > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of Ross A > Sent: Saturday, April 11, 2015 11:51 AM > To: general@developer.marklogic.com > Subject: [MarkLogic Dev General] Error debugging HTTP app server > through Oxygen > > So I'm running couple of the MarkLogic getting started tutorials. I set up my > MarkLogic datasource/connection both in oxygen and eclipse. That part looks > great. > > The problem I'm having is, I can't seem to run the debugger. I keep getting > the following error below in both oxygen and eclipse: > > Description: > com.marklogic.xcc.exceptions.XQueryException.getStack()[Lcom/marklogic > /xcc/exceptions/XQueryStackFrame; > > Severity: > Error > > System ID: > oxygen://user1@localhost/MarkLogic DS$MarkLogic > Connection/fs/AppServer/$AppServer/dump.xqy$/dump.xqy > I'm I missing anything? > Thx > Ross > -------------- next part -------------- An HTML attachment was > scrubbed... > URL: > http://developer.marklogic.com/pipermail/general/attachments/20150411/ > 02fd4032/attachment.html > > ------------------------------ > > _______________________________________________ > General mailing list > General@developer.marklogic.com > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > > > End of General Digest, Vol 130, Issue 37 > **************************************** _______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
