David,
Backups are platform-specific. So a backup from a Windows 32-bit
instance won't restore on a 64-bit Windows instance, or even a 32-bit
Linux instance. It's not clear to me whether that is the problem or not.
I have never tried reindexing with an over-capacity license, but I
wouldn't expect it to work. Every update should fail, because updates
aren't allowed when the license is over capacity. Possibly that's what
you are seeing? Have you looked at the error log?
For whatever it's worth, I have used 4.1-4 on EC2 with good results. I
used the rightscale CentOS 5.2 64-bit linux image.
-- Mike
On 2010-01-18 11:35, Lee, David wrote:
I eventually gave up on this on both EC2 and a desktop .
The restore had run for 3 solid days and was making no progress at all. CPU is
pegged but absolutely no progress in the status, and IO writes and reads are 0 !
My hunch was this has something to do with trying to restore into a comunity
edition ML server with too much data.
So I wiped my target machine (desktop) applied a trial license, and am
repeating the restore.
Same thing. Its been running for hours now and has "stalled" ... no progress
begin made, but CPU is pegged, Memory use goes up and down around 800MB but absolutely 0
IO reads or writes.
It seems "stuck" at this
Removing old fragment root/parent configuration
5,461
99%
My question ... is the concept even valid ? Is it expected that a backup from
one system can be restored onto another ?
If not what is the recommend way to move data from one ML server to another ?
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Lee, David
Sent: Sunday, January 17, 2010 8:54 AM
To: General@developer.marklogic.com
Subject: [MarkLogic Dev General] Datase size, Comunity edition,EC2& Restore
and reindexing question
I noticed recently that ML is supported on Amazon EC2, this Is an exciting
possibility.
As an experiment to see if I can get my experimental database to run on EC2 I am trying
to load it into a Community edition license EC2 instance (have yet to get approval to
purchase a "Standard License" for EC2).
I have a few questions .
1) License size restrictions.
Prior editions before 4.1.4 I noticed that license size was related to "Index
Size" .. or atleast that’s how it seemed.
The same size XML would use different % of my license depending on what
indexing options I selected.
It doesn’t appear to do that anymore ... Is community edition licensed based on
content or index size or both ?
That is, is it possible to decrease the size for licensing purposes by turning
off various search features ?
2) Backup/Restore
I tried first to load the data from XML directly using my program that uses
XCC. I have about 26 GB of XML data
across 10 different data sources (and several million documents). Loading
with XML directly is cumbersome, and I ran into lots of problems trying to load
it to EC2, network problems would cause a batch load to abort and I'd have to
start over.
So instead I tried doing Backup of my master DB, then rsyncing the backup to
EC2 then doing a restore.
I also tried the same thing to a desktop (local) ML community edition license
server just to test.
This worked fine ... Except I know I have too much data (26G) so I turned off
many of the search options like 2 letter searches etc. On both my desktop and
my EC2 instance after restoring the 26G from disk ML went into a re-indexing
and refragmenting phase (expected) ... and is showing 254% above license use
(also expected).
What was NOT expected is that 48 hours later it's still reindexing with no end
in sight.
ML is pegging the CPU and predicts it will be done in 5-10 minutes ... for 2
days now.
Both in my EC2 and desktop instances .. so I know this is not just an EC2 issue.
I want to let this run to completion to see if I can get the data set under
community edition size, so I can at least prove the concept to my manager and
try to justify the concept of an EC2 ML server.
But why is it taking so long ? Is this expected ? Does it have anything to do
with the license ?
That is, because I'm over license size is it just going to run forever ? or
will it eventually complete?
Any clues on how long this is going to take ?
If this has nothing to do with licensing, this is a huge problem if it can take
days to recover from a restore ...
what would happen in production if I had to restore from backup or transfer
data from one server to another ? The server would be offline for days ?
----------------------------------------
David A. Lee
Senior Principal Software Engineer
Epocrates, Inc.
d...@epocrates.com<mailto:d...@epocrates.com>
812-482-5224
_______________________________________________
General mailing list
General@developer.marklogic.com
http://xqzone.com/mailman/listinfo/general
