I think your problem is most likely to be badly sized files .... read on ....
On my Linux machine, I can backup a 1Gb file, which contains over 4 million items (a total of 617 Mb of data) in 2.5 seconds. That is pretty fast in anyone's language. So your problem isn't jbackup as such. In the above case, the 1Gb file is pre-fetched in memory so no disc I/O to retrieve the data, and the output is to /dev/null , so no disc I/O to save the output. The intention is just to show the processing power of jbackup when disc I/O isn't an issue. I did further tests on 3 different files like this file 1. Very large file, but with very little real data in it. file 2. Medium size file, perfectly sized. file 3. Medium size file, not well sized, some of the data is out of group (more explanation on this later) Without going into too much detail, file 1 and file 2 performed extremely well, with disc I/O being the limiting factor. When using jbackup with the output to /dev/null, so I only used disc I/O to retrieve data, I had a throughput of around 68 Mbytes / second. file 3 performed very badly. The file wasn't sized too badly, no more than one extra frame per group, but it made a serious impact. Using the same jbackup test, I only achieved around 5.5 Mbytes / second. Hence, I really think poorly sized files are the issue here. This will also impact your live on-line application as well. Now I'm not sure if you know what I mean by 'badly sized files' or why this should impact your backup so badly, so I'll assume you understand neither and will try to explain both issues. My apologies if you already know all this and it sounds condescending. First, what is a "good sized file" ? When you create a jBASE file (we'll ignore the new JR file type for now), you have to provide a modulo. This is the number of 4096-byte frames of data to allocate initially. So if you have 1 Gb of data, the modulo you use when you create the file would be approximately 1000000000 / 4096. Approximately. As there is overhead in the file and you are trying to avoid linked frames (more on this later), then the modulo would be more like 1000000000 / 3000. If you do this, then as you fill the file you would find it all fits neatly within the space allocated. Which is contiguous disc space. So far so good. When you use jbackup for this, it will traverse the file reading each 4096-byte frame consecutively. With a bit of luck, these consecutive frames will be physically consecutive sectors on the disc (usually, but not always). Most OS and disc hardware have look-ahead cache, and so because you are reading consecutive sectors from the disc, the read will be very very fast as much of it comes from the look-ahead cache which the disc performs on your behalf for precisely these occasions. Now lets go back to your file and add some more data. Not much more. Maybe just 20% more. Well, those 4096 byte frames will be pretty full of data already. Now you are trying to add more data than can fit into a 4096 byte frame. You can't fit a quart into a pint pot, as they say in England. So when the 4096 byte frame fills, jBASE will allocate another 4096 byte frame from the end of the file and link the two together. This means we now have 8192 bytes to fit the data in, of which the first 4096 bytes is in the initial space you allocated with the CREATE-FILE, and the second 4096 bytes is in "secondary" space, which we allocate at the end of the file. Although the file only has say 20% more data than was intended, the consequences for jbackup are disastrous, and it isn't really the fault of jbackup. What happens is this. jbackup will look at the first 4096 byte frame and read that in. It will see there is another 4096 byte frame linked to it, so it reads that in. The problem is that the second frame on disc is on a sector that is nowhere physically near the first frame on the disc. So the disc heads have to move. And it can't use the look-ahead cache for it. Whereas in the first case, jbackup scanned a "good sized" file and so the disc I/O mostly came from look-ahead cache with very little movement of the disc heads, in this second case we have to continually move the disc heads back and forth and can't make much use of the look-ahead cache. Hence the slow jbackup End of lecture ! What can you do about it? First, you could try re-sizing your files. This is always a good option as badly sized files only only impact your backup, but impact your on-line users as well. Secondly, if you have mirror discs (and if not, why not?), you could pause the database, split the mirror, and use a standard OS backup tool to back up the off-line version of the files. Whatever you do, if your files are on-line and active, please only use jbackup. The use of OS backup tools should only be used on jBASE files that are not currently being used. The advantage of using OS tools though is that they know nothing about linked frames and so will simply backup contiguous frames using the look-ahead cache effectively. Do you have the backup being stored on the same disc as your data? If so, then you have a double problem. First, if your disc crashes, you lose the original data AND you lose your backup. Secondly, when doing the jbackup, the disc has to work twice as hard as it not only reads the original data but has to write the backup on the same hard drive. Given the problems I explained about moving the disc heads, and it compounds the problem even more! One final note. You aren't using the new JR files that were introduced with jBASE release 5 are you? My initial testing with these file types gave extremely poor performance. This might have been fixed with a later release, but you should be wary. Hope this helps. Let me know if there is anything else I can clarify. Regards Greg > Use something other than HPUX? ;-) > > Why not just pipe the backup stream through 7zip or bzip2? > Bzip2 is pretty good when you can tell it there will be a lot of > data for it to compress. > > IN terms of speeding up, well that is as fast as jbackup can do > it to be honest. There are other ways to backup though. Why not > use transaction journaling and backup continuously? If you can > bring the system offline or use mirror breaking, then so long > as you are assured that there are no writes going on to the > database, then you can use raw backups such as disk imaging, > tar and so on, which are much faster as they are not formatted > backups. Also, jbackup isn't very sophisticated. > > Jim > > > -----Original Message----- > > From: [email protected] [mailto:[email protected]] On Behalf > > Of jawwi > > Sent: Wednesday, January 20, 2010 4:04 AM > > To: jBASE > > Subject: How to speed up jbackup > > > > Dear All, > > > > We are using jbackup for backing up data files through the following > > commands, > > > > find bnk -print | jbackup -S/home/itops2/stats -f/temenos/backups/ > > jbkp-17112009 -v 2> /home/itop > > s2/Aviion_backup_log > > > > jbackup is taking too long almost 3 hours to backup the entire data > > files which is 72 GB in size. > > > > Is there any method or procedure or switch thats speed up the process > > and also compress the backup file > > > > Stats: > > > > OS : Hp-UX 11i V2 > > DB : JBASE 5.0.16 > > > > > > Many Thanks in advance. > > > >
-- Please read the posting guidelines at: http://groups.google.com/group/jBASE/web/Posting%20Guidelines IMPORTANT: Type T24: at the start of the subject line for questions specific to Globus/T24 To post, send email to [email protected] To unsubscribe, send email to [email protected] For more options, visit this group at http://groups.google.com/group/jBASE?hl=en
