Re: innodb and raw disks
Christian Jaeger wrote: Just to the record: I've written a small program too, and it reports correct sizes for normal disk devices under both linux 2.2 and 2.4, as long as the partition sizes don't exceed the 2GB limit (I don't have large file support, since I use debian potato with just the necessary packages upraded for running 2.4). I've NOT tried with *unbuffered raw* partitions, however (linux 2.4 supports both partition access with and without buffering), I don't know yet how to setup these. database sql query table This is getting interesting! I compiled your program on my home machine (Redhat 6.2) and it gave me the same results that you reported. However, I tried it on the machine at work that I was experimenting with yesterday, and it returned 0 for each partition. Accordingly, I copied my own program to the home machine, and tried that: it reported the correct size for the partitions. The difference? The home machine is running kernel 2.4.3, the machine at work is running 2.2.16. Looks like lseek (and presumably the underlying structures and code) must have been substantially changed. Translated into real terms, this probably means that InnoDB raw disks are much more likely to work under Redhat 7.1+ than Redhat 6.2 (I will verify this as a matter of urgency and report back.) - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb and raw disks
database table sql query OK, here's where I've got to. 1) After fiddling about with fdisk, disk geometry and kernel boot parms, I've managed to define a partition which appears to lseek as exactly 1000M (this was fairly horrible: if anyone wants to replicate this step, I'll go over the process, but just at the moment I'm trying not to think about it!) 2) I start mysql, and get the following: 010719 15:08:17 mysqld started Data file 0 is 1048576000 bytes InnoDB: Log file /usr/local/var/iblogs/ib_logfile0 did not exist: new to be created InnoDB: Log file /usr/local/var/iblogs/ib_logfile1 did not exist: new to be created InnoDB: Log file /usr/local/var/iblogs/ib_logfile2 did not exist: new to be created InnoDB: Cannot initialize created log files because InnoDB: data files are corrupt, or new data files were InnoDB: created when the database was started previous InnoDB: time but the database was not shut down InnoDB: normally after that. 010719 15:08:32 Can't init databases InnoDB: Setting log file /usr/local/var/iblogs/ib_logfile0 size to 52428800 InnoDB: Setting log file /usr/local/var/iblogs/ib_logfile1 size to 52428800 InnoDB: Setting log file /usr/local/var/iblogs/ib_logfile2 size to 52428800 010719 15:08:32 mysqld ended 3) I zapped the entire mysql database directory (/usr/local/var), remade the directories, and reinitialised the mysql database (using /usr/local/bin/mysql_install_db). I then started mysql, and got the same result as in (2). I guess that it's thinking the tablespace is already initialised. Heikki, Christian, any thoughts? - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb and raw disks
At 15:08 Uhr +0100 19.7.2001, Peter Duffy wrote: database table sql query I guess that it's thinking the tablespace is already initialised. Yes, probably: it's the same if you have an ibdata file and no log files and start mysql-innodb. I've noticed this while setting up innodb with filesystem based tablespaces and had given wrong paths to the logfiles - so mysql did quit, I restarted it with correct paths to log directories, and it quit again because of 'corrupt data files'. Removing the data file helped. (PS. it seems that the innodb_log_group_home_dir and innodb_log_arch_dir options require absolute paths (or relative to cwd perhaps). It took me some time to realize this :o].) Christian. - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb and raw disks
Hi! Ok, I will add a special 'raw' keyword to 3.23.40, like: innodb_data_file_path = hdd1:2000Mraw Then it is enough that the raw disk partition is bigger than 2000 x 1024 x 1024 bytes. InnoDB will only use the specified amount of the partition from the low end, and will also skip the size check at startup. Peter, if you compile yourself, you can just try commenting out the size check on about line 367 of mysql/innobase/srv/srv0start.c Your disk partition should just be bigger than the size (1M = 1024 x 1024 bytes) you specify in innodb_file_path. What happens now? Regards, Heikki http://www.innodb.com -Original Message- From: Peter Duffy [EMAIL PROTECTED] To: Heikki Tuuri [EMAIL PROTECTED] Date: Tuesday, July 17, 2001 8:27 PM Subject: Re: innodb and raw disks Hi Heikki, You mean lseek? As far as I'm aware, fseek doesn't report the file size? (the code that I've been looking at in os0file.c definitely uses lseek.) Using the current disk, I can't make a partition which is *precisely* a multiple of 1M - the cylinder geometry doesn't allow it (correct me if I'm wrong, someone! I'm using fdisk, and if I specify 100M, I get 104391 blocks.) Does it have to be a multiple of 1M to satisfy the size-checking? I wrote a little program to experiment with lseek in isolation: it reports 0 bytes for both /dev/hdd and /dev/hdd1 (whatever size I make the latter.) I also tried mapping /dev/raw/raw1 over /dev/hdd1: the program still gives 0 bytes. (As a control, I tested the program against some regular files: for these, it reports the correct sizes.) Is it likely to make any difference if I put a BSD disklabel on /dev/hdd1? (I'm not even sure I can do that in an x86 environment: fdisk understands disklabels, but doesn't appear to be able to create them.) Rather than disable the file size check, would it be possible to check the file type, and for block devices substitute some other check? (Currently not sure what: maybe look up the device in /proc/partitions?) Regards, Peter Heikki Tuuri wrote: Hi! You should make an unformatted partition whose size is divisible by 1 MB. If fseek does not understand the partition size, we can disable the size check from InnoDB code. Please report if you succeed. If not, I change InnoDB-3.23.40 so that you can disable the file size check. I haver not tested raw disks on Unix. Other problems may also arise. For example, what does Linux do if fsync is called on a raw disk partition? I hope Linux ignores such OS calls. Regards, Heikki http://www.innodb.com Copied message: . fdisk /dev/hdd and set up partition 1.Then try using hdd1. Peter Duffy wrote: Hi, In the InnoDB section of the MySQL manual, it mentions that InnoDB can be used with raw disks, rather than data files. I've been trying to get this to work under Linux (RedHat 6.2), but so far without success. I installed a new slave disk on the second IDE bus, brought up the system and verified that the disk was available and partitionable at /dev/hdd. Then, in /etc/my.cnf, I pointed innodb_data_home_dir and innodb_data_file_path at /dev/hdd, with a size calculated from the autodetected values as reported in /var/log/messages. I then started mysql: it reported that the given size was not the same as the actual file size for /dev/hdd, and ended. I delved into the code and ran a few experiments: the problem appears to be that the code uses the lseek system call to determine the file size, and this reports 0 for the /dev/hdd device special file. I tried using a raw device grafted onto the /dev/hdd device, but same result. Has anyone else tried this under Linux and, if so, have they got it to work? (If not, do we have a route towards getting it to work?) - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php-- Gerald L. [EMAIL PROTECTED] - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: innodb and raw disks
database,sql,query,table
Just to the record: I've written a small program too, and it reports
correct sizes for normal disk devices under both linux 2.2 and 2.4,
as long as the partition sizes don't exceed the 2GB limit (I don't
have large file support, since I use debian potato with just the
necessary packages upraded for running 2.4). I've NOT tried with
*unbuffered raw* partitions, however (linux 2.4 supports both
partition access with and without buffering), I don't know yet how to
setup these.
lseek returns 0 if there's no partition on the disk for the device
being opened.
#include iostream
#include sys/types.h
#include fcntl.h
#include unistd.h
#include errno.h
#include string.h
int main(int argn, char * args[]) {
for (int i=1;iargn;i++) {
int fd= open (args[i], O_RDONLY);
if (fd!=-1) {
off_t eof = lseek (fd, 0, SEEK_END);
if (eof != -1)
cout Eof of ` args[i] ' is at: eof \n;
else
cerrCould not lseek `args[i]': strerror(errno)\n;
if (close (fd)==-1)
cerr ??\n;
} else {
cerrCould not open `args[i]': strerror(errno)\n;
}
}
}
./lseektest /dev/hda*
Could not lseek `/dev/hda': Value too large for defined data type
Could not lseek `/dev/hda1': Value too large for defined data type
Eof of `/dev/hda10' is at: 0
Eof of `/dev/hda11' is at: 0
Eof of `/dev/hda12' is at: 0
Eof of `/dev/hda13' is at: 0
Eof of `/dev/hda14' is at: 0
Eof of `/dev/hda15' is at: 0
Eof of `/dev/hda16' is at: 0
Eof of `/dev/hda17' is at: 0
Eof of `/dev/hda18' is at: 0
Eof of `/dev/hda19' is at: 0
Eof of `/dev/hda2' is at: 1024
Eof of `/dev/hda20' is at: 0
Eof of `/dev/hda3' is at: 197406720
Could not lseek `/dev/hda4': Value too large for defined data type
Could not lseek `/dev/hda5': Value too large for defined data type
Could not lseek `/dev/hda6': Value too large for defined data type
Eof of `/dev/hda7' is at: 682665984
Could not lseek `/dev/hda8': Value too large for defined data type
Eof of `/dev/hda9' is at: 0
root@pflanze raw\ partitions# fdisk -l /dev/hda
Disk /dev/hda: 255 heads, 63 sectors, 2110 cylinders
Units = cylinders of 16065 * 512 bytes
Device BootStart EndBlocks Id System
/dev/hda1 * 1 383 3076416b Win95 FAT32
/dev/hda2 384 1659 10249470f Win95 Ext'd (LBA)
/dev/hda3 1660 1683192780 82 Linux swap
/dev/hda4 1684 2110 3429877+ 83 Linux
/dev/hda5 384 766 3076416b Win95 FAT32
/dev/hda6 767 1149 3076416b Win95 FAT32
/dev/hda7 1150 123266b Win95 FAT32
/dev/hda8 1233 1659 3429846b Win95 FAT32
root@pflanze raw\ partitions#
Cheers,
Christian.
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail [EMAIL PROTECTED]
To unsubscribe, e-mail [EMAIL PROTECTED]
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
