Re: tools to improve harddisk performance by short-stroking?
I noticed I can find out files that were never accessed after the 1st month I installed my debian without any accounting mechanism but only by atime. In fact, if I pick up a random file on my system, the chance it was never accessed since after 1month of debian installation is a bit higher than it was. On 2010年10月08日 12:42, Tom Furie wrote: > How do you determine the atime without accessing the file Checking a file's atime is not considered accessing it. almust...@orphalese:~$ find Music/ -atime +147 Music/Flamenco. Una antologia de cante y baile andaluces - Poema de la guitarra (rito y geografia del Toque) - Vol. 1 de 4.avi Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/06-hans_zimmer-hes_killed_the_dog_again.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/08-hans_zimmer-not_in_blood_but_in_bond.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/10-hans_zimmer-panic_sheer_bloody_panic.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/07-hans_zimmer-marital_sabotage.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/05-hans_zimmer-data_data_data.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/09-hans_zimmer-ah_putrefaction.mp3 almust...@orphalese:~$ sleep 10 almust...@orphalese:~$ find Music/ -atime +147 Music/Flamenco. Una antologia de cante y baile andaluces - Poema de la guitarra (rito y geografia del Toque) - Vol. 1 de 4.avi Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/06-hans_zimmer-hes_killed_the_dog_again.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/08-hans_zimmer-not_in_blood_but_in_bond.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/10-hans_zimmer-panic_sheer_bloody_panic.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/07-hans_zimmer-marital_sabotage.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/05-hans_zimmer-data_data_data.mp3 Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/09-hans_zimmer-ah_putrefaction.mp3 -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4cb55270.9010...@realss.com
Re: tools to improve harddisk performance by short-stroking?
On 2010年10月10日 05:45, Andrei Popescu wrote: > IMVHO, I think at least part of the speed increase is based on the fact > that the head(s) *never* travel to the "back" of the harddisk. > A brief search found most of the files on my harddisks are *never* accessed (since the Debian system installation).
Re: tools to improve harddisk performance by short-stroking?
On Jo, 07 oct 10, 23:06:52, Zhang Weiwu wrote: > Finish reading the article it makes obvious to me that, if this > "technology" is really so powerful, it should have already been > implemented in OSes, like Linux, without necessarily abandoning the slow > part of hard-disk space but instead put rarely used data there. IMVHO, I think at least part of the speed increase is based on the fact that the head(s) *never* travel to the "back" of the harddisk. Regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: tools to improve harddisk performance by short-stroking?
On Fri, Oct 08, 2010 at 08:33:01AM +0800, Zhang Weiwu wrote: > To find out infrequently accessed file you need accounting. To find > rarely accessed file you only need to look at atime. How do you determine the atime without accessing the file, or keeping some sort of accounting? How do you update that accounting without accessing the file? What happens when a bunch of files are seen to be rarely accessed? What happens when you start using some of these files? How can you tell how often a file is accessed just by looking at the last time it was accessed? Just some food for thought. Cheers, Tom signature.asc Description: Digital signature
Re: tools to improve harddisk performance by short-stroking?
On 10/07/2010 07:33 PM, Zhang Weiwu wrote: On 2010年10月08日 04:03, Ron Johnson wrote: You'd need to add accounting complexity to the kernel (where would it put the accounting data?) I had been too brief, but if you read the article I referred to, it works best only in case you put rarely accessed file to the posterior, not 'infrequently accessed file' to the posterior. I'm sure that this is a case of ESL, but you apparently do not realize that "rarely" == "infrequently". Also, that still does not define what "posterior" of a RAID set is. -- Seek truth from facts. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4cae689b.50...@cox.net
Re: tools to improve harddisk performance by short-stroking?
On 2010年10月08日 04:03, Ron Johnson wrote: > > You'd need to add accounting complexity to the kernel (where would it > put the accounting data?) I had been too brief, but if you read the article I referred to, it works best only in case you put rarely accessed file to the posterior, not 'infrequently accessed file' to the posterior. To find out infrequently accessed file you need accounting. To find rarely accessed file you only need to look at atime. The difference is due to there are two factors slows down: disk spinning speed and head movement speed. If there is only the disk spin speed factor, then put infrequent file to the bottom helps a lot; if counting in the second factor, than only putting rarely accessed files at the bottom helps a lot. Suppose you have one infrequently accessed file at the posterior, and it's accessed, than the moment accessing it, access to every other file slows down as the head have to travel back afterwards, that's why the article suggest not to put any data at all to the posterior (even infrequent ones), while I think not putting any data at the posterior should have the same performance gain of putting only rarely accessed file there. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4cae66bd.4060...@realss.com
Re: tools to improve harddisk performance by short-stroking?
On 10/07/2010 10:06 AM, Zhang Weiwu wrote: Thanks to the national holiday (Beijing) I begin to read some article marked for free-time reading a few years ago. One of them is short stroking. http://www.tomshardware.com/reviews/short-stroking-hdd,2157.html The article is awfully long just to give a simple idea: by only using the first 20% of hard disk space you get about 3 to 4 times of hard-disk performance gain. The less you use your hard disk space, the more performance gain you get, you can get as much as 5 times faster HDD by using less than 10% of hard-disk. The article says, the only disadvantage of this method is you don't use the rest of the hard disk. Finish reading the article it makes obvious to me that, if this "technology" is really so powerful, it should have already been implemented in OSes, like Linux, without necessarily abandoning the slow part of hard-disk space but instead put rarely used data there. That's "disk file location optimization". Technically it cannot be too difficult to design file system tools in the way that it tends to put rarely-accessed files to the end of the partition that holds the file system. Says who? You'd need to add accounting complexity to the kernel (where would it put the accounting data?), and some utility would have to "know" the intimate details of your h/w (JBOD, fraction of a JBOD, lvm, RAID, what kind of RAID, how many spindles, etc, etc) and determine, on multi-drive devices like RAID and lvm, what exactly is the *interior* of a drive? It's a *really* complicated problem. -- Seek truth from facts. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4cae279a.2060...@cox.net
RE: tools to improve harddisk performance by short-stroking?
> > > > Original Message >From: zhangwe...@realss.com >To: debian-user@lists.debian.org >Subject: RE: tools to improve harddisk performance by short-stroking? >Date: Thu, 07 Oct 2010 23:06:52 +0800 > >>Thanks to the national holiday (Beijing) I begin to read some >article >>marked for free-time reading a few years ago. One of them is short >stroking. >> >>http://www.tomshardware.com/reviews/short-stroking-hdd,2157.html >> >>The article is awfully long just to give a simple idea: by only >using >>the first 20% of hard disk space you get about 3 to 4 times of >hard-disk >>performance gain. The less you use your hard disk space, the more >>performance gain you get, you can get as much as 5 times faster HDD >by >>using less than 10% of hard-disk. >> >>The article says, the only disadvantage of this method is you don't >use >>the rest of the hard disk. >> >>Finish reading the article it makes obvious to me that, if this >>"technology" is really so powerful, it should have already been >>implemented in OSes, like Linux, without necessarily abandoning the >slow >>part of hard-disk space but instead put rarely used data there. >> >>Technically it cannot be too difficult to design file system tools >in >>the way that it tends to put rarely-accessed files to the end of the >>partition that holds the file system. I am sure I have many files on >my >>computer with atime a few years ago (e.g. man page for hier; files I >>moved to Trash a few years ago), they can be moved to the bottom of >HDD >>space with a performance gain. >> >>Since the knowledge seems to be decades old (some references to this >>idea are in 1990s under different names), if the article's >>many-time-hdd-performance-gain is true, in the decade there should >>already been harddisk performance speedup tool that moves rarely >>accessed file to the bottom of file system once a week or so and get >me >>a 3 time boost of HDD performance. >> >>Question is how to find such a tool? A google search for such tool >end >>up in vain, that seem to suggest the tool doesn't exist, which means >the >>article is probably wrong. >> >>Note that I read about methods of partitioning harddisk to get >>short-stroking advantage, that wouldn't be optimal, because human >has to >>decide what are frequently accessed files (put them to first >partition) >>and what are rarely accessed files (put them on later partitions), >and a >>tool can do this thing much better than human. Besides, human doing >>partition optimization can only choose one of /var, /usr, /home, >/tmp >>and "swap" as "rarely accessed", the fact is none of them are rarely >>accessed, while most of them contain rarely accessed files. That's a >>very bad granularity compare to what specialized tool could do. >> >>Thanks in advance for comments! >> >> Just a guess but the short stroking advantage was likely for disk access in which the arm returns to some neutral position after access. OTOH if the access is the usual elevator algorithm in which the arm stays where it is assuming the next access will be nearby the short stroking will be ineffective. Larry >>-- >>To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org >>with a subject of "unsubscribe". Trouble? Contact >listmas...@lists.debian.org >>Archive: http://lists.debian.org/4cade20c.40...@realss.com >> >> -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/380-2201010471959...@netptc.net
tools to improve harddisk performance by short-stroking?
Thanks to the national holiday (Beijing) I begin to read some article marked for free-time reading a few years ago. One of them is short stroking. http://www.tomshardware.com/reviews/short-stroking-hdd,2157.html The article is awfully long just to give a simple idea: by only using the first 20% of hard disk space you get about 3 to 4 times of hard-disk performance gain. The less you use your hard disk space, the more performance gain you get, you can get as much as 5 times faster HDD by using less than 10% of hard-disk. The article says, the only disadvantage of this method is you don't use the rest of the hard disk. Finish reading the article it makes obvious to me that, if this "technology" is really so powerful, it should have already been implemented in OSes, like Linux, without necessarily abandoning the slow part of hard-disk space but instead put rarely used data there. Technically it cannot be too difficult to design file system tools in the way that it tends to put rarely-accessed files to the end of the partition that holds the file system. I am sure I have many files on my computer with atime a few years ago (e.g. man page for hier; files I moved to Trash a few years ago), they can be moved to the bottom of HDD space with a performance gain. Since the knowledge seems to be decades old (some references to this idea are in 1990s under different names), if the article's many-time-hdd-performance-gain is true, in the decade there should already been harddisk performance speedup tool that moves rarely accessed file to the bottom of file system once a week or so and get me a 3 time boost of HDD performance. Question is how to find such a tool? A google search for such tool end up in vain, that seem to suggest the tool doesn't exist, which means the article is probably wrong. Note that I read about methods of partitioning harddisk to get short-stroking advantage, that wouldn't be optimal, because human has to decide what are frequently accessed files (put them to first partition) and what are rarely accessed files (put them on later partitions), and a tool can do this thing much better than human. Besides, human doing partition optimization can only choose one of /var, /usr, /home, /tmp and "swap" as "rarely accessed", the fact is none of them are rarely accessed, while most of them contain rarely accessed files. That's a very bad granularity compare to what specialized tool could do. Thanks in advance for comments! -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4cade20c.40...@realss.com
