> tcook,
> ...
> Regarding the GUI, I don't know how to disable it.
> There are no virtual consoles, and unlike older
> versions of SunOS and Solaris, it comes up in XDM
> and there is no [apparent] way to get a shell
> without running gnome. I am sure that there is, but
> again, I come from the
Re pantzer5's suggestion:
Memory is not a big problem for ZFS, address space is. You may have to
give the kernel more address space on 32-bit CPUs.
eeprom kernelbase=0x8000
This will reduce the usable address space of user processes though.
---
Would you please verify that I understand co
It completed copying 191,xxx MB without issue in 17 hours and 40 minutes,
average transfer rate of 3.0MB/Sec. During the copy (At least the first hour
or so, and an hour in the middle), the machine was reasonably responsive. It
was jerky to a greater or lesser extent, but nothing like even the
On Mon, 1 Dec 2008, Karl Rossing wrote:
>
> I'm not familiar with fs design. There are probably compelling technical
> and compliance reasons not to do any of my suggestions.
Due to ZFS COW design, each re-compression requires allocation of a
new block. This has implications when snapshots and c
Could zfs be configured to use gzip-9 to compress small files or when
the system is idle..
When the system is busy or is handling a large file use lzjb.
Busy/Idle and large/small files would need to be defined somewhere.
Alternatively, write the file out using lzjb if the system is busy and
go
Tim writes:
> On Sat, Nov 29, 2008 at 11:06 AM, Ray Clark <[EMAIL PROTECTED]>wrote:
>
> > Please help me understand what you mean. There is a big difference between
> > being unacceptably slow and not working correctly, or between being
> > unacceptably slow and having an implementation pro
This has been an evolution, see defect.opensolaris.com's 5482 memory leak with
top. I have not run anything bug gzip-9 since fixing that by only running top
in one-time mode.
I will start the same copy with compress=off in about 45 minutes (Got to go do
an errand now). Glad to run tests.
--R
> This is about not having kernel threads completely
> lock out user (and other kernel) processes for
> undesirable lengths of time. It is about improving
> Solaris. It is about having more appropriate CPU
> sharing between all of the threads in the system,
> kernel and user. This is the root cau
> "t" == Tim <[EMAIL PROTECTED]> writes:
> "ic" == Ian Collins <[EMAIL PROTECTED]> writes:
> "mp" == Mattias Pantzare <[EMAIL PROTECTED]> writes:
t> My point is you're not looking at the bigger picture.
And you're not hearing this:
* 100KByte/s
* sudden slowdown 30x after 12
Unless something new develops, from my perspective this thread has served its
purpose. I don't want to drag it out and waste people's time. If any "late
readers" have insights that should be captured for the archive please add them.
Thank you all VERY much for the discussion, insights, suggest
> > I think Chris has the right idea. This would give more little opportunities
> > for user
> > processes to get a word in edgewise. Since the blocks are *obviously*
> > taking a
> > LONG time, this would not be a big hit efficiency in the bogged-down
> > condition.
> I still think you are exp
I'd agree there Ian, but this still sounds like a good idea to me. I don't
like any system becoming unresponsive, and this sounds like a good general
purpose tweak to help Solaris cope with worst case scenarios.
--
This message posted from opensolaris.org
___
I just want to interject here that I think, if memory serves me correctly, that
SPARC has been 64 bit for 10~15 years, and so had LOTS of address space to map
stuff. x86 brought a new restriction.
Regarding the practice of mapping files etc. into virtual memory that does not
exist, now I under
Re: I don't believe either are bundled. Search google for arcstat.pl and
nicstat.pl
--
Thanks. On a related note, there are at leaset 2 or 3 places proclaiming to
provide Solaris 10 packages. How do I know which ones are "safe", both in
terms of quality, and in terms of adhering to a consist
Ray Clark wrote:
> Re: gzip compression works a lot better now the compression is threaded.
> It's a shame userland gzip isn't!
>
> ---
> What does "now than" mean?
I didn't type that.
> I assume you mean the zfs / kernel gzip (right?) at some point became
> threaded. Is this in the past, or
Re: Experimentation will show that the compression ratio does not
increase much at -9 so it is not worth it when you are short on time
or CPU.
---
Yes, and a large part of my experiment was to understand the cost (time) vs.
compression ratio curve. lj?? only gave me 7%, which to me is not worth
Ray Clark wrote:
[some context would be nice for mail list subscribers]
> I think Chris has the right idea. This would give more little opportunities
> for user processes to get a word in edgewise. Since the blocks are
> *obviously* taking a LONG time, this would not be a big hit efficiency i
Re: gzip compression works a lot better now the compression is threaded.
It's a shame userland gzip isn't!
---
What does "now than" mean? I assume you mean the zfs / kernel gzip (right?) at
some point became threaded. Is this in the past, or in a kernel post 208.11 B2?
--Ray
--
This message
I think Chris has the right idea. This would give more little opportunities
for user processes to get a word in edgewise. Since the blocks are *obviously*
taking a LONG time, this would not be a big hit efficiency in the bogged-down
condition. It would however increase overhead in the well-be
Andewk8 at 11:39 on 11/29 said:
Solaris reports "virtual memory" as the sum of physical memory and page file -
so this is where your strange vmstat output comes from. Running ZFS stress
tests on a system with only 768MB of memory is not a good idea since ZFS uses
large amounts of memory for its
Chris Ridd wrote:
> On 30 Nov 2008, at 02:59, Bob Friesenhahn wrote:
>
>
>> On Sun, 30 Nov 2008, Ian Collins wrote:
>>
>>
>>> What did you expect? A 3GHz Opteron core takes about a minutes to
>>> attempt to compress a 1GB .mkv file. So your P3 would probably take
>>> between 5 and 10 minu
On 30 Nov 2008, at 02:59, Bob Friesenhahn wrote:
> On Sun, 30 Nov 2008, Ian Collins wrote:
>
>> What did you expect? A 3GHz Opteron core takes about a minutes to
>> attempt to compress a 1GB .mkv file. So your P3 would probably take
>> between 5 and 10 minutes. Now move that to the kernel and
On Sun, 30 Nov 2008, Ian Collins wrote:
> What did you expect? A 3GHz Opteron core takes about a minutes to
> attempt to compress a 1GB .mkv file. So your P3 would probably take
> between 5 and 10 minutes. Now move that to the kernel and your system
> will crawl. High gzip compressions are onl
For the record, what I said was the on x64, the default size of the UFS
segmap segment, which is the L1 cache for UFS reads and writes, is
64MB. Caches pages will be moved to a cache list in memory if segmap
fills up.
The message I was trying to convey is that the default size of the UFS
segmap
On Sat, Nov 29, 2008 at 8:29 PM, Ray Clark <[EMAIL PROTECTED]>wrote:
> Ref relling's 12:00 post:
> My system does not have arcstat or nicstat. But it is the B2 distribution.
> Would I expect these to be in the final distribution, or where do these
> come from?
> Thanks.
> --Ray
> --
>
I don't
Ref relling's 12:00 post:
My system does not have arcstat or nicstat. But it is the B2 distribution.
Would I expect these to be in the final distribution, or where do these come
from?
Thanks.
--Ray
--
This message posted from opensolaris.org
___
zfs-
Jeff,
Thank you for weighing in, as well as for the additional insight. It is good
to have confidence that I am on the right track.
I like your system ... alot. Got work to do for it to be as slick as a recent
Linux distribution, but you are working on a solid core and just need some
touch
Tim,
I am trying to look at the whole picture. I don't see any unwarranted
assumptions, although I know so little about Solaris and I extrapolated all
over the place based on general knowlege, sort of draping it around and over
what you all said. I see quite a few misconceptions in the thread
Ray Clark wrote:
> Now that it has come out of its slump, I can watch what it is working on vs.
> response. Whenever it is going through a folder with alot of incompressible
> stuff, it gets worse. .mp3 and .flac are horrible. .iso images and .gz and
> .zip files are bad. It is sinking again
> If you have more comments, or especially if you think I reached the wrong
> conclusion, please do post it. I will post my continuing results.
I think your conclusions are correct. The main thing you're seeing is
the combination of gzip-9 being incredibly CPU-intensive with our I/O
pipeline all
On Sun, Nov 30, 2008 at 01:10, Bob Friesenhahn
<[EMAIL PROTECTED]> wrote:
> On Sun, 30 Nov 2008, Mattias Pantzare wrote:
>>>
>>> Another big difference I have heard about is that Solaris 10 on x86 only
>>> uses something like 64MB of filesystem caching by default for UFS. This
>>> is
>>> different
Now that it has come out of its slump, I can watch what it is working on vs.
response. Whenever it is going through a folder with alot of incompressible
stuff, it gets worse. .mp3 and .flac are horrible. .iso images and .gz and
.zip files are bad. It is sinking again, but still works. It de
On Sat, Nov 29, 2008 at 6:31 PM, Ray Clark <[EMAIL PROTECTED]>wrote:
> Tim,
>
> I don't think we would really disagree if we were in the same room. I
> think in the process of the threaded communication that a few things got
> overlooked, or the wrong thing attributed.
>
> You are right that ther
Tim,
I don't think we would really disagree if we were in the same room. I think in
the process of the threaded communication that a few things got overlooked, or
the wrong thing attributed.
You are right that there are many differences. Some of them are:
- Tests done a year ago, I expect th
On Sun, 30 Nov 2008, Mattias Pantzare wrote:
>>
>> Another big difference I have heard about is that Solaris 10 on x86 only
>> uses something like 64MB of filesystem caching by default for UFS. This is
>> different than SPARC where the caching is allowed to grow. I am not sure if
>> OpenSolaris m
> I've got gzip9 running on a dataset right now
> with a nearly identical setup to what he had without
> issue. I'd say let's stop jumping to
> conclusions.
I curious if to know if you have ever tried dumping many gigs (100s at one
time) to your setup. Mine seemed fine when writing some files. It
On Sun, Nov 30, 2008 at 00:04, Bob Friesenhahn
<[EMAIL PROTECTED]> wrote:
> On Sat, 29 Nov 2008, Mattias Pantzare wrote:
>>
>> The big difference in memory usage between UFS and ZFS is that ZFS
>> will have all data it caches mapped in the kernel address space. UFS
>> leaves data unmapped.
>
> Anot
On Sat, Nov 29, 2008 at 3:47 PM, Ray Clark <[EMAIL PROTECTED]>wrote:
> If I shut down the Linux box, I won't have a host to send stuff to the
> Solaris box!
>
> Also, the Solaris box can only support 1024MB. I did have 1024MB in it at
> one time and had essentially the same performance. I might
On Sat, Nov 29, 2008 at 3:26 PM, Ray Clark <[EMAIL PROTECTED]>wrote:
> I get 15 to/from (I don't remember which) Linux LVM to a USB disk. It does
> seem to saturate there. I assume due to interrupt service time between
> transfers. I appreciate the contention for the IDE, but in a 3MB/Sec
> sys
On Sat, 29 Nov 2008, Mattias Pantzare wrote:
>
> The big difference in memory usage between UFS and ZFS is that ZFS
> will have all data it caches mapped in the kernel address space. UFS
> leaves data unmapped.
Another big difference I have heard about is that Solaris 10 on x86
only uses somethin
On Sat, Nov 29, 2008 at 22:19, Ray Clark <[EMAIL PROTECTED]> wrote:
> Pantzer5: Thanks for the "top" "size" explanation.
>
> Re: eeprom kernelbase=0x8000
> So this makes the kernel load at the 2G mark? What is the default, something
> like C00... for 3G?
Yes on both questions (i have not c
If I shut down the Linux box, I won't have a host to send stuff to the Solaris
box!
Also, the Solaris box can only support 1024MB. I did have 1024MB in it at one
time and had essentially the same performance. I might note that I had the
same problem with 1024MB, albiet with TOP eating memory
Thanks for the info about "Free Memory". That also links to another sub-thread
regarding kernel memory space.If disk files are mapped into memory space,
that would be a reason that the kernel could make use of address space larger
that virtual memory (RAM+Swap).
Regarding showing stuff as
I get 15 to/from (I don't remember which) Linux LVM to a USB disk. It does
seem to saturate there. I assume due to interrupt service time between
transfers. I appreciate the contention for the IDE, but in a 3MB/Sec system, I
don't think that it is my bottleneck, much less in a 100KByte/second
Pantzer5: Thanks for the "top" "size" explanation.
Re: eeprom kernelbase=0x8000
So this makes the kernel load at the 2G mark? What is the default, something
like C00... for 3G?
Are PCI and AGP space in there too, such that kernel space is 4G - (kernelbase
+ PCI_Size + AGP_Size) ? (Shot
From personal experience, 3-6MB/s is about what you should expect for NFS if
you're not using any kind of nvram write cache. With write cache, it's easy to
pretty much saturate 100MB/s ethernet.
And as others have said, ZFS needs RAM and plenty of it. I'd have thought 2GB
would be a sensible
> If the critical "working set" of VM pages is larger than available
> memory, then the system will become exceedingly slow. This is
> indicated by a substantial amount of major page fault activity.
> Since disk is 10,000 times slower than RAM, major page faults can
> really slow things down drama
On Sat, Nov 29, 2008 at 2:37 PM, Bob Friesenhahn <
[EMAIL PROTECTED]> wrote:
>
> To be more clear, memory which is claimed to be free is often actually
> still used for caching. Even if the virtual memory system has not
> mapped a VM page to a process, if a minor page fault occurs (due to an
> ac
On Sat, 29 Nov 2008, Ray Clark wrote:
> Regarding the cache, right now there is 150MB of free memory not
> being used by ANYBODY, so I don't think there is a shortage of
> memory for the ZFS cache... and 150MB >> 128K, or even a whole slew
To be more clear, memory which is claimed to be free is
On Sat, Nov 29, 2008 at 1:33 PM, Ray Clark <[EMAIL PROTECTED]>wrote:
> tcook, zpool iostat shows 1.15MB/sec. Is this averaged since boot, or a
> recent running average?
>
> The two drives ARE on a single IDE cable, however again, with a 33MB/sec
> cable rate and 8 or 16MB cache in the disk, 3 or
> Interestingly, the "size" fields under "top" add up to 950GB without getting
> to the bottom of the list, yet it
> shows NO swap being used, and 150MB free out of 768 of RAM! So how can the
> size of the existing processes
> exceed the size of the virtual memory in use by a factor of 2, and th
tcook, zpool iostat shows 1.15MB/sec. Is this averaged since boot, or a recent
running average?
The two drives ARE on a single IDE cable, however again, with a 33MB/sec cable
rate and 8 or 16MB cache in the disk, 3 or 4 MB/sec should be able to
time-share the cable without a significant impact
bfriesen,
Andrew brought up NVRAM by refering me to the following link:
Also, NFS to ZFS filesystems will run slowly under certain conditions
-including with the default configuration. See this link for more information:
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Cache
zpool status -v says "No known data errors" for both the root rpool (separate
non-mirrored 80GB drive) and my pool (mirrored 200GB drives).
It is getting very marginal (sluggish/unresponsive) again. Interesting, top
shows 20~30% cpu idle with most of remainder kernel. I wonder if everything i
On Sat, 29 Nov 2008, Ray Clark wrote:
> through the buffer. Whether the buffer is 128MB or 4GB, once the
> buffer is full the client will have to wait until something flows
> out to the disk. So the system runs at the speed of the slowest
> component. If accesses are done only once, caches d
> "t" == Tim <[EMAIL PROTECTED]> writes:
> "a" == andrew <[EMAIL PROTECTED]> writes:
> "re" == Richard Elling <[EMAIL PROTECTED]> writes:
t> ZFS uses ram, and plenty of it. That's the nature of COW.
a> Running ZFS stress tests on a system with only
a> 768MB of memory
On Sat, Nov 29, 2008 at 12:30 PM, Ray Clark <[EMAIL PROTECTED]>wrote:
> relling, Thank you, you gave me several things to look at. The one thing
> that sticks out for me is that I don't see why you listed IDE. Compared to
> all of the other factors, it is not the bottleneck by a long shot even
bfriesen,
Ultimately stuff flows in and stuff flows out. Data is not reused, so a cache
does not do anything for us. As a buffer, it is simply a rubber band, a FIFO.
So if the client wrote something real quick it would complete quickly. But if
it is writing an unlimited amount of data (like
relling, Thank you, you gave me several things to look at. The one thing that
sticks out for me is that I don't see why you listed IDE. Compared to all of
the other factors, it is not the bottleneck by a long shot even if it is a slow
transfer rate (33MB/Sec) by todays standards. What don't
Servo / mg,
I *have* noticed these effects on my system with lzjb, but they are minor.
Things are a little grainy, not smooth.
Eliminating the algorithm that exposes the shortfall in how the compression is
integrated into the system does not change the shortfall (See opensolaris.com
bug 5483)
> So you had a similar experience to what I had with an
> 800 MHz P3 and 768MB, all the way down to totally
> unresponsive. Probably 5 or 6 x the CPU speed
> (assuming single core) and 5 x the memory. This can
> only be a real design problem or bug, not just
> expected performance.
Our test mac
> Regarding the GUI, I don't know how to disable it.
> There are no virtual consoles, and unlike older
> versions of SunOS and Solaris, it comes up in XDM
> and there is no [apparent] way to get a shell
> without running gnome. I am sure that there is, but
> again, I come from the BSD/SunOS/Linux
On Sat, Nov 29, 2008 at 12:02 PM, Ray Clark <[EMAIL PROTECTED]>wrote:
> tcook,
>
> You bring up a good point. exponentially slow is very different from
> crashed, though they may have the same net effect. Also that other factors
> like timeouts would come into play.
>
> Regarding services, I am
Hakimian,
So you had a similar experience to what I had with an 800 MHz P3 and 768MB, all
the way down to totally unresponsive. Probably 5 or 6 x the CPU speed
(assuming single core) and 5 x the memory. This can only be a real design
problem or bug, not just expected performance.
Is there a
tcook,
You bring up a good point. exponentially slow is very different from crashed,
though they may have the same net effect. Also that other factors like
timeouts would come into play.
Regarding services, I am new to administering "modern" solaris, and that is on
my learning curve. My imm
On Sat, 29 Nov 2008, Ray Clark wrote:
>
> [1] You said "zfs uses large amounts of memory for its cache". If I
> understand correctly, it is not that it uses large amounts, it is
> that it uses all memory available. If this is an accurate picture,
> then it should be just as happy with 128MB as
Andrewk8,
Thanks for the information. I have some questions.
[1] You said "zfs uses large amounts of memory for its cache". If I
understand correctly, it is not that it uses large amounts, it is that it uses
all memory available. If this is an accurate picture, then it should be just
as ha
> I expect it will go SO SLOW, that some function somewhere is eventually
> going to fail/timeout. That system is barely usable WITHOUT
> compression. I hope at the very least you're disabling every single
> unnecessary service before doing any testing, especially the GUI.
>
> ZFS uses ram, and
The machine we tested with was a reasonably fast amd64 with 4Gigs of memory. I
don't know if I had disabled the graphical login yet, but it was probably not
even logged in via the console. Nothing else was running.
--
This message posted from opensolaris.org
_
For us, the machine became increasing unresponsive until it was
indistinguishable from a complete lockup. A top that was running showed triple
digit loads when it occasionally updated. We had to hit the reset button to get
the machine back. While this might simply be unacceptably slow, I was not
On Sat, Nov 29, 2008 at 11:06 AM, Ray Clark <[EMAIL PROTECTED]>wrote:
> Please help me understand what you mean. There is a big difference between
> being unacceptably slow and not working correctly, or between being
> unacceptably slow and having an implementation problem that causes it to
> eve
Please help me understand what you mean. There is a big difference between
being unacceptably slow and not working correctly, or between being
unacceptably slow and having an implementation problem that causes it to
eventually stop. I expect it to be slow, but I expect it to work. Are you
sa
Ray Clark wrote:
> I am [trying to] perform a test prior to moving my data to solaris and zfs.
> Things are going very poorly. Please suggest what I might do to understand
> what is going on, report a meaningful bug report, fix it, whatever!
>
> Both to learn what the compression could be, and
> I am [trying to] perform a test prior to moving my
> data to solaris and zfs. Things are going very
> poorly. Please suggest what I might do to understand
> what is going on, report a meaningful bug report, fix
> it, whatever!
>
> Both to learn what the compression could be, and to
> induce a
We looked at using gzip compression with zfs for a backup solution. Short
answer on our conclusion is it will not work. The gzip -9 compression just
can't sustain a large throughput of copying gigs of files. We ended up doing
non gzip compression and taking the hit on compression ratio (1.2x or
I am [trying to] perform a test prior to moving my data to solaris and zfs.
Things are going very poorly. Please suggest what I might do to understand
what is going on, report a meaningful bug report, fix it, whatever!
Both to learn what the compression could be, and to induce a heavy load to
76 matches
Mail list logo
