Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Stefan Ring]

> I believe this is related to an oddity of the clojure compiler: it
> syncs every time it writes a class file to disk. This appears to be
> necessary for reasons that escape me. (One might naively assume that a
> simple flush() would be enough; but that was not so when I stumbled
> across this myself a year or so ago.)
>
> In any event, it does not surprise me in the least that a storm of
> sync calls would cause a mechanical hard disk to grovel a bit, while
> presenting

The perfect use case for .

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Raju Bitter]

Thanks for all your answers. Makes me feel even better about the money
I spent for my SSD.

- Raju

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Laurent PETIT]

Le 17 juil. 2012 à 02:04, Softaddicts  a écrit :

> Hi, Raju,
>
> We often AOT build here and we dropped standard disks a while ago.
> It brought us down to a build time around 30 seconds before it was several
> minutes.
>
> We have a mix of Clojure and Java code.
>
> Would not get back to a standard drive for all the $ in the world.
> Jar packaging also benefits from an SSD and we have some huge targets
> to build.

So in a way, looks like. I genuinely contributed to expand SSD market
share ! Time for me to reach vendors and get my money back ;)
>
> Synching systematically seems odd however :)
>
>
> Luc P.
>
>
>> I'm a bit concerned that it's a hard disk problem, but I've checked
>> the disk, tested other build processes using Ant, and don't see a
>> similar effect.
>>
>> -- Raju
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with your 
>> first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>>
> --
> Softaddicts sent by ibisMail from my ipad!
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Softaddicts]

Hi, Raju,

We often AOT build here and we dropped standard disks a while ago.
It brought us down to a build time around 30 seconds before it was several
minutes.

We have a mix of Clojure and Java code.

Would not get back to a standard drive for all the $ in the world.
Jar packaging also benefits from an SSD and we have some huge targets
to build.

Synching systematically seems odd however :)


Luc P.


> I'm a bit concerned that it's a hard disk problem, but I've checked
> the disk, tested other build processes using Ant, and don't see a
> similar effect.
> 
> -- Raju
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> 
--
Softaddicts sent by ibisMail from my ipad!

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Laurent PETIT]

Hi,

Le 17 juil. 2012 à 01:05, Craig  a écrit :

>
> I'm happy to now be aware of this issue.
> In amongst other (software, hardware) changes, I migrated a reasonably
> sized clojure application from 1.2 and 1.3

The addition of the call to sync() in the compiler predates 1.2 (1.2
was released in 2010, the call to sync was added in early 2009), so I
doubt the problem you saw between the jump from 1.2 to 1.3 is related
to the behavior this thread is about. Sorry.

> and compile times ballooned. It took me a while to realise it was a
> clojure issue.
>
> On Jul 17, 6:26 am, Andy Fingerhut  wrote:
>> There are links to older discussions on this topic in the description of 
>> ticket CLJ-703:
>>
>> http://dev.clojure.org/jira/browse/CLJ-703
>>
>> Also proposed patches to Clojure, although I don't know whether some of 
>> those may lead to incorrect behavior.
>>
>> Andy
>>
>> On Jul 16, 2012, at 12:48 PM, Raju Bitter wrote:
>>
>>
>>
>>
>>
>>
>>
>>> Thanks a lot for your answers. That explains the behavior I'm seeing.
>>
>>> - Raju
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Craig]

I'm happy to now be aware of this issue.
In amongst other (software, hardware) changes, I migrated a reasonably
sized clojure application from 1.2 and 1.3
and compile times ballooned. It took me a while to realise it was a
clojure issue.

On Jul 17, 6:26 am, Andy Fingerhut  wrote:
> There are links to older discussions on this topic in the description of 
> ticket CLJ-703:
>
> http://dev.clojure.org/jira/browse/CLJ-703
>
> Also proposed patches to Clojure, although I don't know whether some of those 
> may lead to incorrect behavior.
>
> Andy
>
> On Jul 16, 2012, at 12:48 PM, Raju Bitter wrote:
>
>
>
>
>
>
>
> > Thanks a lot for your answers. That explains the behavior I'm seeing.
>
> > - Raju

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Laurent PETIT]

Hello,

Le 16 juil. 2012 à 22:52, David Powell  a écrit :

In my opinion, on-disk consistency isn't and wasn't a goal.  And the
spamming of calls to sync does nothing other than make compilation
ridiculously slow on  file systems that are slow at sync.

sync should not have any user visible effects.  It just seems to me to be a
bit of voodoo code that should be removed.  I always comment it out.

Is there any test case that fails without it?

Search no longer, I think I'm responsible for the addition of this sync()
call.

When I was working on the Builder of Counterclockwise for Eclipse, I was
having weird issues related to the fact that calling the compile function
did not guarantee that when compile() returned, the file was immediately
visible and entirely written down (accessible for read). So as far as I
remember, this was a kind of concurrency problem, not solved despite the
existing call to flush(), and the only guarantee for the behavior was in
calling sync(). We did not expect the additional overhead in other
usecases, tho, and of course I'm annoyed to hear that people have problems
with this :-(

 --
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[David Powell]

In my opinion, on-disk consistency isn't and wasn't a goal.  And the
spamming of calls to sync does nothing other than make compilation
ridiculously slow on  file systems that are slow at sync.

sync should not have any user visible effects.  It just seems to me to be a
bit of voodoo code that should be removed.  I always comment it out.

Is there any test case that fails without it?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Andy Fingerhut]

There are links to older discussions on this topic in the description of ticket 
CLJ-703:

http://dev.clojure.org/jira/browse/CLJ-703

Also proposed patches to Clojure, although I don't know whether some of those 
may lead to incorrect behavior.

Andy

On Jul 16, 2012, at 12:48 PM, Raju Bitter wrote:

> Thanks a lot for your answers. That explains the behavior I'm seeing.
> 
> - Raju

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Raju Bitter]

Thanks a lot for your answers. That explains the behavior I'm seeing.

- Raju

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Ben Smith-Mannschott]

On Mon, Jul 16, 2012 at 9:17 PM, Ben Smith-Mannschott
 wrote:
> On Mon, Jul 16, 2012 at 5:21 PM, Raju Bitter  
> wrote:
>> I've checked out the Clojure source code, and build the JAR using the
>> Ant command. I'm seeing a strange effect, where the compile time on my
>> normal hard disk takes almost 4 min (with most the time being spent in
>> the "compile-clojure" task, and doing the same build process on my SSD
>> taking 30-35s.
>>
>> I've never seen any similar behavior on my machine, normally build
>> processes take 10% more when they run on the HDD vs. the SSD. Has
>> anyone else experienced the same effect? How long does it take to run
>> the "ant build" on your machines?
>>
>> Thanks,
>> Raju
>
> I believe this is related to an oddity of the clojure compiler: it
> syncs every time it writes a class file to disk. This appears to be
> necessary for reasons that escape me. (One might naively assume that a
> simple flush() would be enough; but that was not so when I stumbled
> across this myself a year or so ago.)
>
> In any event, it does not surprise me in the least that a storm of
> sync calls would cause a mechanical hard disk to grovel a bit, while
> presenting

(new keyboard, sorry)

... while presenting little difficulty for an SSD.


static public void writeClassFile(String internalName, byte[]
bytecode) throws IOException{

... skip a bit, brother ...

try
{
cfs.write(bytecode);
cfs.flush();
cfs.getFD().sync(); // <--
}
finally
{
cfs.close();
}
}

// ben

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Ben Smith-Mannschott]

On Mon, Jul 16, 2012 at 5:21 PM, Raju Bitter  wrote:
> I've checked out the Clojure source code, and build the JAR using the
> Ant command. I'm seeing a strange effect, where the compile time on my
> normal hard disk takes almost 4 min (with most the time being spent in
> the "compile-clojure" task, and doing the same build process on my SSD
> taking 30-35s.
>
> I've never seen any similar behavior on my machine, normally build
> processes take 10% more when they run on the HDD vs. the SSD. Has
> anyone else experienced the same effect? How long does it take to run
> the "ant build" on your machines?
>
> Thanks,
> Raju

I believe this is related to an oddity of the clojure compiler: it
syncs every time it writes a class file to disk. This appears to be
necessary for reasons that escape me. (One might naively assume that a
simple flush() would be enough; but that was not so when I stumbled
across this myself a year or so ago.)

In any event, it does not surprise me in the least that a storm of
sync calls would cause a mechanical hard disk to grovel a bit, while
presenting

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Craig Brozefsky]

Raju Bitter  writes:

> I've checked out the Clojure source code, and build the JAR using the
> Ant command. I'm seeing a strange effect, where the compile time on my
> normal hard disk takes almost 4 min (with most the time being spent in
> the "compile-clojure" task, and doing the same build process on my SSD
> taking 30-35s.

Yah, it's called "a fuckton of IO". 8^)

I also had a huge drop in time going from sata2 SSD to sata3 SSD.  In that
case it was also a change in manufacturer of the SSD.  lein startups are
way faster, as is everything else in general.  

Current dev machine is a mid-2012 Macbook Air with 8g RAM, dual core i7
@2ghz, and of course the sata3 SSD.  It's running Arch Linux (latest
installer worked damn near flawlessly) and is blazing fast.  My x100e
Thinkpad was not bad either for it's size/price/weight ratio.

-- 
Craig Brozefsky 
Premature reification is the root of all evil

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Raju Bitter]

I've checked out the Clojure source code, and build the JAR using the
Ant command. I'm seeing a strange effect, where the compile time on my
normal hard disk takes almost 4 min (with most the time being spent in
the "compile-clojure" task, and doing the same build process on my SSD
taking 30-35s.

I've never seen any similar behavior on my machine, normally build
processes take 10% more when they run on the HDD vs. the SSD. Has
anyone else experienced the same effect? How long does it take to run
the "ant build" on your machines?

Thanks,
Raju

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Much longer build time for Clojure on HDD vs. SSD (4 min vs 30s)

[Raju Bitter]

I'm a bit concerned that it's a hard disk problem, but I've checked
the disk, tested other build processes using Ant, and don't see a
similar effect.

-- Raju

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en