Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
On 5/15/17, 1:20 PM, "openssl-dev on behalf of Benjamin Kaduk via openssl-dev" 
 wrote:

> On a semi-related note, I want able to locate mann.h file either.


`man mmap` will list any headers needed for the mmap() declaration and flag 
values.
On the random OS X machine I have handy, it claims  is needed, 

and a /usr/include/sys/mman.h is present.


Thanks – I confirm that `man mmap` mentions /usr/include/sys/mman.h, and that 
file does exist (how could my first `find` miss it?!). 

This file does not contain MLOCK_ONFAULT though.

 

It has MAP_ANON:

 

/*

 * Mapping type

 */

#define MAP_FILE0x  /* map from file (default) */

#define MAP_ANON0x1000  /* allocated from memory, swap space */

#define MAP_ANONYMOUS   MAP_ANON

 

And as I already mentioned, it has a world-accessible (figuratively speaking :) 
/dev/zero that can be opened read/write.

 

Also, with #3455 applied the problem disappeared (thankfully :).

 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Benjamin Kaduk via openssl-dev 
On 05/15/2017 12:15 PM, Blumenthal, Uri - 0553 - MITLL wrote:
>
> On a semi-related note, I want able to locate mann.h file either.

`man mmap` will list any headers needed for the mmap() declaration and
flag values.
On the random OS X machine I have handy, it claims  is
needed, and a /usr/include/sys/mman.h is present.

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
Yes and yes. :-)

Regards,
Uri

Sent from my iPhone

> On May 15, 2017, at 13:18, Short, Todd  wrote:
> 
> s/want/wasn’t/ ?
> 
> So, no MLOCK_ONFAULT for you? That’s only included on Linux systems, which 
> makes sense if you’re on a Mac.
> --
> -Todd Short
> // tsh...@akamai.com
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 15, 2017, at 1:15 PM, Blumenthal, Uri - 0553 - MITLL 
>>  wrote:
>> 
>> My disk is SSD, but computer load is pretty high. Which probably explains 
>> that recovery doesn't take place in 200-400 seconds...
>> 
>> On a semi-related note, I want able to locate mann.h file either.
>> 
>> Regards,
>> Uri
>> 
>> Sent from my iPhone
>> 
>>> On May 15, 2017, at 13:09, Short, Todd  wrote:
>>> 
>>> The time of the hang actually seems dependent on the number of applications 
>>> running and your disk.
>>> 
>>> Since a large amount of memory becomes wired, there is very little 
>>> available for programs and the OS to use (in some instances I have seen 
>>> ~4MB non-wired memory). Things slow down due to swapping, etc. 
>>> 
>>> In my testing:
>>> 
>>> With almost no additional programs open, the hang-time is short, ~200 
>>> seconds.
>>> With a lot of programs open, the hang-time is increased, ~400 seconds; 
>>> twice as long. And the number of swapins is 25x and the swapouts is ~34x 
>>> the original test period.
>>> 
>>> This is on a machine with an SSD (late-2013 MBP)
>>> If you have a spinning HDD, the swapins and swapouts will be significantly 
>>> more expensive in terms of performance/time.
>>> 
>>> If you quit all your programs, (other than Terminal), I suspect the hang 
>>> may eventually recover; but if you have a hard disk that time might be 
>>> quite long.
>>> 
>>> --
>>> -Todd Short
>>> // tsh...@akamai.com
>>> // "One if by land, two if by sea, three if by the Internet."
>>> 
 On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL 
  wrote:
 
 I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I 
 could try it on Sierra 10.12.4, if you really expect it to make a 
 difference.
  
 In my case the hang is not for a short time. It lasts for more than 10 
 minutes, so I’m forced to interfere. For how long did it hang for you?
 — 
 Regards,
 Uri
  
  
 On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via 
 openssl-dev" >>> openssl-dev@openssl.org> wrote:
  
 We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a 
 short period of time with the unit-tests, but it eventually recovers. 
  
 What MacOS version are you running? I can try 10.12 later today.
 --
 -Todd Short
 // tsh...@akamai.com
 // "One if by land, two if by sea, three if by the Internet."
  
> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
>  wrote:  
>  
> Uri: 
>  
> Look at https://github.com/openssl/openssl/pull/3455
>  
> I limited the test that hung your machine to Linux.
>  
> Rich: this removes the OpenSSL_assert() you see.
>  
> --
> -Todd Short
> // tsh...@akamai.com
> // "One if by land, two if by sea, three if by the Internet."
>  
>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
>>  wrote:
>>  
>> It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
>> --
>> -Todd Short
>> // tsh...@akamai.com
>> // "One if by land, two if by sea, three if by the Internet."
>>  
>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>>>  wrote:
>>>  
>>> Todd> Yes, it’s likely this is due to the amount of memory available in 
>>> the machine. I tried to use reasonable values, but apparently not 
>>> reasonable enough
>>>  
>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
>>> stuff, besides these tests :).
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>> 
>>  
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
> 
>  
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
 
  
>>> 
> 


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Short, Todd via openssl-dev 
s/want/wasn’t/ ?

So, no MLOCK_ONFAULT for you? That’s only included on Linux systems, which 
makes sense if you’re on a Mac.
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 15, 2017, at 1:15 PM, Blumenthal, Uri - 0553 - MITLL  
> wrote:
> 
> My disk is SSD, but computer load is pretty high. Which probably explains 
> that recovery doesn't take place in 200-400 seconds...
> 
> On a semi-related note, I want able to locate mann.h file either.
> 
> Regards,
> Uri
> 
> Sent from my iPhone
> 
> On May 15, 2017, at 13:09, Short, Todd  > wrote:
> 
>> The time of the hang actually seems dependent on the number of applications 
>> running and your disk.
>> 
>> Since a large amount of memory becomes wired, there is very little available 
>> for programs and the OS to use (in some instances I have seen ~4MB non-wired 
>> memory). Things slow down due to swapping, etc. 
>> 
>> In my testing:
>> 
>> With almost no additional programs open, the hang-time is short, ~200 
>> seconds.
>> With a lot of programs open, the hang-time is increased, ~400 seconds; twice 
>> as long. And the number of swapins is 25x and the swapouts is ~34x the 
>> original test period.
>> 
>> This is on a machine with an SSD (late-2013 MBP)
>> If you have a spinning HDD, the swapins and swapouts will be significantly 
>> more expensive in terms of performance/time.
>> 
>> If you quit all your programs, (other than Terminal), I suspect the hang may 
>> eventually recover; but if you have a hard disk that time might be quite 
>> long.
>> 
>> --
>> -Todd Short
>> // tsh...@akamai.com 
>> // "One if by land, two if by sea, three if by the Internet."
>> 
>>> On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL 
>>> mailto:u...@ll.mit.edu>> wrote:
>>> 
>>> I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I could 
>>> try it on Sierra 10.12.4, if you really expect it to make a difference.
>>>  
>>> In my case the hang is not for a short time. It lasts for more than 10 
>>> minutes, so I’m forced to interfere. For how long did it hang for you?
>>> — 
>>> Regards,
>>> Uri
>>>  
>>>  
>>> On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via 
>>> openssl-dev" >>  on behalf of 
>>> openssl-dev@openssl.org > wrote:
>>>  
>>> We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a 
>>> short period of time with the unit-tests, but it eventually recovers. 
>>>  
>>> What MacOS version are you running? I can try 10.12 later today.
>>> --
>>> -Todd Short
>>> // tsh...@akamai.com 
>>> // "One if by land, two if by sea, three if by the Internet."
>>>  
 On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
 mailto:openssl-dev@openssl.org>> wrote:  
  
 Uri: 
  
 Look at https://github.com/openssl/openssl/pull/3455 
 
  
 I limited the test that hung your machine to Linux.
  
 Rich: this removes the OpenSSL_assert() you see.
  
 --
 -Todd Short
 // tsh...@akamai.com 
 // "One if by land, two if by sea, three if by the Internet."
  
> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
> mailto:openssl-dev@openssl.org>> wrote:
>  
> It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
> --
> -Todd Short
> // tsh...@akamai.com 
> // "One if by land, two if by sea, three if by the Internet."
>  
>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>> mailto:u...@ll.mit.edu>> wrote:
>>  
>> Todd> Yes, it’s likely this is due to the amount of memory available in 
>> the machine. I tried to use reasonable values, but apparently not 
>> reasonable enough
>>  
>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
>> stuff, besides these tests :).
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
>> 
>  
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
> 
  
 -- 
 openssl-dev mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
 
>>>  
>> 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
My disk is SSD, but computer load is pretty high. Which probably explains that 
recovery doesn't take place in 200-400 seconds...

On a semi-related note, I want able to locate mann.h file either.

Regards,
Uri

Sent from my iPhone

> On May 15, 2017, at 13:09, Short, Todd  wrote:
> 
> The time of the hang actually seems dependent on the number of applications 
> running and your disk.
> 
> Since a large amount of memory becomes wired, there is very little available 
> for programs and the OS to use (in some instances I have seen ~4MB non-wired 
> memory). Things slow down due to swapping, etc. 
> 
> In my testing:
> 
> With almost no additional programs open, the hang-time is short, ~200 seconds.
> With a lot of programs open, the hang-time is increased, ~400 seconds; twice 
> as long. And the number of swapins is 25x and the swapouts is ~34x the 
> original test period.
> 
> This is on a machine with an SSD (late-2013 MBP)
> If you have a spinning HDD, the swapins and swapouts will be significantly 
> more expensive in terms of performance/time.
> 
> If you quit all your programs, (other than Terminal), I suspect the hang may 
> eventually recover; but if you have a hard disk that time might be quite long.
> 
> --
> -Todd Short
> // tsh...@akamai.com
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL 
>>  wrote:
>> 
>> I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I could 
>> try it on Sierra 10.12.4, if you really expect it to make a difference.
>>  
>> In my case the hang is not for a short time. It lasts for more than 10 
>> minutes, so I’m forced to interfere. For how long did it hang for you?
>> — 
>> Regards,
>> Uri
>>  
>>  
>> On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" 
>>  wrote:
>>  
>> We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short 
>> period of time with the unit-tests, but it eventually recovers. 
>>  
>> What MacOS version are you running? I can try 10.12 later today.
>> --
>> -Todd Short
>> // tsh...@akamai.com
>> // "One if by land, two if by sea, three if by the Internet."
>>  
>>> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
>>>  wrote:  
>>>  
>>> Uri: 
>>>  
>>> Look at https://github.com/openssl/openssl/pull/3455
>>>  
>>> I limited the test that hung your machine to Linux.
>>>  
>>> Rich: this removes the OpenSSL_assert() you see.
>>>  
>>> --
>>> -Todd Short
>>> // tsh...@akamai.com
>>> // "One if by land, two if by sea, three if by the Internet."
>>>  
 On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
  wrote:
  
 It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
 --
 -Todd Short
 // tsh...@akamai.com
 // "One if by land, two if by sea, three if by the Internet."
  
> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>  wrote:
>  
> Todd> Yes, it’s likely this is due to the amount of memory available in 
> the machine. I tried to use reasonable values, but apparently not 
> reasonable enough
>  
> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
> stuff, besides these tests :).
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
 
  
 -- 
 openssl-dev mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>>> 
>>>  
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>> 
>>  
> 


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Short, Todd via openssl-dev 
The time of the hang actually seems dependent on the number of applications 
running and your disk.

Since a large amount of memory becomes wired, there is very little available 
for programs and the OS to use (in some instances I have seen ~4MB non-wired 
memory). Things slow down due to swapping, etc.

In my testing:

With almost no additional programs open, the hang-time is short, ~200 seconds.
With a lot of programs open, the hang-time is increased, ~400 seconds; twice as 
long. And the number of swapins is 25x and the swapouts is ~34x the original 
test period.

This is on a machine with an SSD (late-2013 MBP)
If you have a spinning HDD, the swapins and swapouts will be significantly more 
expensive in terms of performance/time.

If you quit all your programs, (other than Terminal), I suspect the hang may 
eventually recover; but if you have a hard disk that time might be quite long.

--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

On May 15, 2017, at 11:51 AM, Blumenthal, Uri - 0553 - MITLL 
mailto:u...@ll.mit.edu>> wrote:

I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try 
it on Sierra 10.12.4, if you really expect it to make a difference.

In my case the hang is not for a short time. It lasts for more than 10 minutes, 
so I’m forced to interfere. For how long did it hang for you?
—
Regards,
Uri


On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" 
mailto:openssl-dev-boun...@openssl.org> on 
behalf of openssl-dev@openssl.org> wrote:

We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short 
period of time with the unit-tests, but it eventually recovers.

What MacOS version are you running? I can try 10.12 later today.
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
mailto:openssl-dev@openssl.org>> wrote:

Uri:

Look at https://github.com/openssl/openssl/pull/3455

I limited the test that hung your machine to Linux.

Rich: this removes the OpenSSL_assert() you see.

--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
mailto:openssl-dev@openssl.org>> wrote:

It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
mailto:u...@ll.mit.edu>> wrote:

Todd> Yes, it’s likely this is due to the amount of memory available in the 
machine. I tried to use reasonable values, but apparently not reasonable enough

Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, 
besides these tests :).
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Blumenthal, Uri - 0553 - MITLL 
I’m tracking the current OpenSSL master only on El Capitan 10.11.6. I could try 
it on Sierra 10.12.4, if you really expect it to make a difference.

 

In my case the hang is not for a short time. It lasts for more than 10 minutes, 
so I’m forced to interfere. For how long did it hang for you?

— 

Regards,

Uri

 

 

On 5/15/17, 11:47 AM, "openssl-dev on behalf of Short, Todd via openssl-dev" 
 wrote:

 

We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short 
period of time with the unit-tests, but it eventually recovers. 

 

What MacOS version are you running? I can try 10.12 later today.

--

-Todd Short

// tsh...@akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
 wrote:  

 

Uri: 

 

Look at https://github.com/openssl/openssl/pull/3455

 

I limited the test that hung your machine to Linux.

 

Rich: this removes the OpenSSL_assert() you see.

 

--

-Todd Short

// tsh...@akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
 wrote:

 

It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...

--

-Todd Short

// tsh...@akamai.com

// "One if by land, two if by sea, three if by the Internet."

 

On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL  
wrote:

 

Todd> Yes, it’s likely this is due to the amount of memory available in the 
machine. I tried to use reasonable values, but apparently not reasonable enough

 

Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of stuff, 
besides these tests :).

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] 90-test_secmem.t hangs the machine for good

2017-05-15 Thread Short, Todd via openssl-dev 
We’ve been able to get some Macs (10.11.6, 8GB and 16GB) to hang for a short 
period of time with the unit-tests, but it eventually recovers.

What MacOS version are you running? I can try 10.12 later today.
--
-Todd Short
// tsh...@akamai.com
// "One if by land, two if by sea, three if by the Internet."

> On May 12, 2017, at 4:50 PM, Short, Todd via openssl-dev 
>  wrote:
> 
> Uri:
> 
> Look at https://github.com/openssl/openssl/pull/3455 
> 
> 
> I limited the test that hung your machine to Linux.
> 
> Rich: this removes the OpenSSL_assert() you see.
> 
> --
> -Todd Short
> // tsh...@akamai.com 
> // "One if by land, two if by sea, three if by the Internet."
> 
>> On May 12, 2017, at 4:46 PM, Short, Todd via openssl-dev 
>> mailto:openssl-dev@openssl.org>> wrote:
>> 
>> It’s trying to reserve 1<<34 bytes of memory… there goes your 16GB...
>> --
>> -Todd Short
>> // tsh...@akamai.com 
>> // "One if by land, two if by sea, three if by the Internet."
>> 
>>> On May 12, 2017, at 4:05 PM, Blumenthal, Uri - 0553 - MITLL 
>>> mailto:u...@ll.mit.edu>> wrote:
>>> 
>>> Todd> Yes, it’s likely this is due to the amount of memory available in the 
>>> machine. I tried to use reasonable values, but apparently not reasonable 
>>> enough
>>>  
>>> Yep. In case it matters, my machine has 16GB of RAM (and runs a ton of 
>>> stuff, besides these tests :).
>>> -- 
>>> openssl-dev mailing list
>>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
>>> 
>> 
>> -- 
>> openssl-dev mailing list
>> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev 
>> 
> 
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev