RE: [julia-users] Pkg "threadsafe" or interprocess mutex package

[David Anthoff]

Ah, interesting. I guess for my use case (global system wide lock) this doesn’t 
matter.

 

From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On 
Behalf Of Isaiah Norton
Sent: Friday, July 8, 2016 8:23 PM
To: julia-users@googlegroups.com
Subject: Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

 

Libuv mostly abstracts away the differences, but named pipes are released 
automatically when the last handle closes, whereas named sockets are associated 
with an inode and must be checked/cleaned-up manually.

 

On Thu, Jul 7, 2016 at 10:24 PM, David Anthoff <anth...@berkeley.edu 
<mailto:anth...@berkeley.edu> > wrote:

Yes, seems to work.

 

You need to use a different naming scheme on Windows, namely ``  
 .\\pipe\\yourname``, but then one can just 
use the normal julia socket functions for it. In particular, if one process 
listens on such a named pipe, calling ``listen`` from another process with the 
same name will give an error.

 

Actually, are there really different semantics? I haven’t come across any when 
I use such a named pipe on Windows for inter-process communication via the 
julia socket API…

 

From:  <mailto:julia-users@googlegroups.com> julia-users@googlegroups.com 
[mailto: <mailto:julia-users@googlegroups.com> julia-users@googlegroups.com] On 
Behalf Of Isaiah Norton
Sent: Thursday, July 7, 2016 5:34 PM


To: julia-users@googlegroups.com <mailto:julia-users@googlegroups.com> 
Subject: Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

 

Does that work on Windows? I briefly tried earlier and it didn't seem to work. 
IIUC Windows doesn't support Unix-style named sockets. (it does have named 
pipes, but the semantics are a bit different)

 

On Thu, Jul 7, 2016 at 1:07 PM, David Anthoff <anth...@berkeley.edu 
<mailto:anth...@berkeley.edu> > wrote:

Ah, I had actually come up with the same solution that Amit suggested in #7176 
and am using a named socket as a lock, that seems to work for now.

 

I’m also running into another race condition with respect to precompile, I 
opened a new issue for that:

 <https://github.com/JuliaLang/julia/issues/17320> 
https://github.com/JuliaLang/julia/issues/17320

 

Cheers,

David

 

From:  <mailto:julia-users@googlegroups.com> julia-users@googlegroups.com 
[mailto: <mailto:julia-users@googlegroups.com> julia-users@googlegroups.com] On 
Behalf Of Isaiah Norton
Sent: Thursday, July 7, 2016 6:52 AM
To:  <mailto:julia-users@googlegroups.com> julia-users@googlegroups.com
Subject: Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

 

See

https://github.com/JuliaLang/julia/issues/5622 and

https://github.com/JuliaLang/julia/issues/7176

 

If you are on Windows, LockFileEx is reliable and you could ccall it yourself. 
Probably simpler to handle the problem at a higher level for now though (i.e. 
run one script that checks installation *before* launching your analysis jobs).

 

 

On Wed, Jul 6, 2016 at 5:09 PM, David Anthoff <anth...@berkeley.edu 
<mailto:anth...@berkeley.edu> > wrote:

Hi,

 

I have a script that is using some packages. The script is set up such that if 
the package it is trying to use is not installed, it will do a ``Pkg.add`` to 
add the package before ``using`` it.

 

This script is triggered by a third process, and it might be triggered multiple 
times simultaneously, i.e. multiple instances of julia might run the same 
script at the same time. Therefore the second julia instance might issue the 
same ``Pkg.add`` command as the first instance, while the first instance is not 
done with the ``Pkg.add`` yet.

 

I assume the package manager per se is not set up to handle that situation 
gracefully?

 

If not, is there some package that allows me to have something like a 
mutex/lock between my julia processes, so that I can guard the code section 
that fiddles with packages and make sure only one julia instance at a time runs 
that code?

 

Thanks,

David

 

--

David Anthoff

University of California, Berkeley

 

http://www.david-anthoff.com

 

 

 

 

Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

[Isaiah Norton]

Libuv mostly abstracts away the differences, but named pipes are released
automatically when the last handle closes, whereas named sockets are
associated with an inode and must be checked/cleaned-up manually.

On Thu, Jul 7, 2016 at 10:24 PM, David Anthoff <anth...@berkeley.edu> wrote:

> Yes, seems to work.
>
>
>
> You need to use a different naming scheme on Windows, namely ``
> .\\pipe\\yourname``, but then one can just use the normal julia
> socket functions for it. In particular, if one process listens on such a
> named pipe, calling ``listen`` from another process with the same name will
> give an error.
>
>
>
> Actually, are there really different semantics? I haven’t come across any
> when I use such a named pipe on Windows for inter-process communication via
> the julia socket API…
>
>
>
> *From:* julia-users@googlegroups.com [mailto:julia-users@googlegroups.com]
> *On Behalf Of *Isaiah Norton
> *Sent:* Thursday, July 7, 2016 5:34 PM
>
> *To:* julia-users@googlegroups.com
> *Subject:* Re: [julia-users] Pkg "threadsafe" or interprocess mutex
> package
>
>
>
> Does that work on Windows? I briefly tried earlier and it didn't seem to
> work. IIUC Windows doesn't support Unix-style named sockets. (it does have
> named pipes, but the semantics are a bit different)
>
>
>
> On Thu, Jul 7, 2016 at 1:07 PM, David Anthoff <anth...@berkeley.edu>
> wrote:
>
> Ah, I had actually come up with the same solution that Amit suggested in
> #7176 and am using a named socket as a lock, that seems to work for now.
>
>
>
> I’m also running into another race condition with respect to precompile, I
> opened a new issue for that:
>
> https://github.com/JuliaLang/julia/issues/17320
>
>
>
> Cheers,
>
> David
>
>
>
> *From:* julia-users@googlegroups.com [mailto:julia-users@googlegroups.com]
> *On Behalf Of *Isaiah Norton
> *Sent:* Thursday, July 7, 2016 6:52 AM
> *To:* julia-users@googlegroups.com
> *Subject:* Re: [julia-users] Pkg "threadsafe" or interprocess mutex
> package
>
>
>
> See
>
> https://github.com/JuliaLang/julia/issues/5622 and
>
> https://github.com/JuliaLang/julia/issues/7176
>
>
>
> If you are on Windows, LockFileEx is reliable and you could ccall it
> yourself. Probably simpler to handle the problem at a higher level for now
> though (i.e. run one script that checks installation *before* launching
> your analysis jobs).
>
>
>
>
>
> On Wed, Jul 6, 2016 at 5:09 PM, David Anthoff <anth...@berkeley.edu>
> wrote:
>
> Hi,
>
>
>
> I have a script that is using some packages. The script is set up such
> that if the package it is trying to use is not installed, it will do a
> ``Pkg.add`` to add the package before ``using`` it.
>
>
>
> This script is triggered by a third process, and it might be triggered
> multiple times simultaneously, i.e. multiple instances of julia might run
> the same script at the same time. Therefore the second julia instance might
> issue the same ``Pkg.add`` command as the first instance, while the first
> instance is not done with the ``Pkg.add`` yet.
>
>
>
> I assume the package manager per se is not set up to handle that situation
> gracefully?
>
>
>
> If not, is there some package that allows me to have something like a
> mutex/lock between my julia processes, so that I can guard the code section
> that fiddles with packages and make sure only one julia instance at a time
> runs that code?
>
>
>
> Thanks,
>
> David
>
>
>
> --
>
> David Anthoff
>
> University of California, Berkeley
>
>
>
> http://www.david-anthoff.com
>
>
>
>
>
>
>

RE: [julia-users] Pkg "threadsafe" or interprocess mutex package

[David Anthoff]

Yes, seems to work.

 

You need to use a different naming scheme on Windows, namely `` 
.\\pipe\\yourname``  , but then one can just 
use the normal julia socket functions for it. In particular, if one process 
listens on such a named pipe, calling ``listen`` from another process with the 
same name will give an error.

 

Actually, are there really different semantics? I haven’t come across any when 
I use such a named pipe on Windows for inter-process communication via the 
julia socket API…

 

From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On 
Behalf Of Isaiah Norton
Sent: Thursday, July 7, 2016 5:34 PM
To: julia-users@googlegroups.com
Subject: Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

 

Does that work on Windows? I briefly tried earlier and it didn't seem to work. 
IIUC Windows doesn't support Unix-style named sockets. (it does have named 
pipes, but the semantics are a bit different)

 

On Thu, Jul 7, 2016 at 1:07 PM, David Anthoff <anth...@berkeley.edu 
<mailto:anth...@berkeley.edu> > wrote:

Ah, I had actually come up with the same solution that Amit suggested in #7176 
and am using a named socket as a lock, that seems to work for now.

 

I’m also running into another race condition with respect to precompile, I 
opened a new issue for that:

https://github.com/JuliaLang/julia/issues/17320

 

Cheers,

David

 

From: julia-users@googlegroups.com <mailto:julia-users@googlegroups.com>  
[mailto:julia-users@googlegroups.com <mailto:julia-users@googlegroups.com> ] On 
Behalf Of Isaiah Norton
Sent: Thursday, July 7, 2016 6:52 AM
To: julia-users@googlegroups.com <mailto:julia-users@googlegroups.com> 
Subject: Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

 

See

https://github.com/JuliaLang/julia/issues/5622 and

https://github.com/JuliaLang/julia/issues/7176

 

If you are on Windows, LockFileEx is reliable and you could ccall it yourself. 
Probably simpler to handle the problem at a higher level for now though (i.e. 
run one script that checks installation *before* launching your analysis jobs).

 

 

On Wed, Jul 6, 2016 at 5:09 PM, David Anthoff <anth...@berkeley.edu 
<mailto:anth...@berkeley.edu> > wrote:

Hi,

 

I have a script that is using some packages. The script is set up such that if 
the package it is trying to use is not installed, it will do a ``Pkg.add`` to 
add the package before ``using`` it.

 

This script is triggered by a third process, and it might be triggered multiple 
times simultaneously, i.e. multiple instances of julia might run the same 
script at the same time. Therefore the second julia instance might issue the 
same ``Pkg.add`` command as the first instance, while the first instance is not 
done with the ``Pkg.add`` yet.

 

I assume the package manager per se is not set up to handle that situation 
gracefully?

 

If not, is there some package that allows me to have something like a 
mutex/lock between my julia processes, so that I can guard the code section 
that fiddles with packages and make sure only one julia instance at a time runs 
that code?

 

Thanks,

David

 

--

David Anthoff

University of California, Berkeley

 

http://www.david-anthoff.com

 

 

 

Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

[Isaiah Norton]

Does that work on Windows? I briefly tried earlier and it didn't seem to
work. IIUC Windows doesn't support Unix-style named sockets. (it does have
named pipes, but the semantics are a bit different)

On Thu, Jul 7, 2016 at 1:07 PM, David Anthoff <anth...@berkeley.edu> wrote:

> Ah, I had actually come up with the same solution that Amit suggested in
> #7176 and am using a named socket as a lock, that seems to work for now.
>
>
>
> I’m also running into another race condition with respect to precompile, I
> opened a new issue for that:
>
> https://github.com/JuliaLang/julia/issues/17320
>
>
>
> Cheers,
>
> David
>
>
>
> *From:* julia-users@googlegroups.com [mailto:julia-users@googlegroups.com]
> *On Behalf Of *Isaiah Norton
> *Sent:* Thursday, July 7, 2016 6:52 AM
> *To:* julia-users@googlegroups.com
> *Subject:* Re: [julia-users] Pkg "threadsafe" or interprocess mutex
> package
>
>
>
> See
>
> https://github.com/JuliaLang/julia/issues/5622 and
>
> https://github.com/JuliaLang/julia/issues/7176
>
>
>
> If you are on Windows, LockFileEx is reliable and you could ccall it
> yourself. Probably simpler to handle the problem at a higher level for now
> though (i.e. run one script that checks installation *before* launching
> your analysis jobs).
>
>
>
>
>
> On Wed, Jul 6, 2016 at 5:09 PM, David Anthoff <anth...@berkeley.edu>
> wrote:
>
> Hi,
>
>
>
> I have a script that is using some packages. The script is set up such
> that if the package it is trying to use is not installed, it will do a
> ``Pkg.add`` to add the package before ``using`` it.
>
>
>
> This script is triggered by a third process, and it might be triggered
> multiple times simultaneously, i.e. multiple instances of julia might run
> the same script at the same time. Therefore the second julia instance might
> issue the same ``Pkg.add`` command as the first instance, while the first
> instance is not done with the ``Pkg.add`` yet.
>
>
>
> I assume the package manager per se is not set up to handle that situation
> gracefully?
>
>
>
> If not, is there some package that allows me to have something like a
> mutex/lock between my julia processes, so that I can guard the code section
> that fiddles with packages and make sure only one julia instance at a time
> runs that code?
>
>
>
> Thanks,
>
> David
>
>
>
> --
>
> David Anthoff
>
> University of California, Berkeley
>
>
>
> http://www.david-anthoff.com
>
>
>
>
>

RE: [julia-users] Pkg "threadsafe" or interprocess mutex package

[David Anthoff]

Ah, I had actually come up with the same solution that Amit suggested in #7176 
and am using a named socket as a lock, that seems to work for now.

 

I’m also running into another race condition with respect to precompile, I 
opened a new issue for that:

https://github.com/JuliaLang/julia/issues/17320

 

Cheers,

David

 

From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On 
Behalf Of Isaiah Norton
Sent: Thursday, July 7, 2016 6:52 AM
To: julia-users@googlegroups.com
Subject: Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

 

See

https://github.com/JuliaLang/julia/issues/5622 and

https://github.com/JuliaLang/julia/issues/7176

 

If you are on Windows, LockFileEx is reliable and you could ccall it yourself. 
Probably simpler to handle the problem at a higher level for now though (i.e. 
run one script that checks installation *before* launching your analysis jobs).

 

 

On Wed, Jul 6, 2016 at 5:09 PM, David Anthoff <anth...@berkeley.edu 
<mailto:anth...@berkeley.edu> > wrote:

Hi,

 

I have a script that is using some packages. The script is set up such that if 
the package it is trying to use is not installed, it will do a ``Pkg.add`` to 
add the package before ``using`` it.

 

This script is triggered by a third process, and it might be triggered multiple 
times simultaneously, i.e. multiple instances of julia might run the same 
script at the same time. Therefore the second julia instance might issue the 
same ``Pkg.add`` command as the first instance, while the first instance is not 
done with the ``Pkg.add`` yet.

 

I assume the package manager per se is not set up to handle that situation 
gracefully?

 

If not, is there some package that allows me to have something like a 
mutex/lock between my julia processes, so that I can guard the code section 
that fiddles with packages and make sure only one julia instance at a time runs 
that code?

 

Thanks,

David

 

--

David Anthoff

University of California, Berkeley

 

http://www.david-anthoff.com

 

 

Re: [julia-users] Pkg "threadsafe" or interprocess mutex package

[Isaiah Norton]

See
https://github.com/JuliaLang/julia/issues/5622 and
https://github.com/JuliaLang/julia/issues/7176

If you are on Windows, LockFileEx is reliable and you could ccall it
yourself. Probably simpler to handle the problem at a higher level for now
though (i.e. run one script that checks installation *before* launching
your analysis jobs).


On Wed, Jul 6, 2016 at 5:09 PM, David Anthoff  wrote:

> Hi,
>
>
>
> I have a script that is using some packages. The script is set up such
> that if the package it is trying to use is not installed, it will do a
> ``Pkg.add`` to add the package before ``using`` it.
>
>
>
> This script is triggered by a third process, and it might be triggered
> multiple times simultaneously, i.e. multiple instances of julia might run
> the same script at the same time. Therefore the second julia instance might
> issue the same ``Pkg.add`` command as the first instance, while the first
> instance is not done with the ``Pkg.add`` yet.
>
>
>
> I assume the package manager per se is not set up to handle that situation
> gracefully?
>
>
>
> If not, is there some package that allows me to have something like a
> mutex/lock between my julia processes, so that I can guard the code section
> that fiddles with packages and make sure only one julia instance at a time
> runs that code?
>
>
>
> Thanks,
>
> David
>
>
>
> --
>
> David Anthoff
>
> University of California, Berkeley
>
>
>
> http://www.david-anthoff.com
>
>
>

[julia-users] Pkg "threadsafe" or interprocess mutex package

[David Anthoff]

Hi,

 

I have a script that is using some packages. The script is set up such that
if the package it is trying to use is not installed, it will do a
``Pkg.add`` to add the package before ``using`` it.

 

This script is triggered by a third process, and it might be triggered
multiple times simultaneously, i.e. multiple instances of julia might run
the same script at the same time. Therefore the second julia instance might
issue the same ``Pkg.add`` command as the first instance, while the first
instance is not done with the ``Pkg.add`` yet.

 

I assume the package manager per se is not set up to handle that situation
gracefully?

 

If not, is there some package that allows me to have something like a
mutex/lock between my julia processes, so that I can guard the code section
that fiddles with packages and make sure only one julia instance at a time
runs that code?

 

Thanks,

David

 

--

David Anthoff

University of California, Berkeley

 

http://www.david-anthoff.com