[julia-users] Re: JuliaBox limitations on parallel computing
Thanks André! I'll be very interested to hear more about what you are building on top of DistributedArrays. The API is open and the CLI will be open source in a few weeks. The server will likely be partially open sourced in the future. And any generally applicable changes to the Julia codebase will be definitely submitted upstream. Best, Iliyan On Tuesday, October 27, 2015 at 6:07:03 PM UTC, André Lage wrote: > > Hi Iliyan, > > I just read your message now. > > Evoqus is a great idea! We are also working on a transparent support for > Julia computing in the cloud, based on DistributedArrays. We intend to > include pricing features as you did. > > Do you intend to share your code (even partially)? > > Congratulations for the work! > > > André. > > On Tuesday, October 27, 2015 at 2:23:33 AM UTC-3, Iliyan Zarov wrote: >> >> For anyone wishing to run large parallel programs I may be able to help - >> I'm working on a cloud computing platform for Julia that takes care of >> provisioning and managing all resources. More info at https://evoqus.com >> . >> >> It's currently in beta, get in touch if interested. >> >> >> Best, >> >> Iliyan >> >> On Monday, October 26, 2015 at 6:41:57 PM UTC, cdm wrote: >>> >>> >>> perhaps time for an asterisk on the JuliaBox banner: >>> >>> "The Julia community is doing amazing things. We want you in on it!*" >>> >>> * ... but not massively in parallel, consuming the resources of the >>> community. >>> >>> >>> >>> there are positives and negatives to this sort of problem ... >>> >>> the fact that the problem can even arise says great things >>> about the capabilities of the language and the service. >>> >>> well done ... >>> >>> >>> >>> On Sunday, October 25, 2015 at 2:11:25 AM UTC-7, Tanmay K. Mohapatra >>> wrote: Hi, from the past few days we have had some users run large parallel programs on JuliaBox sessions. While in some cases they succeed, we see a lot of failures due to resource constraints. Though we have plans to enable large programs in future, we do not allocate enough resources for that now. Also, since JuliaBox sessions run on shared infrastructure, this affects other sessions that are co-located on the same machine. We will be putting in place more checks and restrictions soon to prevent co-located sessions from being impacted. We would request users to refrain from running large parallel programs for now. If this is being done as part of some university class, please write to us. Probably a separately provisioned cluster will be more appropriate for that. - JuliaBox Team >>>
[julia-users] Re: JuliaBox limitations on parallel computing
Hi Iliyan, I just read your message now. Evoqus is a great idea! We are also working on a transparent support for Julia computing in the cloud, based on DistributedArrays. We intend to include pricing features as you did. Do you intend to share your code (even partially)? Congratulations for the work! André. On Tuesday, October 27, 2015 at 2:23:33 AM UTC-3, Iliyan Zarov wrote: > > For anyone wishing to run large parallel programs I may be able to help - > I'm working on a cloud computing platform for Julia that takes care of > provisioning and managing all resources. More info at https://evoqus.com > . > > It's currently in beta, get in touch if interested. > > > Best, > > Iliyan > > On Monday, October 26, 2015 at 6:41:57 PM UTC, cdm wrote: >> >> >> perhaps time for an asterisk on the JuliaBox banner: >> >> "The Julia community is doing amazing things. We want you in on it!*" >> >> * ... but not massively in parallel, consuming the resources of the >> community. >> >> >> >> there are positives and negatives to this sort of problem ... >> >> the fact that the problem can even arise says great things >> about the capabilities of the language and the service. >> >> well done ... >> >> >> >> On Sunday, October 25, 2015 at 2:11:25 AM UTC-7, Tanmay K. Mohapatra >> wrote: >>> >>> Hi, >>> >>> from the past few days we have had some users run large parallel >>> programs on JuliaBox sessions. While in some cases they succeed, we see a >>> lot of failures due to resource constraints. Though we have plans to enable >>> large programs in future, we do not allocate enough resources for that now. >>> >>> Also, since JuliaBox sessions run on shared infrastructure, this affects >>> other sessions that are co-located on the same machine. We will be putting >>> in place more checks and restrictions soon to prevent co-located sessions >>> from being impacted. >>> >>> We would request users to refrain from running large parallel programs >>> for now. If this is being done as part of some university class, please >>> write to us. Probably a separately provisioned cluster will be more >>> appropriate for that. >>> >>> - JuliaBox Team >>> >>
[julia-users] Re: JuliaBox limitations on parallel computing
For anyone wishing to run large parallel programs I may be able to help - I'm working on a cloud computing platform for Julia that takes care of provisioning and managing all resources. More info at https://evoqus.com . It's currently in beta, get in touch if interested. Best, Iliyan On Monday, October 26, 2015 at 6:41:57 PM UTC, cdm wrote: > > > perhaps time for an asterisk on the JuliaBox banner: > > "The Julia community is doing amazing things. We want you in on it!*" > > * ... but not massively in parallel, consuming the resources of the > community. > > > > there are positives and negatives to this sort of problem ... > > the fact that the problem can even arise says great things > about the capabilities of the language and the service. > > well done ... > > > > On Sunday, October 25, 2015 at 2:11:25 AM UTC-7, Tanmay K. Mohapatra wrote: >> >> Hi, >> >> from the past few days we have had some users run large parallel programs >> on JuliaBox sessions. While in some cases they succeed, we see a lot of >> failures due to resource constraints. Though we have plans to enable large >> programs in future, we do not allocate enough resources for that now. >> >> Also, since JuliaBox sessions run on shared infrastructure, this affects >> other sessions that are co-located on the same machine. We will be putting >> in place more checks and restrictions soon to prevent co-located sessions >> from being impacted. >> >> We would request users to refrain from running large parallel programs >> for now. If this is being done as part of some university class, please >> write to us. Probably a separately provisioned cluster will be more >> appropriate for that. >> >> - JuliaBox Team >> >
[julia-users] Re: JuliaBox limitations on parallel computing
Hi! I believe this is my fault. I've been running some items in parallel for class. I believe only one session ran away (1+ hours) while the others were < 10 minutes. Sorry for the trouble, I didn't realize there was a negative impact on others / the infrastructure. In the future I'll run processor intensive items on my local machine. Incidentally I am unable to login to Juliabox now (is this related?). I have not backed up my work to github and as such would respectfully request the ability to copy my files over if I have been banned from the service. Some of them have due dates :). Cheers, _Chris On Sunday, October 25, 2015 at 5:11:19 AM UTC-4, tanmaykm wrote: > > Hi, > > from the past few days we have had some users run large parallel programs > on JuliaBox sessions. While in some cases they succeed, we see a lot of > failures due to resource constraints. Though we have plans to enable large > programs in future, we do not allocate enough resources for that now. > > Also, since JuliaBox sessions run on shared infrastructure, this affects > other sessions that are co-located on the same machine. We will be putting > in place more checks and restrictions soon to prevent co-located sessions > from being impacted. > > We would request users to refrain from running large parallel programs for > now. If this is being done as part of some university class, please write > to us. Probably a separately provisioned cluster will be more appropriate > for that. > > - JuliaBox Team >
[julia-users] Re: JuliaBox limitations on parallel computing
Hi Iliyan, this sounds great! On Tuesday, October 27, 2015 at 10:53:33 AM UTC+5:30, Iliyan Zarov wrote: > > For anyone wishing to run large parallel programs I may be able to help - > I'm working on a cloud computing platform for Julia that takes care of > provisioning and managing all resources. More info at https://evoqus.com > . > > It's currently in beta, get in touch if interested. > > > Best, > > Iliyan > > On Monday, October 26, 2015 at 6:41:57 PM UTC, cdm wrote: >> >> >> perhaps time for an asterisk on the JuliaBox banner: >> >> "The Julia community is doing amazing things. We want you in on it!*" >> >> * ... but not massively in parallel, consuming the resources of the >> community. >> >> >> >> there are positives and negatives to this sort of problem ... >> >> the fact that the problem can even arise says great things >> about the capabilities of the language and the service. >> >> well done ... >> >> >> >> On Sunday, October 25, 2015 at 2:11:25 AM UTC-7, Tanmay K. Mohapatra >> wrote: >>> >>> Hi, >>> >>> from the past few days we have had some users run large parallel >>> programs on JuliaBox sessions. While in some cases they succeed, we see a >>> lot of failures due to resource constraints. Though we have plans to enable >>> large programs in future, we do not allocate enough resources for that now. >>> >>> Also, since JuliaBox sessions run on shared infrastructure, this affects >>> other sessions that are co-located on the same machine. We will be putting >>> in place more checks and restrictions soon to prevent co-located sessions >>> from being impacted. >>> >>> We would request users to refrain from running large parallel programs >>> for now. If this is being done as part of some university class, please >>> write to us. Probably a separately provisioned cluster will be more >>> appropriate for that. >>> >>> - JuliaBox Team >>> >>
[julia-users] Re: JuliaBox limitations on parallel computing
Hi André, I think it should be fine as long as the code is not compute intensive and across <10 workers. Best, Tanmay On Tuesday, October 27, 2015 at 10:53:33 AM UTC+5:30, André Lage wrote: > > Hi, > > I'll be giving a hands-on tutorial on Julia next week. I expect about 30 > people in 6th November, from 12h30 to 16h (UTC-3), to use IJulia notebook > sessions to learn Julia and run some simple parallel code with few Workers. > > Would it be a problem? > > Thanks, > > > André Lage. > > On Sunday, October 25, 2015 at 6:11:19 AM UTC-3, tanmaykm wrote: >> >> Hi, >> >> from the past few days we have had some users run large parallel programs >> on JuliaBox sessions. While in some cases they succeed, we see a lot of >> failures due to resource constraints. Though we have plans to enable large >> programs in future, we do not allocate enough resources for that now. >> >> Also, since JuliaBox sessions run on shared infrastructure, this affects >> other sessions that are co-located on the same machine. We will be putting >> in place more checks and restrictions soon to prevent co-located sessions >> from being impacted. >> >> We would request users to refrain from running large parallel programs >> for now. If this is being done as part of some university class, please >> write to us. Probably a separately provisioned cluster will be more >> appropriate for that. >> >> - JuliaBox Team >> >
[julia-users] Re: JuliaBox limitations on parallel computing
Hi Chris, a few minutes of running parallel Julia will not really have a negative impact. The parallel programs I was referring to were 100 odd workers each, and there were more than one users running similar tasks at some point which was too much for a single machine to handle. Also, we have not banned you or anyone from JuliaBox because of this. :) Do let us know if you are still unable to login. Cheers, Tanmay On Tuesday, October 27, 2015 at 10:53:33 AM UTC+5:30, Christopher Fusting wrote: > > Hi! > > I believe this is my fault. I've been running some items in parallel for > class. I believe only one session ran away (1+ hours) while the others were > < 10 minutes. Sorry for the trouble, I didn't realize there was a negative > impact on others / the infrastructure. In the future I'll run processor > intensive items on my local machine. > > Incidentally I am unable to login to Juliabox now (is this related?). I > have not backed up my work to github and as such would respectfully request > the ability to copy my files over if I have been banned from the service. > Some of them have due dates :). > > Cheers, > > _Chris > > On Sunday, October 25, 2015 at 5:11:19 AM UTC-4, tanmaykm wrote: >> >> Hi, >> >> from the past few days we have had some users run large parallel programs >> on JuliaBox sessions. While in some cases they succeed, we see a lot of >> failures due to resource constraints. Though we have plans to enable large >> programs in future, we do not allocate enough resources for that now. >> >> Also, since JuliaBox sessions run on shared infrastructure, this affects >> other sessions that are co-located on the same machine. We will be putting >> in place more checks and restrictions soon to prevent co-located sessions >> from being impacted. >> >> We would request users to refrain from running large parallel programs >> for now. If this is being done as part of some university class, please >> write to us. Probably a separately provisioned cluster will be more >> appropriate for that. >> >> - JuliaBox Team >> >
[julia-users] Re: JuliaBox limitations on parallel computing
Hi, I'll be giving a hands-on tutorial on Julia next week. I expect about 30 people in 6th November, from 12h30 to 16h (UTC-3), to use IJulia notebook sessions to learn Julia and run some simple parallel code with few Workers. Would it be a problem? Thanks, André Lage. On Sunday, October 25, 2015 at 6:11:19 AM UTC-3, tanmaykm wrote: > > Hi, > > from the past few days we have had some users run large parallel programs > on JuliaBox sessions. While in some cases they succeed, we see a lot of > failures due to resource constraints. Though we have plans to enable large > programs in future, we do not allocate enough resources for that now. > > Also, since JuliaBox sessions run on shared infrastructure, this affects > other sessions that are co-located on the same machine. We will be putting > in place more checks and restrictions soon to prevent co-located sessions > from being impacted. > > We would request users to refrain from running large parallel programs for > now. If this is being done as part of some university class, please write > to us. Probably a separately provisioned cluster will be more appropriate > for that. > > - JuliaBox Team >
[julia-users] Re: JuliaBox limitations on parallel computing
perhaps time for an asterisk on the JuliaBox banner: "The Julia community is doing amazing things. We want you in on it!*" * ... but not massively in parallel, consuming the resources of the community. there are positives and negatives to this sort of problem ... the fact that the problem can even arise says great things about the capabilities of the language and the service. well done ... On Sunday, October 25, 2015 at 2:11:25 AM UTC-7, Tanmay K. Mohapatra wrote: > > Hi, > > from the past few days we have had some users run large parallel programs > on JuliaBox sessions. While in some cases they succeed, we see a lot of > failures due to resource constraints. Though we have plans to enable large > programs in future, we do not allocate enough resources for that now. > > Also, since JuliaBox sessions run on shared infrastructure, this affects > other sessions that are co-located on the same machine. We will be putting > in place more checks and restrictions soon to prevent co-located sessions > from being impacted. > > We would request users to refrain from running large parallel programs for > now. If this is being done as part of some university class, please write > to us. Probably a separately provisioned cluster will be more appropriate > for that. > > - JuliaBox Team >
