Re: Spark on Mesos: Spark issuing hundreds of SUBSCRIBE requests / second and crashing Mesos

2018-07-23 Thread Susan X. Huynh
spark framework: https://gist. > github.com/nemosupremo/d098ef4def28ebf96c14d8f87aecd133 which also just > repeats 'Transport endpoint is not connected' over and over. > > Thanks for any insights > > > -- Susan X. Huynh Software engineer, Data Agility xhu...@mesosphere.com

Re: Interest in adding ability to request GPU's to the spark client?

2018-07-23 Thread Susan X. Huynh
e computer nodes such that each node will have only one >>> GPU. Of course, I realize that many Databricks customers are using Azure or >>> AWS, which allow you to do this facilely. Is this what people normally do >>> in industry? >>> >>> This is something I a

Re: Spark on Mesos - Weird behavior

2018-07-23 Thread Susan X. Huynh
k applications are not >>>> the only one that run on the cluster. I guess that Spark starts executors >>>> on the available offers even if it does not satisfy our needs. Is there any >>>> configuration that we can use in order to prevent Spark from starting when >>>> there are no resource offers for the total number of executors? >>>> >>>> Thank you >>>> - Thodoris >>>> >>>> - >>>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>>> >>>> >> -- Susan X. Huynh Software engineer, Data Agility xhu...@mesosphere.com

Re: Advice on multiple streaming job

2018-05-06 Thread Susan X. Huynh
new >> port. Also, Spark automatically checks port from 4040 to 4056, post that it >> fails. One of the workaround, is to provide port explicitly. >> >> Is there a way to tackle this situation? or Am I missing any thing? >> >> Thanking you in advance. >> >> Regards, >> Dhaval Modi >> dhavalmod...@gmail.com >> > -- Susan X. Huynh Software engineer, Data Agility xhu...@mesosphere.com

Re: [Mesos] How to Disable Blacklisting on Mesos?

2018-04-09 Thread Susan X. Huynh
Spark 2.3.0? Creative ideas are welcome :) > > Best, > Han > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- Susan X. Huynh Software engineer, Data Agility xhu...@mesosphere.com

Re: external shuffle service in mesos

2018-01-22 Thread Susan X. Huynh
from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- Susan X. Huynh Software engineer, Data Agility xhu...@mesosphere.com

Re: external shuffle service in mesos

2018-01-20 Thread Susan X. Huynh
lly number of instances config) > > So any production ops advices will be welcome > Igor > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- Susan X. Huynh Software engineer, Data Agility xhu...@mesosphere.com

Re: Spark job only starts tasks on a single node

2017-12-07 Thread Susan X. Huynh
> disclosure, copying, distribution or any action taken or omitted to be >>> taken in reliance on it, is prohibited and may be unlawful. >>> >> >> > > The information in this email is confidential and may be legally > privileged. It is intended solely for the addressee. Access to this email > by anyone else is unauthorized. If you are not the intended recipient, any > disclosure, copying, distribution or any action taken or omitted to be > taken in reliance on it, is prohibited and may be unlawful. > -- Susan X. Huynh Software engineer, Data Agility xhu...@mesosphere.com