It is probably the time taken by the system to figure out that the worker is down. Could you see in the logs to find what goes on when you kill the worker?
TD On Wed, Mar 4, 2015 at 6:20 AM, Nastooh Avessta (navesta) <nave...@cisco.com > wrote: > Indeed. And am wondering if this switchover time can be decreased. > > Cheers, > > > > [image: http://www.cisco.com/web/europe/images/email/signature/logo05.jpg] > > *Nastooh Avessta* > ENGINEER.SOFTWARE ENGINEERING > nave...@cisco.com > Phone: *+1 604 647 1527 <%2B1%20604%20647%201527>* > > *Cisco Systems Limited* > 595 Burrard Street, Suite 2123 Three Bentall Centre, PO Box 49121 > VANCOUVER > BRITISH COLUMBIA > V7X 1J1 > CA > Cisco.com <http://www.cisco.com/> > > > > [image: Think before you print.]Think before you print. > > This email may contain confidential and privileged material for the sole > use of the intended recipient. Any review, use, distribution or disclosure > by others is strictly prohibited. If you are not the intended recipient (or > authorized to receive for the recipient), please contact the sender by > reply email and delete all copies of this message. > > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/index.html > > Cisco Systems Canada Co, 181 Bay St., Suite 3400, Toronto, ON, Canada, M5J > 2T3. Phone: 416-306-7000; Fax: 416-306-7099. *Preferences > <http://www.cisco.com/offer/subscribe/?sid=000478326> - Unsubscribe > <http://www.cisco.com/offer/unsubscribe/?sid=000478327> – Privacy > <http://www.cisco.com/web/siteassets/legal/privacy.html>* > > > > *From:* Tathagata Das [mailto:t...@databricks.com] > *Sent:* Tuesday, March 03, 2015 11:11 PM > > *To:* Nastooh Avessta (navesta) > *Cc:* user@spark.apache.org > *Subject:* Re: Spark Streaming Switchover Time > > > > I am confused. Are you killing the 1st worker node to see whether the > system restarts the receiver on the second worker? > > > > TD > > > > On Tue, Mar 3, 2015 at 10:49 PM, Nastooh Avessta (navesta) < > nave...@cisco.com> wrote: > > This is the time that it takes for the driver to start receiving data once > again, from the 2nd worker, when the 1st worker, where streaming thread > was initially running, is shutdown. > > Cheers, > > > > [image: http://www.cisco.com/web/europe/images/email/signature/logo05.jpg] > > *Nastooh Avessta* > ENGINEER.SOFTWARE ENGINEERING > nave...@cisco.com > Phone: *+1 604 647 1527 <%2B1%20604%20647%201527>* > > *Cisco Systems Limited* > 595 Burrard Street, Suite 2123 Three Bentall Centre, PO Box 49121 > VANCOUVER > BRITISH COLUMBIA > V7X 1J1 > CA > Cisco.com <http://www.cisco.com/> > > > > [image: Think before you print.]Think before you print. > > This email may contain confidential and privileged material for the sole > use of the intended recipient. Any review, use, distribution or disclosure > by others is strictly prohibited. If you are not the intended recipient (or > authorized to receive for the recipient), please contact the sender by > reply email and delete all copies of this message. > > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/index.html > > Cisco Systems Canada Co, 181 Bay St., Suite 3400, Toronto, ON, Canada, M5J > 2T3. Phone: 416-306-7000; Fax: 416-306-7099. *Preferences > <http://www.cisco.com/offer/subscribe/?sid=000478326> - Unsubscribe > <http://www.cisco.com/offer/unsubscribe/?sid=000478327> – Privacy > <http://www.cisco.com/web/siteassets/legal/privacy.html>* > > > > *From:* Tathagata Das [mailto:t...@databricks.com] > *Sent:* Tuesday, March 03, 2015 10:24 PM > *To:* Nastooh Avessta (navesta) > *Cc:* user@spark.apache.org > *Subject:* Re: Spark Streaming Switchover Time > > > > Can you elaborate on what is this switchover time? > > > > TD > > > > On Tue, Mar 3, 2015 at 9:57 PM, Nastooh Avessta (navesta) < > nave...@cisco.com> wrote: > > Hi > > On a standalone, Spark 1.0.0, with 1 master and 2 workers, operating in > client mode, running a udp streaming application, I am noting around 2 > second elapse time on switchover, upon shutting down the streaming worker, > where streaming window length is 1 sec. I am wondering what parameters are > available to the developer to shorten this switchover time. > > Cheers, > > > > [image: http://www.cisco.com/web/europe/images/email/signature/logo05.jpg] > > *Nastooh Avessta* > ENGINEER.SOFTWARE ENGINEERING > nave...@cisco.com > Phone: *+1 604 647 1527 <%2B1%20604%20647%201527>* > > *Cisco Systems Limited* > 595 Burrard Street, Suite 2123 Three Bentall Centre, PO Box 49121 > VANCOUVER > BRITISH COLUMBIA > V7X 1J1 > CA > Cisco.com <http://www.cisco.com/> > > > > [image: Think before you print.]Think before you print. > > This email may contain confidential and privileged material for the sole > use of the intended recipient. Any review, use, distribution or disclosure > by others is strictly prohibited. If you are not the intended recipient (or > authorized to receive for the recipient), please contact the sender by > reply email and delete all copies of this message. > > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/index.html > > Cisco Systems Canada Co, 181 Bay St., Suite 3400, Toronto, ON, Canada, M5J > 2T3. Phone: 416-306-7000; Fax: 416-306-7099. *Preferences > <http://www.cisco.com/offer/subscribe/?sid=000478326> - Unsubscribe > <http://www.cisco.com/offer/unsubscribe/?sid=000478327> – Privacy > <http://www.cisco.com/web/siteassets/legal/privacy.html>* > > > > > > >
