[jira] [Issue Comment Deleted] (SPARK-19300) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19300: Comment: was deleted (was: [~zsxwing] I also met this issue, and you can look

[jira] [Commented] (SPARK-19300) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902988#comment-15902988 ] hustfxj commented on SPARK-19300: - [~zsxwing] I also met this issue, and you can look

[jira] [Closed] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj closed SPARK-19829. --- Resolution: Invalid > The log about driver should support rolling like executor >

[jira] [Commented] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902982#comment-15902982 ] hustfxj commented on SPARK-19829: - ok. > The log about driver should support rolling like executor >

[jira] [Commented] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902980#comment-15902980 ] hustfxj commented on SPARK-19883: - [~srowen] Saw this issue again today. So spark may not solve this

[jira] [Comment Edited] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902972#comment-15902972 ] hustfxj edited comment on SPARK-19883 at 3/9/17 12:25 PM: -- [~srowen] A stage is

[jira] [Reopened] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj reopened SPARK-19883: - I this it is a question, like that https://issues.apache.org/jira/browse/SPARK-19300 > Executor is waiting

[jira] [Commented] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902972#comment-15902972 ] hustfxj commented on SPARK-19883: - [~srowen] A stage is still blocked because a task of this stage is

[jira] [Issue Comment Deleted] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19883: Comment: was deleted (was: And the netty is netty from 4.0.43.Final) > Executor is waiting for lock >

[jira] [Commented] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902962#comment-15902962 ] hustfxj commented on SPARK-19883: - And the netty is netty from 4.0.43.Final > Executor is waiting for

[jira] [Updated] (SPARK-19883) Executor is waiting for lock

[ https://issues.apache.org/jira/browse/SPARK-19883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19883: Attachment: jstack the full thread dump > Executor is waiting for lock > > >

[jira] [Created] (SPARK-19883) Executor is waiting for lock

hustfxj created SPARK-19883: --- Summary: Executor is waiting for lock Key: SPARK-19883 URL: https://issues.apache.org/jira/browse/SPARK-19883 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Comment Edited] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898704#comment-15898704 ] hustfxj edited comment on SPARK-19831 at 3/7/17 4:15 AM: - [~zsxwing]. I only find

[jira] [Commented] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898713#comment-15898713 ] hustfxj commented on SPARK-19829: - [~sowen] Yes, this is handled by systems like YARN. But my spark

[jira] [Commented] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15898704#comment-15898704 ] hustfxj commented on SPARK-19831: - [~zsxwing]. I only find the code which handles *ApplicationFinished*

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Summary: Sending the heartbeat master from worker maybe blocked by other rpc messages (was: Sending the

[jira] [Created] (SPARK-19831) Sending the heartbeat to master maybe blocked by other rpc messages

hustfxj created SPARK-19831: --- Summary: Sending the heartbeat to master maybe blocked by other rpc messages Key: SPARK-19831 URL: https://issues.apache.org/jira/browse/SPARK-19831 Project: Spark

[jira] [Updated] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19829: Description: We should rollback the log of the driver , or the log maybe large！！！

[jira] [Created] (SPARK-19829) The log about driver should support rolling like executor

hustfxj created SPARK-19829: --- Summary: The log about driver should support rolling like executor Key: SPARK-19829 URL: https://issues.apache.org/jira/browse/SPARK-19829 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828402#comment-15828402 ] hustfxj commented on SPARK-19264: - @Sean Owen Why not solve it like AM of Yarn. I remember the

[jira] [Commented] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828366#comment-15828366 ] hustfxj commented on SPARK-19264: - Maybe you are right. We can't hard-kill the driver. But I don't

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827201#comment-15827201 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:39 AM: -- So I must change the

[jira] [Commented] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827201#comment-15827201 ] hustfxj commented on SPARK-19264: - So I must change the last code like that {code} try {

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:18 AM: -- [~srowen] sorry, I

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:16 AM: -- [~srowen] sorry, I

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:16 AM: -- [~srowen] sorry, I

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:12 AM: -- [~srowen] sorry, I

[jira] [Commented] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827181#comment-15827181 ] hustfxj commented on SPARK-19264: - [~srowen] sorry, I didn't say it clearly. I means an spark application

[jira] [Updated] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19264: Description: I think work can't start driver by "ProcessBuilderLike", thus we can't know the

[jira] [Updated] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19264: Description: I think work can't start driver by "ProcessBuilderLike", thus we can't know the

[jira] [Updated] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19264: Description: I think work can't start driver by "ProcessBuilderLike", thus we can't know the

[jira] [Updated] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19264: Description: I think work can't start driver by "ProcessBuilderLike", thus we can't know the

[jira] [Updated] (SPARK-19264) Work should start driver, the same to AM of yarn

[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19264: Summary: Work should start driver, the same to AM of yarn (was: Work maybe should start driver, the

[jira] [Created] (SPARK-19264) Work maybe should start driver, the same to AM of yarn

hustfxj created SPARK-19264: --- Summary: Work maybe should start driver, the same to AM of yarn Key: SPARK-19264 URL: https://issues.apache.org/jira/browse/SPARK-19264 Project: Spark Issue Type:

[jira] [Created] (SPARK-19052) the rest api don't support multiple standby masters on standalone cluster

hustfxj created SPARK-19052: --- Summary: the rest api don't support multiple standby masters on standalone cluster Key: SPARK-19052 URL: https://issues.apache.org/jira/browse/SPARK-19052 Project: Spark

[jira] [Commented] (SPARK-18959) invalid resource statistics for standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791056#comment-15791056 ] hustfxj commented on SPARK-18959: - Yes, I have a fix. You can see the link

[jira] [Commented] (SPARK-19042) Remove query string from jar url for executor

[ https://issues.apache.org/jira/browse/SPARK-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15789488#comment-15789488 ] hustfxj commented on SPARK-19042: - https://github.com/apache/spark/pull/16443 > Remove query string

[jira] [Created] (SPARK-19042) Remove query string from jar url for executor

hustfxj created SPARK-19042: --- Summary: Remove query string from jar url for executor Key: SPARK-19042 URL: https://issues.apache.org/jira/browse/SPARK-19042 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-19011) ApplicationDescription should add the Submission ID for the standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19011: Description: A large standalone cluster may have logs of applications and drivers, So I may not know which

[jira] [Commented] (SPARK-19011) ApplicationDescription should add the Submission ID for the standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-19011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15780310#comment-15780310 ] hustfxj commented on SPARK-19011: - A large standalone cluster may have logs of applications and drivers,

[jira] [Created] (SPARK-19011) ApplicationDescription should add the Submission ID for the standalone cluster

hustfxj created SPARK-19011: --- Summary: ApplicationDescription should add the Submission ID for the standalone cluster Key: SPARK-19011 URL: https://issues.apache.org/jira/browse/SPARK-19011 Project: Spark

[jira] [Commented] (SPARK-18959) invalid resource statistics for standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15768696#comment-15768696 ] hustfxj commented on SPARK-18959: - 2.2.0-SNAPSHOT > invalid resource statistics for standalone cluster >

[jira] [Updated] (SPARK-18959) invalid resource statistics for standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-18959: Attachment: 屏幕快照 2016-12-21 11.49.12.png The attachment is the master page > invalid resource statistics

[jira] [Commented] (SPARK-18959) invalid resource statistics for standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766020#comment-15766020 ] hustfxj commented on SPARK-18959: - the attachment is the master page > invalid resource statistics for

[jira] [Created] (SPARK-18959) invalid resource statistics for standalone cluster

hustfxj created SPARK-18959: --- Summary: invalid resource statistics for standalone cluster Key: SPARK-18959 URL: https://issues.apache.org/jira/browse/SPARK-18959 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-18706) Can spark support exactly once based kafka ? Due to these following question.

[ https://issues.apache.org/jira/browse/SPARK-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj closed SPARK-18706. --- Resolution: Duplicate > Can spark support exactly once based kafka ? Due to these following question. >

[jira] [Commented] (SPARK-18706) Can spark support exactly once based kafka ? Due to these following question.

[ https://issues.apache.org/jira/browse/SPARK-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15720087#comment-15720087 ] hustfxj commented on SPARK-18706: - 1. If a task complete the operation, it will notify driver. The driver

[jira] [Created] (SPARK-18706) Can spark support exactly once based kafka ? Due to these following question.

hustfxj created SPARK-18706: --- Summary: Can spark support exactly once based kafka ? Due to these following question. Key: SPARK-18706 URL: https://issues.apache.org/jira/browse/SPARK-18706 Project: Spark

[jira] [Created] (SPARK-18707) Can spark support exactly once based kafka ? Due to these following question?

hustfxj created SPARK-18707: --- Summary: Can spark support exactly once based kafka ? Due to these following question? Key: SPARK-18707 URL: https://issues.apache.org/jira/browse/SPARK-18707 Project: Spark

[jira] [Commented] (SPARK-18573) when restart yarn cluster, the spark streaming job will lost ?

[ https://issues.apache.org/jira/browse/SPARK-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691986#comment-15691986 ] hustfxj commented on SPARK-18573: - when I restart yarn cluster, the spark streaming jobs will lost . So

[jira] [Created] (SPARK-18573) when restart yarn cluster, the spark streaming job will lost ?

hustfxj created SPARK-18573: --- Summary: when restart yarn cluster, the spark streaming job will lost ? Key: SPARK-18573 URL: https://issues.apache.org/jira/browse/SPARK-18573 Project: Spark Issue