[jira] [Updated] (AURORA-1621) Support redirect to leading scheduler host
[ https://issues.apache.org/jira/browse/AURORA-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1621: -- Labels: features uber (was: features) > Support redirect to leading scheduler host > -- > > Key: AURORA-1621 > URL: https://issues.apache.org/jira/browse/AURORA-1621 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Ashwin Murthy >Assignee: Ashwin Murthy >Priority: Minor > Labels: features, uber > > Support a health endpoint that returns 200 if the host corresponds to the one > of the leading scheduler. If not, return 500. This allows support for > redirect to the leading scheduler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1621) Support redirect to leading scheduler host
[ https://issues.apache.org/jira/browse/AURORA-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1621: -- Component/s: (was: Service Discovery) Scheduler > Support redirect to leading scheduler host > -- > > Key: AURORA-1621 > URL: https://issues.apache.org/jira/browse/AURORA-1621 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Ashwin Murthy >Assignee: Ashwin Murthy >Priority: Minor > Labels: features > > Support a health endpoint that returns 200 if the host corresponds to the one > of the leading scheduler. If not, return 500. This allows support for > redirect to the leading scheduler -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AURORA-1597) User guide missing throttled state
[ https://issues.apache.org/jira/browse/AURORA-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15117679#comment-15117679 ] Chris Lambert commented on AURORA-1597: --- Sorry, I mean whatever the original, pre-png, version is so that it's easy to edit it. > User guide missing throttled state > -- > > Key: AURORA-1597 > URL: https://issues.apache.org/jira/browse/AURORA-1597 > Project: Aurora > Issue Type: Task > Components: Documentation >Reporter: Chris Lambert >Priority: Minor > Attachments: lifeofatask.png > > > It looks like the user guide makes no mention of the throttled state. Seems > like this should be included there, no? > http://aurora.apache.org/documentation/latest/user-guide/ > Where's the original version of the lifeofatask.png diagram? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AURORA-1597) User guide missing throttled state
Chris Lambert created AURORA-1597: - Summary: User guide missing throttled state Key: AURORA-1597 URL: https://issues.apache.org/jira/browse/AURORA-1597 Project: Aurora Issue Type: Task Components: Documentation Reporter: Chris Lambert Priority: Minor It looks like the user guide makes no mention of the throttled state. Seems like this should be included there, no? http://aurora.apache.org/documentation/latest/user-guide/ Where's the original version of the lifeofatask.png diagram? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (AURORA-118) Add percentiles to @Timed, or write a new decorator to add percentiles
[ https://issues.apache.org/jira/browse/AURORA-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert reassigned AURORA-118: Assignee: (was: Chris Lambert) [~zmanji] thanks for the heads up. However, that assignment was actually not intended. Sorry for the noise. > Add percentiles to @Timed, or write a new decorator to add percentiles > -- > > Key: AURORA-118 > URL: https://issues.apache.org/jira/browse/AURORA-118 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Priority: Minor > Labels: newbie > > The @Timed annotation is really nice for 'sprinkling on' instrumentation, but > doesn't expose percentiles. We've seen several areas where a long tail of > slow operations caused major performance issues, so spotting these with > percentiles would be very helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (AURORA-118) Add percentiles to @Timed, or write a new decorator to add percentiles
[ https://issues.apache.org/jira/browse/AURORA-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert reassigned AURORA-118: Assignee: Chris Lambert > Add percentiles to @Timed, or write a new decorator to add percentiles > -- > > Key: AURORA-118 > URL: https://issues.apache.org/jira/browse/AURORA-118 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Chris Lambert >Priority: Minor > Labels: newbie > > The @Timed annotation is really nice for 'sprinkling on' instrumentation, but > doesn't expose percentiles. We've seen several areas where a long tail of > slow operations caused major performance issues, so spotting these with > percentiles would be very helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1332) Updater does not skip already updated instances with /INSTANCES option.
[ https://issues.apache.org/jira/browse/AURORA-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1332: -- Story Points: 2 > Updater does not skip already updated instances with /INSTANCES option. > --- > > Key: AURORA-1332 > URL: https://issues.apache.org/jira/browse/AURORA-1332 > Project: Aurora > Issue Type: Bug > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Bill Farner > > The new updater is too restrictive when processing /INSTANCES option. E.g. if > instance 0 is already updated but the client sends /0-3 range, the entire > request is rejected with: "updateOnlyTheseInstances contains instances > irrelevant to the update: [0]". > https://github.com/apache/aurora/blob/2dc1d59f1e772e220b3bfb26480c3b90c688800f/src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java#L1175-L1181 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (AURORA-1332) Updater does not skip already updated instances with /INSTANCES option.
[ https://issues.apache.org/jira/browse/AURORA-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert reassigned AURORA-1332: - Assignee: Bill Farner > Updater does not skip already updated instances with /INSTANCES option. > --- > > Key: AURORA-1332 > URL: https://issues.apache.org/jira/browse/AURORA-1332 > Project: Aurora > Issue Type: Bug > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Bill Farner > > The new updater is too restrictive when processing /INSTANCES option. E.g. if > instance 0 is already updated but the client sends /0-3 range, the entire > request is rejected with: "updateOnlyTheseInstances contains instances > irrelevant to the update: [0]". > https://github.com/apache/aurora/blob/2dc1d59f1e772e220b3bfb26480c3b90c688800f/src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java#L1175-L1181 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1096) Scheduler updater should limit the number of job/instance events
[
https://issues.apache.org/jira/browse/AURORA-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1096:
--
Story Points: 5
> Scheduler updater should limit the number of job/instance events
>
>
> Key: AURORA-1096
> URL: https://issues.apache.org/jira/browse/AURORA-1096
> Project: Aurora
> Issue Type: Story
> Components: Scheduler
>Reporter: Maxim Khutornenko
>Assignee: Joe Smith
>
> Large/flapping scheduler job updates may generate too many events in the
> update store. The update settings are fully controlled by the user and there
> is a potential for a misconfigured job update to completely overwhelm our
> in-memory DB storage with job update instance events.
> For example, a large flapping update with {{max_per_shard_failures}} and
> {{max_total_failures}} set to max INT when left unattended can quickly
> consume all available RAM and kill the scheduler. A manual cleanup of the
> scheduler log would be needed to bring the scheduler up.
> This can be especially relevant with the introduction of update heartbeats
> (AURORA-690) that can further exacerbate the problem (e.g. when
> {{blockIfNoPulseAfterMs}} set too low wrt the external service pulse rate).
> We need to cap the max per-job lifetime count of {{JobUpdateEvent}} and
> {{JobInstanceUpdateEvent}} instances. A nice bonus would be providing a hint
> in the UI when the event sequence is cut off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1352) Audit log doesn't capture Shiro subject principal
[ https://issues.apache.org/jira/browse/AURORA-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1352: -- Sprint: Twitter Aurora Q2'15 Sprint 5, Twitter Aurora Q2'15 Sprint 7 (was: Twitter Aurora Q2'15 Sprint 5) > Audit log doesn't capture Shiro subject principal > - > > Key: AURORA-1352 > URL: https://issues.apache.org/jira/browse/AURORA-1352 > Project: Aurora > Issue Type: Story > Components: Scheduler, Security >Reporter: Kevin Sweeney >Assignee: Kevin Sweeney > > Scheduler RPCs that were authenticated by Shiro show UNSECURE in their audit > messages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1332) Updater does not skip already updated instances with /INSTANCES option.
[ https://issues.apache.org/jira/browse/AURORA-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1332: -- Sprint: Twitter Aurora Q2'15 Sprint 7 > Updater does not skip already updated instances with /INSTANCES option. > --- > > Key: AURORA-1332 > URL: https://issues.apache.org/jira/browse/AURORA-1332 > Project: Aurora > Issue Type: Bug > Components: Scheduler >Reporter: Maxim Khutornenko > > The new updater is too restrictive when processing /INSTANCES option. E.g. if > instance 0 is already updated but the client sends /0-3 range, the entire > request is rejected with: "updateOnlyTheseInstances contains instances > irrelevant to the update: [0]". > https://github.com/apache/aurora/blob/2dc1d59f1e772e220b3bfb26480c3b90c688800f/src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java#L1175-L1181 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1096) Scheduler updater should limit the number of job/instance events
[
https://issues.apache.org/jira/browse/AURORA-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1096:
--
Sprint: Twitter Aurora Q2'15 Sprint 7
> Scheduler updater should limit the number of job/instance events
>
>
> Key: AURORA-1096
> URL: https://issues.apache.org/jira/browse/AURORA-1096
> Project: Aurora
> Issue Type: Story
> Components: Scheduler
>Reporter: Maxim Khutornenko
>Assignee: Joe Smith
>
> Large/flapping scheduler job updates may generate too many events in the
> update store. The update settings are fully controlled by the user and there
> is a potential for a misconfigured job update to completely overwhelm our
> in-memory DB storage with job update instance events.
> For example, a large flapping update with {{max_per_shard_failures}} and
> {{max_total_failures}} set to max INT when left unattended can quickly
> consume all available RAM and kill the scheduler. A manual cleanup of the
> scheduler log would be needed to bring the scheduler up.
> This can be especially relevant with the introduction of update heartbeats
> (AURORA-690) that can further exacerbate the problem (e.g. when
> {{blockIfNoPulseAfterMs}} set too low wrt the external service pulse rate).
> We need to cap the max per-job lifetime count of {{JobUpdateEvent}} and
> {{JobInstanceUpdateEvent}} instances. A nice bonus would be providing a hint
> in the UI when the event sequence is cut off.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1379) Referential integrity violation when replaying storage
[
https://issues.apache.org/jira/browse/AURORA-1379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1379:
--
Sprint: Twitter Aurora Q2'15 Sprint 6, Twitter Aurora Q2'15 Sprint 7 (was:
Twitter Aurora Q2'15 Sprint 6)
> Referential integrity violation when replaying storage
> --
>
> Key: AURORA-1379
> URL: https://issues.apache.org/jira/browse/AURORA-1379
> Project: Aurora
> Issue Type: Bug
> Components: Scheduler
>Reporter: Kevin Sweeney
>Assignee: Bill Farner
>
> At startup in the Vagrant environment I observed:
> {noformat}
> E0630 20:45:54.427 THREAD1
> org.apache.aurora.scheduler.SchedulerLifecycle$9.execute: Caught unchecked
> exception: org.apache.aurora.scheduler.storage.Storage$StorageExce
> ption:
> ### Error updating database. Cause: org.h2.jdbc.JdbcSQLException:
> Referential integrity constraint violation: "CONSTRAINT_4BE: PUBLIC.TASKS
> FOREIGN KEY(SLAVE_ROW_ID) R
> EFERENCES PUBLIC.HOST_ATTRIBUTES(ID) (1)"; SQL statement:
> DELETE FROM host_attributes
> WHERE host = ? [23503-187]
> ### The error may involve defaultParameterMap
> ### The error occurred while setting parameters
> ### SQL: DELETE FROM host_attributes WHERE host = ?
> ### Cause: org.h2.jdbc.JdbcSQLException: Referential integrity constraint
> violation: "CONSTRAINT_4BE: PUBLIC.TASKS FOREIGN KEY(SLAVE_ROW_ID) REFERENCES
> PUBLIC.HOST_ATTR
> IBUTES(ID) (1)"; SQL statement:
> DELETE FROM host_attributes
> WHERE host = ? [23503-187]
> org.apache.aurora.scheduler.storage.Storage$StorageException:
> ### Error updating database. Cause: org.h2.jdbc.JdbcSQLException:
> Referential integrity constraint violation: "CONSTRAINT_4BE: PUBLIC.TASKS
> FOREIGN KEY(SLAVE_ROW_ID) R
> EFERENCES PUBLIC.HOST_ATTRIBUTES(ID) (1)"; SQL statement:
> DELETE FROM host_attributes
> WHERE host = ? [23503-187]
> ### The error may involve defaultParameterMap
> ### The error occurred while setting parameters
> ### SQL: DELETE FROM host_attributes WHERE host = ?
> ### Cause: org.h2.jdbc.JdbcSQLException: Referential integrity constraint
> violation: "CONSTRAINT_4BE: PUBLIC.TASKS FOREIGN KEY(SLAVE_ROW_ID) REFERENCES
> PUBLIC.HOST_ATTR
> IBUTES(ID) (1)"; SQL statement:
> DELETE FROM host_attributes
> WHERE host = ? [23503-187]
> at
> org.apache.aurora.scheduler.storage.db.DbStorage.write(DbStorage.java:144)
> at
> org.mybatis.guice.transactional.TransactionalMethodInterceptor.invoke(TransactionalMethodInterceptor.java:101)
> at
> com.twitter.common.inject.TimedInterceptor.invoke(TimedInterceptor.java:87)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage.write(LogStorage.java:632)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage$3.execute(LogStorage.java:316)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage$3.execute(LogStorage.java:313)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage.replay(LogStorage.java:525)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage.access$1200(LogStorage.java:116)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage$21$1.execute(LogStorage.java:503)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage$21$1.execute(LogStorage.java:500)
> at
> org.apache.aurora.scheduler.storage.log.StreamManagerImpl.readFromBeginning(StreamManagerImpl.java:127)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage$21.execute(LogStorage.java:500)
> at
> org.apache.aurora.scheduler.storage.Storage$MutateWork$NoResult.apply(Storage.java:131)
> at
> org.apache.aurora.scheduler.storage.db.DbStorage.bulkLoad(DbStorage.java:165)
> at
> com.twitter.common.inject.TimedInterceptor.invoke(TimedInterceptor.java:87)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage.recover(LogStorage.java:496)
> at
> com.twitter.common.inject.TimedInterceptor.invoke(TimedInterceptor.java:87)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage$20.execute(LogStorage.java:477)
> at
> org.apache.aurora.scheduler.storage.Storage$MutateWork$NoResult.apply(Storage.java:131)
> at
> org.apache.aurora.scheduler.storage.Storage$MutateWork$NoResult$Quiet.apply(Storage.java:148)
> at
> org.apache.aurora.scheduler.storage.db.DbStorage.write(DbStorage.java:142)
> at
> org.mybatis.guice.transactional.TransactionalMethodInterceptor.invoke(TransactionalMethodInterceptor.java:101)
> at
> com.twitter.common.inject.TimedInterceptor.invoke(TimedInterceptor.java:87)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage.write(LogStorage.java:632)
> at
> org.apache.aurora.scheduler.storage.log.LogStorage.start(LogStorage.java:471)
> at
> o
[jira] [Updated] (AURORA-1298) Make DbTaskStore performance suitable for use in production
[ https://issues.apache.org/jira/browse/AURORA-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1298: -- Sprint: Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 6 (was: Twitter Aurora Q2'15 Sprint 2) > Make DbTaskStore performance suitable for use in production > --- > > Key: AURORA-1298 > URL: https://issues.apache.org/jira/browse/AURORA-1298 > Project: Aurora > Issue Type: Story > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > > The DbTaskStore implementation has so far only targeted behavioral compliance > with InMemTaskStore. Use benchmarks to determine areas that require perf > improvements and address them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1334) Remove GCExecutorLauncher code
[ https://issues.apache.org/jira/browse/AURORA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1334: -- Sprint: Twitter Aurora Q2'15 Sprint 6 > Remove GCExecutorLauncher code > -- > > Key: AURORA-1334 > URL: https://issues.apache.org/jira/browse/AURORA-1334 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1333) Remove GC executor code
[ https://issues.apache.org/jira/browse/AURORA-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1333: -- Sprint: Twitter Aurora Q2'15 Sprint 6 > Remove GC executor code > --- > > Key: AURORA-1333 > URL: https://issues.apache.org/jira/browse/AURORA-1333 > Project: Aurora > Issue Type: Task > Components: Executor >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-814) Migrate client off of deprecated SessionKey APIs
[ https://issues.apache.org/jira/browse/AURORA-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-814: - Sprint: (was: Twitter Aurora Q2'15 Sprint 6) > Migrate client off of deprecated SessionKey APIs > > > Key: AURORA-814 > URL: https://issues.apache.org/jira/browse/AURORA-814 > Project: Aurora > Issue Type: Task > Components: Security >Reporter: Chris Lambert > > Remove SessionKey injection from the scheduler client, instead relying on > transport-level authentication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-814) Migrate client off of deprecated SessionKey APIs
[ https://issues.apache.org/jira/browse/AURORA-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-814: - Sprint: Twitter Aurora Q2'15 Sprint 6 > Migrate client off of deprecated SessionKey APIs > > > Key: AURORA-814 > URL: https://issues.apache.org/jira/browse/AURORA-814 > Project: Aurora > Issue Type: Task > Components: Security >Reporter: Chris Lambert > > Remove SessionKey injection from the scheduler client, instead relying on > transport-level authentication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-698) aurora executor _shutdown deadline calls should be daemonized
[
https://issues.apache.org/jira/browse/AURORA-698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-698:
-
Sprint: Twitter Aurora Q2'15 Sprint 3, Twitter Aurora Q2'15 Sprint 6 (was:
Twitter Aurora Q2'15 Sprint 3)
> aurora executor _shutdown deadline calls should be daemonized
> -
>
> Key: AURORA-698
> URL: https://issues.apache.org/jira/browse/AURORA-698
> Project: Aurora
> Issue Type: Bug
> Components: Executor
>Reporter: brian wickman
>Assignee: brian wickman
>
> In the aurora executor shutdown method, we have deadline() calls:
> {noformat}
> def _shutdown(self, status_result):
> runner_status = self._runner.status
> try:
> deadline(self._runner.stop, timeout=self.STOP_TIMEOUT)
> except Timeout:
> log.error('Failed to stop runner within deadline.')
> try:
> deadline(self._chained_checker.stop, timeout=self.STOP_TIMEOUT)
> except Timeout:
> log.error('Failed to stop all checkers within deadline.')
> # If the runner was alive when _shutdown was called, defer to the
> status_result,
> # otherwise the runner's terminal state is the preferred state.
> exit_status = runner_status or status_result
> self.send_update(
> self._driver,
> self._task_id,
> exit_status.status,
> status_result.reason)
> self.terminated.set()
> defer(self._driver.stop, delay=self.PERSISTENCE_WAIT)
> {noformat}
> However if runner.stop fails with a Timeout exception, the spawned
> AnonymousThread is not daemonized and causes the executor to fail to exit.
> This means that the cgroup will not be torn down and if the runner.stop
> actually failed, the process can stay alive even if TASK_KILLED was delivered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (AURORA-715) Retire GC Executor
[ https://issues.apache.org/jira/browse/AURORA-715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert reassigned AURORA-715: Assignee: Maxim Khutornenko > Retire GC Executor > -- > > Key: AURORA-715 > URL: https://issues.apache.org/jira/browse/AURORA-715 > Project: Aurora > Issue Type: Epic > Components: Technical Debt >Reporter: Chris Lambert >Assignee: Maxim Khutornenko > > Mesos plans to provide a task reconciliation API and sandbox history so that > we can retire the GC Executor. This Epic captures the work necessary to > complete this process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-698) aurora executor _shutdown deadline calls should be daemonized
[
https://issues.apache.org/jira/browse/AURORA-698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-698:
-
Sprint: Twitter Aurora Q2'15 Sprint 3 (was: Twitter Aurora Q2'15 Sprint 3,
Twitter Aurora Q2'15 Sprint 4)
> aurora executor _shutdown deadline calls should be daemonized
> -
>
> Key: AURORA-698
> URL: https://issues.apache.org/jira/browse/AURORA-698
> Project: Aurora
> Issue Type: Bug
> Components: Executor
>Reporter: brian wickman
>Assignee: brian wickman
>
> In the aurora executor shutdown method, we have deadline() calls:
> {noformat}
> def _shutdown(self, status_result):
> runner_status = self._runner.status
> try:
> deadline(self._runner.stop, timeout=self.STOP_TIMEOUT)
> except Timeout:
> log.error('Failed to stop runner within deadline.')
> try:
> deadline(self._chained_checker.stop, timeout=self.STOP_TIMEOUT)
> except Timeout:
> log.error('Failed to stop all checkers within deadline.')
> # If the runner was alive when _shutdown was called, defer to the
> status_result,
> # otherwise the runner's terminal state is the preferred state.
> exit_status = runner_status or status_result
> self.send_update(
> self._driver,
> self._task_id,
> exit_status.status,
> status_result.reason)
> self.terminated.set()
> defer(self._driver.stop, delay=self.PERSISTENCE_WAIT)
> {noformat}
> However if runner.stop fails with a Timeout exception, the spawned
> AnonymousThread is not daemonized and causes the executor to fail to exit.
> This means that the cgroup will not be torn down and if the runner.stop
> actually failed, the process can stay alive even if TASK_KILLED was delivered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1287) Scheduler in-memory DB needs a direct console access
[ https://issues.apache.org/jira/browse/AURORA-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1287: -- Story Points: 5 > Scheduler in-memory DB needs a direct console access > > > Key: AURORA-1287 > URL: https://issues.apache.org/jira/browse/AURORA-1287 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > We need a direct way of getting to scheduler SQL data for troubleshooting and > ad-hoc analysis purposes. Consider adding support for H2 web console: > http://www.h2database.com/html/quickstart.html. > The ideal solution should have a way to expose data in read-only and > read-write mode depending on the connection/credential settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1337) aurora executor should write checkpoints into sandbox
[ https://issues.apache.org/jira/browse/AURORA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1337: -- Story Points: 5 > aurora executor should write checkpoints into sandbox > - > > Key: AURORA-1337 > URL: https://issues.apache.org/jira/browse/AURORA-1337 > Project: Aurora > Issue Type: Task > Components: Executor >Reporter: brian wickman >Assignee: Maxim Khutornenko > > The thermos observer dual reads from /var/run/thermos and from the sandbox. > The executor should start writing checkpoints to the sandbox by default now > that 0.8.0 has been released. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1287) Scheduler in-memory DB needs a direct console access
[ https://issues.apache.org/jira/browse/AURORA-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1287: -- Sprint: Twitter Aurora Q2'15 Sprint 4 > Scheduler in-memory DB needs a direct console access > > > Key: AURORA-1287 > URL: https://issues.apache.org/jira/browse/AURORA-1287 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > We need a direct way of getting to scheduler SQL data for troubleshooting and > ad-hoc analysis purposes. Consider adding support for H2 web console: > http://www.h2database.com/html/quickstart.html. > The ideal solution should have a way to expose data in read-only and > read-write mode depending on the connection/credential settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1337) aurora executor should write checkpoints into sandbox
[ https://issues.apache.org/jira/browse/AURORA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1337: -- Sprint: Twitter Aurora Q2'15 Sprint 4 > aurora executor should write checkpoints into sandbox > - > > Key: AURORA-1337 > URL: https://issues.apache.org/jira/browse/AURORA-1337 > Project: Aurora > Issue Type: Task > Components: Executor >Reporter: brian wickman >Assignee: Maxim Khutornenko > > The thermos observer dual reads from /var/run/thermos and from the sandbox. > The executor should start writing checkpoints to the sandbox by default now > that 0.8.0 has been released. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1322) DB task store is enabled by default when staging a recovery
[ https://issues.apache.org/jira/browse/AURORA-1322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1322: -- Sprint: Twitter Aurora Q2'15 Sprint 3, Twitter Aurora Q2'15 Sprint 4 (was: Twitter Aurora Q2'15 Sprint 3) > DB task store is enabled by default when staging a recovery > --- > > Key: AURORA-1322 > URL: https://issues.apache.org/jira/browse/AURORA-1322 > Project: Aurora > Issue Type: Bug > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > The new DB task store implementation is turned on by default for the > TemporaryStorage since it uses a unit test DB init path (1). This makes > recovery from backup impossible if there is any task data triggering > constraint violations during staging a recovery from backup file. > (1) - > https://github.com/apache/aurora/blob/65df91bfd7e3a2ada38a5fe4d620e6373d0f59bf/src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java#L69 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1043) /logconfig does not work when the scheduler is behind SSL
[
https://issues.apache.org/jira/browse/AURORA-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1043:
--
Sprint: Twitter Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15 Sprint 2,
Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora
Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6, Twitter Aurora Q2'15 Sprint 1,
Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 3, Twitter Aurora
Q2'15 Sprint 4 (was: Twitter Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15
Sprint 2, Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter
Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6, Twitter Aurora Q2'15
Sprint 1, Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 3)
> /logconfig does not work when the scheduler is behind SSL
> -
>
> Key: AURORA-1043
> URL: https://issues.apache.org/jira/browse/AURORA-1043
> Project: Aurora
> Issue Type: Bug
>Reporter: Zameer Manji
>Assignee: Joshua Cohen
>
> When the /logconfig endpoint is behind SSL the JavaScript console is filled
> with the following errors:
> {noformat}
> Mixed Content: The page at 'https://localhost/logconfig' was loaded over
> HTTPS, but requested an insecure script
> 'http://ajax.googleapis.com/ajax/libs/jquery/1.10.1/jquery.min.js'. This
> request has been blocked; the content must be served over HTTPS.
> logconfig:4402 Uncaught ReferenceError: $ is not defined
> {noformat}
> As a result the endpoint does not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-698) aurora executor _shutdown deadline calls should be daemonized
[
https://issues.apache.org/jira/browse/AURORA-698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-698:
-
Sprint: Twitter Aurora Q2'15 Sprint 3, Twitter Aurora Q2'15 Sprint 4 (was:
Twitter Aurora Q2'15 Sprint 3)
> aurora executor _shutdown deadline calls should be daemonized
> -
>
> Key: AURORA-698
> URL: https://issues.apache.org/jira/browse/AURORA-698
> Project: Aurora
> Issue Type: Bug
> Components: Executor
>Reporter: brian wickman
>Assignee: brian wickman
>
> In the aurora executor shutdown method, we have deadline() calls:
> {noformat}
> def _shutdown(self, status_result):
> runner_status = self._runner.status
> try:
> deadline(self._runner.stop, timeout=self.STOP_TIMEOUT)
> except Timeout:
> log.error('Failed to stop runner within deadline.')
> try:
> deadline(self._chained_checker.stop, timeout=self.STOP_TIMEOUT)
> except Timeout:
> log.error('Failed to stop all checkers within deadline.')
> # If the runner was alive when _shutdown was called, defer to the
> status_result,
> # otherwise the runner's terminal state is the preferred state.
> exit_status = runner_status or status_result
> self.send_update(
> self._driver,
> self._task_id,
> exit_status.status,
> status_result.reason)
> self.terminated.set()
> defer(self._driver.stop, delay=self.PERSISTENCE_WAIT)
> {noformat}
> However if runner.stop fails with a Timeout exception, the spawned
> AnonymousThread is not daemonized and causes the executor to fail to exit.
> This means that the cgroup will not be torn down and if the runner.stop
> actually failed, the process can stay alive even if TASK_KILLED was delivered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1318) Benchmark snapshot restore
[ https://issues.apache.org/jira/browse/AURORA-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1318: -- Description: Details TBD by [~kevints] Story Points: 3 > Benchmark snapshot restore > -- > > Key: AURORA-1318 > URL: https://issues.apache.org/jira/browse/AURORA-1318 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Chris Lambert >Assignee: Kevin Sweeney > > Details TBD by [~kevints] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-274) Use JDK8 features in scheduler development
[ https://issues.apache.org/jira/browse/AURORA-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-274: - Assignee: Bill Farner Story Points: 2 > Use JDK8 features in scheduler development > -- > > Key: AURORA-274 > URL: https://issues.apache.org/jira/browse/AURORA-274 > Project: Aurora > Issue Type: Story > Components: Scheduler >Reporter: Kevin Sweeney >Assignee: Bill Farner >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-274) Use JDK8 features in scheduler development
[ https://issues.apache.org/jira/browse/AURORA-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-274: - Sprint: Twitter Aurora Q2'15 Sprint 3 > Use JDK8 features in scheduler development > -- > > Key: AURORA-274 > URL: https://issues.apache.org/jira/browse/AURORA-274 > Project: Aurora > Issue Type: Story > Components: Scheduler >Reporter: Kevin Sweeney >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-274) Use JDK8 features in scheduler development
[ https://issues.apache.org/jira/browse/AURORA-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-274: - Epic Colour: (was: ghx-label-3) Epic Name: (was: JDK8) Epic Status: (was: To Do) Issue Type: Story (was: Epic) > Use JDK8 features in scheduler development > -- > > Key: AURORA-274 > URL: https://issues.apache.org/jira/browse/AURORA-274 > Project: Aurora > Issue Type: Story > Components: Scheduler >Reporter: Kevin Sweeney >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-556) H2-backed implementation of TaskStore
[ https://issues.apache.org/jira/browse/AURORA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-556: - Sprint: Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 3 (was: Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15 Sprint 2) > H2-backed implementation of TaskStore > - > > Key: AURORA-556 > URL: https://issues.apache.org/jira/browse/AURORA-556 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > > This tracks behavior matching, but not necessarily performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1228) Use explicit status update acknowledgements to improve driver throughput.
[ https://issues.apache.org/jira/browse/AURORA-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1228: -- Sprint: Twitter Aurora Q2'15 Sprint 3 > Use explicit status update acknowledgements to improve driver throughput. > - > > Key: AURORA-1228 > URL: https://issues.apache.org/jira/browse/AURORA-1228 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Benjamin Mahler >Assignee: Benjamin Mahler > > Mesos 0.22.0 is released and includes the ability for schedulers to perform > explicit acknowledgments of status updates. > This allows schedulers to process status updates asynchronously from the > driver thread. Updates can then be processed in batches to reduce the > per-update processing overhead. These should help provide substantial > improvements to the schedulers update processing throughput. > I've placed this under the GC executor retirement effort, per AURORA-1047. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1296) INSTANCES_SPEC_ARGUMENT client option does not validate instance range
[ https://issues.apache.org/jira/browse/AURORA-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1296: -- Sprint: Twitter Aurora Q2'15 Sprint 3 > INSTANCES_SPEC_ARGUMENT client option does not validate instance range > -- > > Key: AURORA-1296 > URL: https://issues.apache.org/jira/browse/AURORA-1296 > Project: Aurora > Issue Type: Bug > Components: Client >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > An of order instance range (e.g. "/10-2") results in empty list of instances. > In case of a scheduler updater this leads to updating the entire job instead > of specific instances. > The offending line: > https://github.com/apache/aurora/blob/439a168f9cdd6fe58a12009b51bedad4ac152948/src/main/python/apache/aurora/client/cli/options.py#L74 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1047) Implement state reconciliation within the scheduler
[ https://issues.apache.org/jira/browse/AURORA-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1047: -- Sprint: Twitter Aurora Q2'15 Sprint 3 > Implement state reconciliation within the scheduler > --- > > Key: AURORA-1047 > URL: https://issues.apache.org/jira/browse/AURORA-1047 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: brian wickman >Assignee: Maxim Khutornenko > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1318) Benchmark snapshot restore
[ https://issues.apache.org/jira/browse/AURORA-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1318: -- Sprint: Twitter Aurora Q2'15 Sprint 3 > Benchmark snapshot restore > -- > > Key: AURORA-1318 > URL: https://issues.apache.org/jira/browse/AURORA-1318 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Chris Lambert >Assignee: Kevin Sweeney > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AURORA-1318) Benchmark snapshot restore
Chris Lambert created AURORA-1318: - Summary: Benchmark snapshot restore Key: AURORA-1318 URL: https://issues.apache.org/jira/browse/AURORA-1318 Project: Aurora Issue Type: Task Components: Scheduler Reporter: Chris Lambert Assignee: Kevin Sweeney -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1043) /logconfig does not work when the scheduler is behind SSL
[
https://issues.apache.org/jira/browse/AURORA-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1043:
--
Sprint: Twitter Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15 Sprint 2,
Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora
Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6, Twitter Aurora Q2'15 Sprint 1,
Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 3 (was: Twitter
Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15 Sprint 2, Twitter Aurora Q1'15
Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15 Sprint 5, Twitter
Aurora Q1'15 Sprint 6, Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15
Sprint 2)
> /logconfig does not work when the scheduler is behind SSL
> -
>
> Key: AURORA-1043
> URL: https://issues.apache.org/jira/browse/AURORA-1043
> Project: Aurora
> Issue Type: Bug
>Reporter: Zameer Manji
>Assignee: Joshua Cohen
>
> When the /logconfig endpoint is behind SSL the JavaScript console is filled
> with the following errors:
> {noformat}
> Mixed Content: The page at 'https://localhost/logconfig' was loaded over
> HTTPS, but requested an insecure script
> 'http://ajax.googleapis.com/ajax/libs/jquery/1.10.1/jquery.min.js'. This
> request has been blocked; the content must be served over HTTPS.
> logconfig:4402 Uncaught ReferenceError: $ is not defined
> {noformat}
> As a result the endpoint does not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1298) Make DbTaskStore performance suitable for use in production
[ https://issues.apache.org/jira/browse/AURORA-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1298: -- Sprint: Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 3 (was: Twitter Aurora Q2'15 Sprint 2) > Make DbTaskStore performance suitable for use in production > --- > > Key: AURORA-1298 > URL: https://issues.apache.org/jira/browse/AURORA-1298 > Project: Aurora > Issue Type: Story > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > > The DbTaskStore implementation has so far only targeted behavioral compliance > with InMemTaskStore. Use benchmarks to determine areas that require perf > improvements and address them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1311) Upgrade to latest h2 version
[ https://issues.apache.org/jira/browse/AURORA-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1311: -- Sprint: Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 3 (was: Twitter Aurora Q2'15 Sprint 2) > Upgrade to latest h2 version > > > Key: AURORA-1311 > URL: https://issues.apache.org/jira/browse/AURORA-1311 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > > We have been holding back on upgrading h2 since the latest release are > labeled as beta. As it turns out, the beta marker only applies to the new > default MVCC storage implementation, and the developers consider the builds > stable when MV_STORE is disabled \[1\]. > \[1\] https://groups.google.com/forum/#!topic/h2-database/3CDRKex_EgA -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-556) H2-backed implementation of TaskStore
[ https://issues.apache.org/jira/browse/AURORA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-556: - Sprint: Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15 Sprint 2, Twitter Aurora Q2'15 Sprint 3 (was: Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15 Sprint 2) > H2-backed implementation of TaskStore > - > > Key: AURORA-556 > URL: https://issues.apache.org/jira/browse/AURORA-556 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > > This tracks behavior matching, but not necessarily performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-556) H2-backed implementation of TaskStore
[ https://issues.apache.org/jira/browse/AURORA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-556: - Story Points: (was: 20) > H2-backed implementation of TaskStore > - > > Key: AURORA-556 > URL: https://issues.apache.org/jira/browse/AURORA-556 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1277) Create a design document for Health Checks for Updates
[ https://issues.apache.org/jira/browse/AURORA-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1277: -- Story Points: 5 Summary: Create a design document for Health Checks for Updates (was: Create a design document outlining the proposed changes) > Create a design document for Health Checks for Updates > -- > > Key: AURORA-1277 > URL: https://issues.apache.org/jira/browse/AURORA-1277 > Project: Aurora > Issue Type: Task > Components: Documentation >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1277) Create a design document outlining the proposed changes
[ https://issues.apache.org/jira/browse/AURORA-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1277: -- Sprint: Twitter Aurora Q2'15 Sprint 2 > Create a design document outlining the proposed changes > --- > > Key: AURORA-1277 > URL: https://issues.apache.org/jira/browse/AURORA-1277 > Project: Aurora > Issue Type: Task > Components: Documentation >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1231) Blocked updates missing from cluster-wide in progress view
[ https://issues.apache.org/jira/browse/AURORA-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1231: -- Sprint: Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15 Sprint 2 (was: Twitter Aurora Q2'15 Sprint 1) > Blocked updates missing from cluster-wide in progress view > -- > > Key: AURORA-1231 > URL: https://issues.apache.org/jira/browse/AURORA-1231 > Project: Aurora > Issue Type: Story > Components: Scheduler, UI >Reporter: David McLaughlin > > In the /updates endpoint, updates that are in ROLL_FORWARD_AWAITING_PULSE > state are not included under in progress updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-817) Add documentation for HTTP basic and kerberos authentication
[
https://issues.apache.org/jira/browse/AURORA-817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-817:
-
Sprint: Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15 Sprint 2 (was:
Twitter Aurora Q2'15 Sprint 1)
> Add documentation for HTTP basic and kerberos authentication
>
>
> Key: AURORA-817
> URL: https://issues.apache.org/jira/browse/AURORA-817
> Project: Aurora
> Issue Type: Task
> Components: Documentation, Security
>Reporter: Bill Farner
>Assignee: Kevin Sweeney
>
> Add a new markdown page linked from {{deploying-aurora-scheduler.md}} that
> offers instructions for using HTTP basic authentication for the web UI and
> API, as well as an example for integrating with kerberos.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-556) H2-backed implementation of TaskStore
[ https://issues.apache.org/jira/browse/AURORA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-556: - Sprint: Twitter Aurora Q2'15 Sprint 1, Twitter Aurora Q2'15 Sprint 2 (was: Twitter Aurora Q2'15 Sprint 1) > H2-backed implementation of TaskStore > - > > Key: AURORA-556 > URL: https://issues.apache.org/jira/browse/AURORA-556 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1043) /logconfig does not work when the scheduler is behind SSL
[
https://issues.apache.org/jira/browse/AURORA-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1043:
--
Sprint: Twitter Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15 Sprint 2,
Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora
Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6, Twitter Aurora Q2'15 Sprint 1,
Twitter Aurora Q2'15 Sprint 2 (was: Twitter Aurora Q1'15 Sprint 1, Twitter
Aurora Q1'15 Sprint 2, Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15
Sprint 4, Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6, Twitter
Aurora Q2'15 Sprint 1)
> /logconfig does not work when the scheduler is behind SSL
> -
>
> Key: AURORA-1043
> URL: https://issues.apache.org/jira/browse/AURORA-1043
> Project: Aurora
> Issue Type: Bug
>Reporter: Zameer Manji
>Assignee: Joshua Cohen
>
> When the /logconfig endpoint is behind SSL the JavaScript console is filled
> with the following errors:
> {noformat}
> Mixed Content: The page at 'https://localhost/logconfig' was loaded over
> HTTPS, but requested an insecure script
> 'http://ajax.googleapis.com/ajax/libs/jquery/1.10.1/jquery.min.js'. This
> request has been blocked; the content must be served over HTTPS.
> logconfig:4402 Uncaught ReferenceError: $ is not defined
> {noformat}
> As a result the endpoint does not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1231) Blocked updates missing from cluster-wide in progress view
[ https://issues.apache.org/jira/browse/AURORA-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1231: -- Sprint: Twitter Aurora Q2'15 Sprint 1 > Blocked updates missing from cluster-wide in progress view > -- > > Key: AURORA-1231 > URL: https://issues.apache.org/jira/browse/AURORA-1231 > Project: Aurora > Issue Type: Story > Components: Scheduler, UI >Reporter: David McLaughlin > > In the /updates endpoint, updates that are in ROLL_FORWARD_AWAITING_PULSE > state are not included under in progress updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1231) Blocked updates missing from cluster-wide in progress view
[ https://issues.apache.org/jira/browse/AURORA-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1231: -- Story Points: 2 > Blocked updates missing from cluster-wide in progress view > -- > > Key: AURORA-1231 > URL: https://issues.apache.org/jira/browse/AURORA-1231 > Project: Aurora > Issue Type: Story > Components: Scheduler, UI >Reporter: David McLaughlin > > In the /updates endpoint, updates that are in ROLL_FORWARD_AWAITING_PULSE > state are not included under in progress updates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-817) Add documentation for HTTP basic and kerberos authentication
[
https://issues.apache.org/jira/browse/AURORA-817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-817:
-
Story Points: 2
> Add documentation for HTTP basic and kerberos authentication
>
>
> Key: AURORA-817
> URL: https://issues.apache.org/jira/browse/AURORA-817
> Project: Aurora
> Issue Type: Task
> Components: Documentation, Security
>Reporter: Bill Farner
>Assignee: Kevin Sweeney
>
> Add a new markdown page linked from {{deploying-aurora-scheduler.md}} that
> offers instructions for using HTTP basic authentication for the web UI and
> API, as well as an example for integrating with kerberos.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-817) Add documentation for HTTP basic and kerberos authentication
[
https://issues.apache.org/jira/browse/AURORA-817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-817:
-
Sprint: Twitter Aurora Q2'15 Sprint 1
> Add documentation for HTTP basic and kerberos authentication
>
>
> Key: AURORA-817
> URL: https://issues.apache.org/jira/browse/AURORA-817
> Project: Aurora
> Issue Type: Task
> Components: Documentation, Security
>Reporter: Bill Farner
>Assignee: Kevin Sweeney
>
> Add a new markdown page linked from {{deploying-aurora-scheduler.md}} that
> offers instructions for using HTTP basic authentication for the web UI and
> API, as well as an example for integrating with kerberos.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1219) Improve preemptor efficiency
[ https://issues.apache.org/jira/browse/AURORA-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1219: -- Story Points: 5 > Improve preemptor efficiency > > > Key: AURORA-1219 > URL: https://issues.apache.org/jira/browse/AURORA-1219 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > While AURORA-1158 makes the preemptor asynchronous and helps scheduling loop > performance, it loses some preemption efficiency due to adapting the original > algorithm to work asynchronously: > - Preemptor repeatedly searches for preemption slots with the same > ClusterState increasing the probability of victim collisions. > - Slave/offer/resource mappings are redundantly recomputed for every pending > task slot search. > - Pending tasks are repeatedly sized up against every slave (as opposed to > the other way around) making internal loop optimizations more expensive (i.e. > pending task count is expected to be << slave count under normal conditions). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (AURORA-1214) Reevaluate "max_schedule_attempts_per_sec" with asynchronous preemptor
[
https://issues.apache.org/jira/browse/AURORA-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert reassigned AURORA-1214:
-
Assignee: Maxim Khutornenko
> Reevaluate "max_schedule_attempts_per_sec" with asynchronous preemptor
> --
>
> Key: AURORA-1214
> URL: https://issues.apache.org/jira/browse/AURORA-1214
> Project: Aurora
> Issue Type: Task
> Components: Scheduler
>Reporter: Maxim Khutornenko
>Assignee: Maxim Khutornenko
>
> With step 2 of AURORA-1158 completed, the preemptor is removed from the
> critical scheduling loop making scheduling much faster (perf comparison is
> available here: https://reviews.apache.org/r/32225/)
> Reevaluate what the {{max_schedule_attempts_per_sec}} should be now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1214) Reevaluate "max_schedule_attempts_per_sec" with asynchronous preemptor
[
https://issues.apache.org/jira/browse/AURORA-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1214:
--
Story Points: 2
> Reevaluate "max_schedule_attempts_per_sec" with asynchronous preemptor
> --
>
> Key: AURORA-1214
> URL: https://issues.apache.org/jira/browse/AURORA-1214
> Project: Aurora
> Issue Type: Task
> Components: Scheduler
>Reporter: Maxim Khutornenko
>
> With step 2 of AURORA-1158 completed, the preemptor is removed from the
> critical scheduling loop making scheduling much faster (perf comparison is
> available here: https://reviews.apache.org/r/32225/)
> Reevaluate what the {{max_schedule_attempts_per_sec}} should be now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1214) Reevaluate "max_schedule_attempts_per_sec" with asynchronous preemptor
[
https://issues.apache.org/jira/browse/AURORA-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1214:
--
Sprint: Twitter Aurora Q2'15 Sprint 1
> Reevaluate "max_schedule_attempts_per_sec" with asynchronous preemptor
> --
>
> Key: AURORA-1214
> URL: https://issues.apache.org/jira/browse/AURORA-1214
> Project: Aurora
> Issue Type: Task
> Components: Scheduler
>Reporter: Maxim Khutornenko
>
> With step 2 of AURORA-1158 completed, the preemptor is removed from the
> critical scheduling loop making scheduling much faster (perf comparison is
> available here: https://reviews.apache.org/r/32225/)
> Reevaluate what the {{max_schedule_attempts_per_sec}} should be now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-556) H2-backed implementation of TaskStore
[ https://issues.apache.org/jira/browse/AURORA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-556: - Story Points: 20 > H2-backed implementation of TaskStore > - > > Key: AURORA-556 > URL: https://issues.apache.org/jira/browse/AURORA-556 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-556) H2-backed implementation of TaskStore
[ https://issues.apache.org/jira/browse/AURORA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-556: - Sprint: Twitter Aurora Q2'15 Sprint 1 > H2-backed implementation of TaskStore > - > > Key: AURORA-556 > URL: https://issues.apache.org/jira/browse/AURORA-556 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (AURORA-556) H2-backed implementation of TaskStore
[ https://issues.apache.org/jira/browse/AURORA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert reassigned AURORA-556: Assignee: Bill Farner > H2-backed implementation of TaskStore > - > > Key: AURORA-556 > URL: https://issues.apache.org/jira/browse/AURORA-556 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Bill Farner >Assignee: Bill Farner > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1219) Improve preemptor efficiency
[ https://issues.apache.org/jira/browse/AURORA-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1219: -- Sprint: Twitter Aurora Q2'15 Sprint 1 > Improve preemptor efficiency > > > Key: AURORA-1219 > URL: https://issues.apache.org/jira/browse/AURORA-1219 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > While AURORA-1158 makes the preemptor asynchronous and helps scheduling loop > performance, it loses some preemption efficiency due to adapting the original > algorithm to work asynchronously: > - Preemptor repeatedly searches for preemption slots with the same > ClusterState increasing the probability of victim collisions. > - Slave/offer/resource mappings are redundantly recomputed for every pending > task slot search. > - Pending tasks are repeatedly sized up against every slave (as opposed to > the other way around) making internal loop optimizations more expensive (i.e. > pending task count is expected to be << slave count under normal conditions). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1043) /logconfig does not work when the scheduler is behind SSL
[
https://issues.apache.org/jira/browse/AURORA-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1043:
--
Sprint: Twitter Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15 Sprint 2,
Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora
Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6, Twitter Aurora Q2'15 Sprint 1
(was: Twitter Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15 Sprint 2, Twitter
Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15
Sprint 5, Twitter Aurora Q1'15 Sprint 6)
> /logconfig does not work when the scheduler is behind SSL
> -
>
> Key: AURORA-1043
> URL: https://issues.apache.org/jira/browse/AURORA-1043
> Project: Aurora
> Issue Type: Bug
>Reporter: Zameer Manji
>Assignee: Joshua Cohen
>
> When the /logconfig endpoint is behind SSL the JavaScript console is filled
> with the following errors:
> {noformat}
> Mixed Content: The page at 'https://localhost/logconfig' was loaded over
> HTTPS, but requested an insecure script
> 'http://ajax.googleapis.com/ajax/libs/jquery/1.10.1/jquery.min.js'. This
> request has been blocked; the content must be served over HTTPS.
> logconfig:4402 Uncaught ReferenceError: $ is not defined
> {noformat}
> As a result the endpoint does not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1054) src.test.python.apache.aurora.executor.thermos_task_runner appears to be flaky
[
https://issues.apache.org/jira/browse/AURORA-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1054:
--
Sprint: Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4,
Twitter Aurora Q1'15 Sprint 5 (was: Twitter Aurora Q1'15 Sprint 3, Twitter
Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15
Sprint 6)
> src.test.python.apache.aurora.executor.thermos_task_runner appears to be flaky
> --
>
> Key: AURORA-1054
> URL: https://issues.apache.org/jira/browse/AURORA-1054
> Project: Aurora
> Issue Type: Story
> Components: Executor
>Reporter: Bill Farner
>Assignee: brian wickman
>
> I've seen this test fail on a few reviews recently, but succeeds on a retry.
> {noformat}
> FAILURES
> TestThermosTaskRunnerIntegration.test_integration_stop
>
> self =
> 0x7ff3e1dc6fd0>
>
> [1mdef test_integration_stop(self):[0m
> [1m with self.yield_sleepy(ThermosTaskRunner,
> sleep=1000, exit_code=0) as task_runner:[0m
> [1mtask_runner.start()[0m
> [1mtask_runner.forked.wait()[0m
> [1m[0m
> [1massert task_runner.status is None[0m
> [1m[0m
> [1mtask_runner.stop()[0m
> [1m[0m
> [1m> assert task_runner.status is not None[0m
> [1m[31mE assert None is not None[0m
> [1m[31mE+ where None =
> 0x7ff3e490>.status[0m
>
>
> src/test/python/apache/aurora/executor/test_thermos_task_runner.py:175:
> AssertionError
> -- Captured stderr call --
> Writing log files to disk in /tmp/user/2396/tmpqHF5bz
> ERROR] Could not quitquitquit runner: Cannot take
> control of a task in terminal state.
> generated xml file:
> /home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/src.test.python.apache.aurora.executor.thermos_task_runner.xml
>
> [1m[31m== 1 failed, 7 passed in 81.16 seconds
> ===[0m
> src.test.python.apache.aurora.admin.admin
>. SUCCESS
> src.test.python.apache.aurora.admin.host_maintenance
>. SUCCESS
> src.test.python.apache.aurora.admin.maintenance
>. SUCCESS
> src.test.python.apache.aurora.client.api.api
>. SUCCESS
>
> src.test.python.apache.aurora.client.api.instance_watcher
> . SUCCESS
> src.test.python.apache.aurora.client.api.job_monitor
>. SUCCESS
> src.test.python.apache.aurora.client.api.mux
>. SUCCESS
> src.test.python.apache.aurora.client.api.quota_check
>. SUCCESS
> src.test.python.apache.aurora.client.api.restarter
>. SUCCESS
>
> src.test.python.apache.aurora.client.api.scheduler_client
> . SUCCESS
> src.test.python.apache.aurora.client.api.sla
>. SUCCESS
> src.test.python.apache.aurora.client.api.task_util
>. SUCCESS
> src.test.python.apache.aurora.client.api.updater
>. SUCCESS
> src.test.python.apache.aurora.client.api.updater_util
>. SUCCESS
> src.test.python.apache.aurora.client.base
>. SUCCESS
> src.test.python.apache.aurora.client.binding_helper
>. SUCCESS
> src.test.python.apache.aurora.client.cli.api
>. SUCCESS
> src.test.python.apache.aurora.client.cli.client
>. SUCCESS
> src.test.python.apache.aurora.client.cli.command_hooks
>. SUC
[jira] [Deleted] (AURORA-1270) Leverage the new Mesos task reconciliation API in the scheduler
[ https://issues.apache.org/jira/browse/AURORA-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert deleted AURORA-1270: -- > Leverage the new Mesos task reconciliation API in the scheduler > --- > > Key: AURORA-1270 > URL: https://issues.apache.org/jira/browse/AURORA-1270 > Project: Aurora > Issue Type: Epic >Reporter: Chris Lambert >Assignee: Benjamin Mahler > > State inconsistencies can arise between the framework scheduler's view of > tasks and the view of tasks within Mesos. > Frameworks, like Aurora, have had to compensate for these inconsistencies by > running a specialized executor on the slave that reconciles what happened on > the slave against what the scheduler thinks is the current state of tasks. > Instead, Aurora should leverage the new Mesos reconciliation API (MESOS-1407) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AURORA-1270) Leverage the new Mesos task reconciliation API in the scheduler
Chris Lambert created AURORA-1270: - Summary: Leverage the new Mesos task reconciliation API in the scheduler Key: AURORA-1270 URL: https://issues.apache.org/jira/browse/AURORA-1270 Project: Aurora Issue Type: Epic Components: Scheduler Reporter: Chris Lambert Assignee: Benjamin Mahler State inconsistencies can arise between the framework scheduler's view of tasks and the view of tasks within Mesos. Frameworks, like Aurora, have had to compensate for these inconsistencies by running a specialized executor on the slave that reconciles what happened on the slave against what the scheduler thinks is the current state of tasks. Instead, Aurora should leverage the new Mesos reconciliation API (MESOS-1407) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-915) create strict mode for .aurora config
[
https://issues.apache.org/jira/browse/AURORA-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-915:
-
Assignee: brian wickman
Story Points: 8
> create strict mode for .aurora config
> -
>
> Key: AURORA-915
> URL: https://issues.apache.org/jira/browse/AURORA-915
> Project: Aurora
> Issue Type: Task
> Components: Client
>Reporter: brian wickman
>Assignee: brian wickman
>
> I propose we have a strict mode for .aurora configuration (pystachio) that
> prevents importing python modules (including os and sys.) Possibly we
> snapshot os.environ and provide a binding helper to give access to it. For
> people who need things like the current user, perhaps provide a default
> binding like {{\{\{system.user\}\}}} and the like. We are getting bitten by
> people adding too much sophistication into .aurora configuration like full
> blown sys.args introspection and web clients, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (AURORA-334) Replace in-memory storage with ORM backed by H2
[
https://issues.apache.org/jira/browse/AURORA-334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert reassigned AURORA-334:
Assignee: Bill Farner
> Replace in-memory storage with ORM backed by H2
> ---
>
> Key: AURORA-334
> URL: https://issues.apache.org/jira/browse/AURORA-334
> Project: Aurora
> Issue Type: Epic
> Components: Scheduler
>Reporter: Bill Farner
>Assignee: Bill Farner
>
> See thread \[1\] on the dev mailing list for more context. Ultimately we've
> found creeping complexity by hand-rolling the in-memory storage schema, and
> believe that a relational database will make development easier. We've
> chosen H2 to serve as the in-memory storage engine, and will replace
> individual Store implementations until all of
> {{org.apache.aurora.scheduler.storage.mem}} is removed.
> \[1\]
> http://mail-archives.apache.org/mod_mbox/incubator-aurora-dev/201404.mbox/%3CCAGRA8uPKh81F3eCp%2Bf7fE84%2B2m%2ByzounTEcbhNpTaUFWqFt-Xw%40mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-334) Replace in-memory storage with ORM backed by H2
[
https://issues.apache.org/jira/browse/AURORA-334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-334:
-
Epic Name: In-memory DB (was: Replace in-memory storage with ORM backed by
H2)
> Replace in-memory storage with ORM backed by H2
> ---
>
> Key: AURORA-334
> URL: https://issues.apache.org/jira/browse/AURORA-334
> Project: Aurora
> Issue Type: Epic
> Components: Scheduler
>Reporter: Bill Farner
>
> See thread \[1\] on the dev mailing list for more context. Ultimately we've
> found creeping complexity by hand-rolling the in-memory storage schema, and
> believe that a relational database will make development easier. We've
> chosen H2 to serve as the in-memory storage engine, and will replace
> individual Store implementations until all of
> {{org.apache.aurora.scheduler.storage.mem}} is removed.
> \[1\]
> http://mail-archives.apache.org/mod_mbox/incubator-aurora-dev/201404.mbox/%3CCAGRA8uPKh81F3eCp%2Bf7fE84%2B2m%2ByzounTEcbhNpTaUFWqFt-Xw%40mail.gmail.com%3E
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1211) Spread snapshot creation and backup file write
[ https://issues.apache.org/jira/browse/AURORA-1211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1211: -- Adding to the AURORA-999 performance improvements epic. > Spread snapshot creation and backup file write > -- > > Key: AURORA-1211 > URL: https://issues.apache.org/jira/browse/AURORA-1211 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > Currently, backup files are written asynchronously and overlay with the peak > of GC cleaning up after persisting snapshot in native log. We should > generally avoid overlaying resource-heavy activities when possible to avoid > troubleshooting complicated performance related issues. > Consider delaying backup file write (e.g. for 1 minute) to clearly separate > its impact from the snapshot GC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1087) Investigate truncating the replicated log asynchronously
[ https://issues.apache.org/jira/browse/AURORA-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1087: -- Component/s: Technical Debt Removing this from the AURORA-999 epic, as it is not urgent. > Investigate truncating the replicated log asynchronously > > > Key: AURORA-1087 > URL: https://issues.apache.org/jira/browse/AURORA-1087 > Project: Aurora > Issue Type: Story > Components: Scheduler, Stress Testing, Technical Debt >Reporter: Kevin Sweeney >Priority: Minor > > Right now we perform a log truncation operation after writing a snapshot > while holding the global write lock. However this isn't necessary - after > we've written a Snapshot we can asynchronously truncate to the new log head > position. > Investigate if there's a performance benefit here to justify this refactor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1241) Wrong label on table heading for in progress updates
[ https://issues.apache.org/jira/browse/AURORA-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1241: -- Component/s: Technical Debt > Wrong label on table heading for in progress updates > > > Key: AURORA-1241 > URL: https://issues.apache.org/jira/browse/AURORA-1241 > Project: Aurora > Issue Type: Story > Components: Technical Debt, UI >Reporter: David McLaughlin >Priority: Minor > Labels: newbie > > On the /updates page the in progress table has a column with a heading of > "ended." The update is still in progress so it should read more like "last > modified". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1031) Replace libmesos w/ Pesos
[ https://issues.apache.org/jira/browse/AURORA-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1031: -- Sprint: Twitter Aurora Q1'15 Sprint 5 (was: Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6) > Replace libmesos w/ Pesos > - > > Key: AURORA-1031 > URL: https://issues.apache.org/jira/browse/AURORA-1031 > Project: Aurora > Issue Type: Task > Components: Executor, Technical Debt >Reporter: Chris Lambert >Assignee: brian wickman >Priority: Minor > > Replace the libmesos native dependency with > [Pesos|https://github.com/wickman/pesos], to simplify the build and reduce > memory footprint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1043) /logconfig does not work when the scheduler is behind SSL
[
https://issues.apache.org/jira/browse/AURORA-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1043:
--
Sprint: Twitter Aurora Q1'15 Sprint 1, Twitter Aurora Q1'15 Sprint 2,
Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora
Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was: Twitter Aurora Q1'15
Sprint 1, Twitter Aurora Q1'15 Sprint 2, Twitter Aurora Q1'15 Sprint 3, Twitter
Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15 Sprint 5)
> /logconfig does not work when the scheduler is behind SSL
> -
>
> Key: AURORA-1043
> URL: https://issues.apache.org/jira/browse/AURORA-1043
> Project: Aurora
> Issue Type: Bug
>Reporter: Zameer Manji
>Assignee: Joshua Cohen
>
> When the /logconfig endpoint is behind SSL the JavaScript console is filled
> with the following errors:
> {noformat}
> Mixed Content: The page at 'https://localhost/logconfig' was loaded over
> HTTPS, but requested an insecure script
> 'http://ajax.googleapis.com/ajax/libs/jquery/1.10.1/jquery.min.js'. This
> request has been blocked; the content must be served over HTTPS.
> logconfig:4402 Uncaught ReferenceError: $ is not defined
> {noformat}
> As a result the endpoint does not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-1158) Consider finding preemption slots asynchronously
[ https://issues.apache.org/jira/browse/AURORA-1158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1158: -- Sprint: Twitter Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was: Twitter Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15 Sprint 5) > Consider finding preemption slots asynchronously > - > > Key: AURORA-1158 > URL: https://issues.apache.org/jira/browse/AURORA-1158 > Project: Aurora > Issue Type: Task > Components: Scheduler >Reporter: Maxim Khutornenko >Assignee: Maxim Khutornenko > > The current Preemptor implementation performs synchronous preemption victim > search. This isn't necessary as resource swap cannot be performed within the > same scheduling loop due to victims taking time to vacate their premises > (being killed). > Consider asynchronous victim search/killing instead. That will move the heavy > lifting out of the critical scheduling loop and remove "flat" preemptor perf > tax applied on every failed task scheduling/assignment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-813) Add ability to configure client to use Kerberos
[ https://issues.apache.org/jira/browse/AURORA-813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-813: - Sprint: Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was: Twitter Aurora Q1'15 Sprint 5) > Add ability to configure client to use Kerberos > --- > > Key: AURORA-813 > URL: https://issues.apache.org/jira/browse/AURORA-813 > Project: Aurora > Issue Type: Task > Components: Security >Reporter: Chris Lambert >Assignee: Maxim Khutornenko > > Add ability to configure client to use Kerberos (depends on AURORA-812, can > be developed in parallel). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1031) Replace libmesos w/ Pesos
[ https://issues.apache.org/jira/browse/AURORA-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-1031: -- Sprint: Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was: Twitter Aurora Q1'15 Sprint 5) > Replace libmesos w/ Pesos > - > > Key: AURORA-1031 > URL: https://issues.apache.org/jira/browse/AURORA-1031 > Project: Aurora > Issue Type: Task > Components: Executor, Technical Debt >Reporter: Chris Lambert >Assignee: brian wickman >Priority: Minor > > Replace the libmesos native dependency with > [Pesos|https://github.com/wickman/pesos], to simplify the build and reduce > memory footprint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-1054) src.test.python.apache.aurora.executor.thermos_task_runner appears to be flaky
[
https://issues.apache.org/jira/browse/AURORA-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1054:
--
Sprint: Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4,
Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was: Twitter
Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15
Sprint 5)
> src.test.python.apache.aurora.executor.thermos_task_runner appears to be flaky
> --
>
> Key: AURORA-1054
> URL: https://issues.apache.org/jira/browse/AURORA-1054
> Project: Aurora
> Issue Type: Story
> Components: Executor
>Reporter: Bill Farner
>Assignee: brian wickman
>
> I've seen this test fail on a few reviews recently, but succeeds on a retry.
> {noformat}
> FAILURES
> TestThermosTaskRunnerIntegration.test_integration_stop
>
> self =
> 0x7ff3e1dc6fd0>
>
> [1mdef test_integration_stop(self):[0m
> [1m with self.yield_sleepy(ThermosTaskRunner,
> sleep=1000, exit_code=0) as task_runner:[0m
> [1mtask_runner.start()[0m
> [1mtask_runner.forked.wait()[0m
> [1m[0m
> [1massert task_runner.status is None[0m
> [1m[0m
> [1mtask_runner.stop()[0m
> [1m[0m
> [1m> assert task_runner.status is not None[0m
> [1m[31mE assert None is not None[0m
> [1m[31mE+ where None =
> 0x7ff3e490>.status[0m
>
>
> src/test/python/apache/aurora/executor/test_thermos_task_runner.py:175:
> AssertionError
> -- Captured stderr call --
> Writing log files to disk in /tmp/user/2396/tmpqHF5bz
> ERROR] Could not quitquitquit runner: Cannot take
> control of a task in terminal state.
> generated xml file:
> /home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/src.test.python.apache.aurora.executor.thermos_task_runner.xml
>
> [1m[31m== 1 failed, 7 passed in 81.16 seconds
> ===[0m
> src.test.python.apache.aurora.admin.admin
>. SUCCESS
> src.test.python.apache.aurora.admin.host_maintenance
>. SUCCESS
> src.test.python.apache.aurora.admin.maintenance
>. SUCCESS
> src.test.python.apache.aurora.client.api.api
>. SUCCESS
>
> src.test.python.apache.aurora.client.api.instance_watcher
> . SUCCESS
> src.test.python.apache.aurora.client.api.job_monitor
>. SUCCESS
> src.test.python.apache.aurora.client.api.mux
>. SUCCESS
> src.test.python.apache.aurora.client.api.quota_check
>. SUCCESS
> src.test.python.apache.aurora.client.api.restarter
>. SUCCESS
>
> src.test.python.apache.aurora.client.api.scheduler_client
> . SUCCESS
> src.test.python.apache.aurora.client.api.sla
>. SUCCESS
> src.test.python.apache.aurora.client.api.task_util
>. SUCCESS
> src.test.python.apache.aurora.client.api.updater
>. SUCCESS
> src.test.python.apache.aurora.client.api.updater_util
>. SUCCESS
> src.test.python.apache.aurora.client.base
>. SUCCESS
> src.test.python.apache.aurora.client.binding_helper
>. SUCCESS
> src.test.python.apache.aurora.client.cli.api
>. SUCCESS
> src.test.python.apache.aurora.client.cli.client
>. SUCCESS
> src.test.python.apache.aurora.client.cli.command_hooks
>. SUC
[jira] [Updated] (AURORA-1187) Create JobKey-scoped Permissions and apply to AuroraSchedulerManager
[
https://issues.apache.org/jira/browse/AURORA-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Lambert updated AURORA-1187:
--
Sprint: Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was:
Twitter Aurora Q1'15 Sprint 5)
> Create JobKey-scoped Permissions and apply to AuroraSchedulerManager
>
>
> Key: AURORA-1187
> URL: https://issues.apache.org/jira/browse/AURORA-1187
> Project: Aurora
> Issue Type: Story
> Components: Scheduler, Security
>Reporter: Kevin Sweeney
>Assignee: Kevin Sweeney
>
> https://reviews.apache.org/r/31820/ added support for coarse-grained
> permissions based on the thrift API method name. Support finer-grained
> permissions scoped down to the job
> For example calls to
> {code}
> auroraSchedulerManager.descheduleCronJob(JobKey(role='ksweeney',
> environment='prod', name='download_lunch_menu'), None, None)
> {code}
> currently check for the {{thrift.AuroraSchedulerManager:descheduleCronJob}}
> Permission when they should check for the
> {{thrift.AuroraSchedulerManager:descheduleCronJob:ksweeney:prod:download_lunch_menu}}
> Permission.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AURORA-810) Add an end-to-end test case for http basic auth
[ https://issues.apache.org/jira/browse/AURORA-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-810: - Sprint: Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was: Twitter Aurora Q1'15 Sprint 3, Twitter Aurora Q1'15 Sprint 4, Twitter Aurora Q1'15 Sprint 5) > Add an end-to-end test case for http basic auth > --- > > Key: AURORA-810 > URL: https://issues.apache.org/jira/browse/AURORA-810 > Project: Aurora > Issue Type: Task > Components: Security >Reporter: Chris Lambert >Assignee: Kevin Sweeney > > * Simple, netrc-based > * Including end-to-end tests -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AURORA-812) Wire in SPNEGO HTTP module
[ https://issues.apache.org/jira/browse/AURORA-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambert updated AURORA-812: - Sprint: Twitter Aurora Q1'15 Sprint 5, Twitter Aurora Q1'15 Sprint 6 (was: Twitter Aurora Q1'15 Sprint 5) > Wire in SPNEGO HTTP module > -- > > Key: AURORA-812 > URL: https://issues.apache.org/jira/browse/AURORA-812 > Project: Aurora > Issue Type: Task > Components: Security >Reporter: Chris Lambert >Assignee: Kevin Sweeney > > Wire in SPNEGO HTTP module, expose as flags in ShiroAuthModule. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
