Re: [Spacewalk-devel] [PATCH] 'es improve channel status display
On Friday 01 of March 2013 13:49:55 Duncan Mac-Vicar P. wrote: On 02/28/2013 10:13 PM, Tomas Lestach wrote: The only part of code, that generates the repodata is the taskomatic reporegen worker. I'd suggest to set the state to 'in progress' after a generation starts, to set it to 'failed' if the worker throws an exception (we saw some out of memory exceptions in case of large channels) and to 'completed', when the worker succeeds. Yes, but then I need persistence for the state, which is what I have been avoiding until now. No, you don't. Just do an update using a CachedStatement. Initial state may be set according to the presence of the repomd.xml file in the appropriate directory to either unknown(/missing) or completed. I wouldn't consider a 'scheduled' state due to the need to periodically query the rhnRepoRegenQueue table. Why would that be a problem? Well, ok. :-) We can check for the queries in the rhnRepoRegenQueue at page request. Tomas -- Tomas Lestach RHN Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] 'es improve channel status display
While it is easy to calculate the status of the repo-sync, -- as every channel sync has _one_ taskomatic job which has a result --, the status of the metadata generation is in _one_ Taskomatic job which creates workers for every channel. However the status is lost one it runs. The items in the worker table are basically the scheduled and running. If we want to improve here, I see 2 alternatives: - Improve this table (workers) to keep the failed and successful workers for a channel metadata generation run. - Keep it like it is, but make some assumptions: - If the metadata directory does not exists, it has not run yet (so it is not completed, but unknown). - If the metadata dir is there, but there is a FAILED file (written when the worker throws) then it is failed - Michael also suggested looking for .new files to detect failed state. - If the metadata dir is there but not the above, then it is completed. What do you think? Duncan ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] 'es improve channel status display
- Original Message - On 02/14/2013 05:59 PM, Tomas Lestach wrote: Hi Duncan, thanks for your patches. I committed following: eb7beb3277fb8db48376e44c058bde8ad5d1be45 1db45d611fbc307b4eac1fa35d25ed3b30dc6fac 668f52d928590b4bc7e2d3c955f2b6a59e5d3fb8 I didn't commit following ones: *TaskoRun Log tail: Handle the case where more bytes are requested than the current size of the log. - I rather return the whole log file in case more bytes are requested, so I committed this code: 6012adfcc5d4ae182c164a3e64e442255f44751c - anyway thank you for pointing on that situation Makes sense * both ChannelStatus commits - generally I like the idea, implementation, spinning icons, but I mean they aren't complete - I didn't find a way the 'Repo Cache Status' would say 'Completed' or 'Failed'. For the reposync, the result of the task will determine those statuses. For the repodata generation, only in-progress and scheduled is supported. But you are right in this second case if there are no items in the queue, it may be seen as completed instead of unknown. - are you sure you sent everything? - even if my repo was regenerated, I saw just '(none)' on the page Probably because of the above. The repodata-generation should return Completed in the else case and not Unknown. - please check and I'd be happy to commit your patches Btw. would you run checkstyle before sending patches, please? I am working on a ant-task to do checkstyle from the command line. Currently it is done in the rpm and not very comfortable. You always can run 'ant checkstyle' from the command line as we have the ant task defined. However because of the different incompatible checkstyle versions you may use it on RHEL6 only. Thanks for the review, I will improve the patches, merge the ones you committed and try again soon. Thank you, Tomas -- Tomas Lestach RHN Satellite Engineering, Red Hat Duncan ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] 'es improve channel status display
On 02/14/2013 05:59 PM, Tomas Lestach wrote: Hi Duncan, thanks for your patches. I committed following: eb7beb3277fb8db48376e44c058bde8ad5d1be45 1db45d611fbc307b4eac1fa35d25ed3b30dc6fac 668f52d928590b4bc7e2d3c955f2b6a59e5d3fb8 I didn't commit following ones: *TaskoRun Log tail: Handle the case where more bytes are requested than the current size of the log. - I rather return the whole log file in case more bytes are requested, so I committed this code: 6012adfcc5d4ae182c164a3e64e442255f44751c - anyway thank you for pointing on that situation Makes sense * both ChannelStatus commits - generally I like the idea, implementation, spinning icons, but I mean they aren't complete - I didn't find a way the 'Repo Cache Status' would say 'Completed' or 'Failed'. For the reposync, the result of the task will determine those statuses. For the repodata generation, only in-progress and scheduled is supported. But you are right in this second case if there are no items in the queue, it may be seen as completed instead of unknown. - are you sure you sent everything? - even if my repo was regenerated, I saw just '(none)' on the page Probably because of the above. The repodata-generation should return Completed in the else case and not Unknown. - please check and I'd be happy to commit your patches Btw. would you run checkstyle before sending patches, please? I am working on a ant-task to do checkstyle from the command line. Currently it is done in the rpm and not very comfortable. Thanks for the review, I will improve the patches, merge the ones you committed and try again soon. Duncan ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
[Spacewalk-devel] [PATCH] 'es improve channel status display
The following patches improve the status of a channel by taking into
account both download/mirror _and_ repodata generation.
The download/sync status was a bit tricky to get, as the taskomatic
table is made polymorphic by serializing java objects which in turn
contain the channel_id, so you can't just SQL it.
Some bugs are fixed in order to get everything working, specially
failing when exit code of external programs is not successful, and
getting the tail log for logs smaller than the requested tail.
The status is exposed as a ChannelStatus class.
This could be displayed as icons (spinning wheel, exclamation) in the
overview lists, but as I don't have a concept for the UI I kept it in
the channel details page, as the half status was already there.
The patches are intended for review, but are not yet rebased against master.
commit c523b18667ace8acd5dd835378e87d0be45f9473
Author: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Thu Jan 31 17:45:25 2013 +0100
Improve the Channel Details page by using the status generated
by ChannelStatus, which includes also repo-sync information.
commit 080232549c07e6c38caaebbaa640530efebcd614
Author: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Thu Jan 31 11:31:36 2013 +0100
Introduce ChannelStatus as an agregate way to query both the repo-sync
and repodata generation status of a channel.
The class also carries a log to transport debug information to the
admin. In the future, more data or sub-statuses could be added to
this class.
The ChannelStatus is exposed in ChannelManager::statusForLabel(label)
and ChannelManager::isChannelLabelInProgress(label) gets deprecated
and reimplemented using ChannelManager::statusForLabel(label).
The Task queries get a new query to ask for candidate repodata
generation workers and the active ones gets renamed to keep the
names meanful and consistent.
commit caec97d8d2b88f008cb0e70391cb572207f6b1da
Author: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Wed Jan 30 18:37:52 2013 +0100
RhnJavaJob: Do not ignore the exit code for external programs.
commit 630870f813d2deaf2b95d8a133fdfd78e110c05a
Author: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Wed Jan 30 18:31:01 2013 +0100
TaskoRun Log tail: Handle the case where more bytes are requested
than the current size of the log.
commit 6f445ce3e1f85fecbe4010f3cfce1a5724dab286
Author: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Wed Jan 30 18:30:09 2013 +0100
Do not silence catched exceptions. Debugging can be hard.
commit ce3c088b4890e40273aa846bad0501bd62367080
Author: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Wed Jan 30 18:29:25 2013 +0100
FileNotFoundException inherits IOException so no need for a separate
catch block if the catch code is the same (none).
From ce3c088b4890e40273aa846bad0501bd62367080 Mon Sep 17 00:00:00 2001
From: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Wed, 30 Jan 2013 18:29:25 +0100
Subject: [PATCH 1/6] FileNotFoundException inherits IOException so no need
for a separate catch block if the catch code is the
same (none).
---
java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java |3 ---
1 file changed, 3 deletions(-)
diff --git a/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java b/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java
index c58a296..b521513 100644
--- a/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java
+++ b/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java
@@ -188,9 +188,6 @@ public class TaskoRun {
file.close();
return tail;
}
-catch (FileNotFoundException e) {
-// return ;
-}
catch (IOException e) {
// return ;
}
--
1.7.10.4
From 6f445ce3e1f85fecbe4010f3cfce1a5724dab286 Mon Sep 17 00:00:00 2001
From: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Wed, 30 Jan 2013 18:30:09 +0100
Subject: [PATCH 2/6] Do not silence catched exceptions. Debugging can be
hard.
---
java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java |1 +
1 file changed, 1 insertion(+)
diff --git a/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java b/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java
index b521513..3dfd595 100644
--- a/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java
+++ b/java/code/src/com/redhat/rhn/taskomatic/TaskoRun.java
@@ -189,6 +189,7 @@ public class TaskoRun {
return tail;
}
catch (IOException e) {
+log.error(Can't tail + fileName + : + e.toString());
// return ;
}
}
--
1.7.10.4
From 630870f813d2deaf2b95d8a133fdfd78e110c05a Mon Sep 17 00:00:00 2001
From: Duncan Mac-Vicar P dmacvi...@suse.de
Date: Wed, 30 Jan 2013 18:31:01 +0100
Subject: [PATCH 3/6] TaskoRun Log tail: Handle the case where more bytes are
requested than the current size of the log.
---
