Re: [FOSSology] Scheduling failed for Agent agent_unpack
-Original Message-
1) Was the upload successful? Please check to see you have
objects in the Software Repository folder using the top
level Browse menu item.
Nothing is listed under the Software Repository. However,
Admin - Dashboard reports 68.16 GB of the repository disk
are used. This disk was freshly formatted specifically to
hold the repository and was empty when I started this, so
something is there. An examination of the file system shows
that the space is being used by a 68GB file called
d5961574ffc7efb1f89dc21d3ad680130520b386.a5f60815a90e75be98b41
88abf42812f.72908984320.
I'm not sure why the web interface says that the repository is empty.
du --si /srv
73G /srv/fossology/repository/localhost/gold/d5/96/15
73G /srv/fossology/repository/localhost/gold/d5/96
73G /srv/fossology/repository/localhost/gold/d5
73G /srv/fossology/repository/localhost/gold
4.1k /srv/fossology/repository/localhost/files/d5/96/15
8.2k /srv/fossology/repository/localhost/files/d5/96
13k /srv/fossology/repository/localhost/files/d5
17k /srv/fossology/repository/localhost/files
73G /srv/fossology/repository/localhost
73G /srv/fossology/repository
73G /srv/fossology
17k /srv/lost+found
73G /srv
The listing above indicates the upload was successfully stored in the gold
subdirectory. However, the absence of content in the files subdirectory
confirms that unpack did not run successfully. (Otherwise you would see a lot
of stuff in there.)
2. Before trying to upload anything, please do the following:
- sudo /etc/init.d/fossology stop
- sudo /usr/lib/fossology/fossology-scheduler -t -L stdout
(this runs the fossology scheduler in test mode with
the output to
stdout. It should end with no agent failures.if not there is
something wrong with the install no uploads will work
unless this test passes).
It appears to have run successfully.
Yup, your output looks normal.
snip
2010-03-02 18:06:37 scheduler[24947] : STATUS: All scheduler
agents are operational.
2010-03-02 18:06:37 scheduler[24947] : *** 0 agent failures.
Scheduler exiting.
3. The material you tried to upload... it wasn't a source
repository was it?
(cvs, svn, git, etc)... we don't support that type of upload at
this point.
I am trying to feed it the gNewSense repository for their
upcoming version 3.0.
/usr/bin/rsync -av -P --delete --delete-before
us.archive.gnewsense.org::metad /middle
That should be OK.
4. mary's suggestion of uploading a single file is a good one, does
that work?
I grabbed a copy of the GPL from gnu.org and fed it to the
one-shot license thing and it did report it as being the GPL.
One-shot is not quite the same as uploading a file.
But, the next part is problematic...
2) The scheduler appears to be having trouble queuing up
jobs (Failed to insert job record). Please check the log
file (/var/log/fossology/fossology.log) for additional
FOSSology errors AND the PostgreSQL log file
(/var/log/postgresql/podtgresql-ver-main.log) for db errors.
The Apache logs do not show anything interested, other than
my browsing of the web interface. Checking
/var/log/postgresql/postgresql-8.3-main.log did prove to be
interesting. I'm wondering if the syntax errors it's
complaining about are at fault? Here's everything from last
night's log...
snip
This is the root of your problem. There should NOT be a syntax error in the
INSERT statement below. Bob I are looking into it now and will figure a
workaround for you.
TTYL,
Mary
2010-03-01 22:39:15 PST ERROR: invalid input syntax for integer:
2010-03-01 22:39:15 PST STATEMENT: INSERT INTO job
(job_user_fk,job_queued,job_priority,job_email_notify,job_name
,job_upload_fk)
VALUES
('',now(),'0','fo...@localhost','unpack','3');
2010-03-01 22:39:15 PST ERROR: invalid input syntax for integer:
2010-03-01 22:39:15 PST STATEMENT: INSERT INTO job
(job_user_fk,job_queued,job_priority,job_email_notify,job_name
,job_upload_fk)
VALUES
('',now(),'0','fo...@localhost','unpack','3');
2010-03-01 22:39:15 PST ERROR: syntax error at or near
order at character 58
2010-03-01 22:39:15 PST STATEMENT: SELECT jq_pk, jq_job_fk
FROM jobqueue WHERE jq_job_fk = order by jq_pk desc limit 1;
2010-03-01 22:39:15 PST ERROR: invalid input syntax for integer:
2010-03-01 22:39:15 PST STATEMENT: INSERT INTO job
(job_user_fk,job_queued,job_priority,job_email_notify,job_name
,job_upload_fk)
VALUES
('',now(),'-1','fo...@localhost','fo_notify','3');
___
fossology mailing list
fossology@fossology.org
http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Scheduling failed for Agent agent_unpack
Jason, Please do not remove your upload. What you did should have worked. There is nothing wrong with your data. I'm going to look into this today. Bob Gobeille ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Fosology now has a Plume entry
On Mar 3, 2010, at 8:05 AM, Bruno Cornec wrote: The Plume project (http://www.projet-plume.org/en) which is promoting FLOSS for Education and Research now includes an entry for FOSSology. Cf: http://www.projet-plume.org/fr/fiche/fossology Thanks Bruno. It will be nice when our interface is not only in english. Bob ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Scheduling failed for Agent agent_unpack
Jason, Upload from Server looks like it has problems to me. With a quick code scan and test, I can see multiple failures. :-( So this is going to take more time than I can give it right now. Maybe someone else has time to fix this. There is a workaround if your upload was recorded in the db: select from the main menu Jobs Agents and tell it what agent you want to run. If your upload shows in the menu but the agent does not, then you will have to remove the license analysis (even though it did not run, there may be a record that it did - another bug): Organize Uploads Remove License Analysis Then go back to Jobs Agents and schedule the agents you want. But there is another problem with what you want to do. Analyzing 67 GB is going to take forever on a single machine. So choose your agents carefully. If you only want the license analysis, then only select that agent, none of the others are needed. V 1.2 that we are working on now and hope to have a beta by the end of March, has a much faster license scanner (5-20x). Since you have access to us.archive.gnusense.org, you might consider taring up a piece at a time of http://us.archive.gnewsense.org/gnewsense-metad/ and doing an upload from URL or upload from file. This way you could do the most important pieces first. I hope that helps. Bob Gobeille ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Scheduling failed for Agent agent_unpack
I would remove it: Organize =uploads = delete Uploaded file Organize - Uploads - Delete Uploaded File is empty When you login to the fossoslogy web site are you using the user fossy? I am logged in as the user fossy but have since changed fossy's name password to something else through Admin - Users - Account Settings. Please check that the user fossy has the correct permissions: Admin = Users = Edit Users sigh, I just discovered you will need to use the fossy account to create another full admin account, then login as other account and check fossy. this is silly will fix this in the next release. I agree that's stilly and yes, my access is set to Full Administrator. select from the main menu Jobs Agents and tell it what agent you want to run. Jobs - Agents doesn't list any upload for me to analyze. If your upload shows in the menu but the agent does not, then you will have to remove the license analysis (even though it did not run, there may be a record that it did - another bug): Organize Uploads Remove License Analysis Then go back to Jobs Agents and schedule the agents you want. Organize - Uploads - Remove License Analysis is also empty. There doesn't appear to be any way to work with this ~68GB file sitting in the repository. Other than reporting the disk usage under Admin - Dashboard there doesn't seem to be anything I can do with that ~68GB file through the web interface. Can you upload the iso instead? That will work. I could get the gNewSense .iso but that only contains the gNewSense installer. I'd like to be able to analyze the rest of the software repository as well. Sorry, but one-shot doesn't touch the db. Please try to upload the file. Put it on your pc and upload with upload from file. It should work. I've since done two things early this morning in response to Mark Donohoe's message. 1. I seleced Upload From URL and gave it this: http://archive.gnewsense.org/gnewsense-metad/installer/3.0~beta1/i386/netboot/mini.iso I also downloaded this file and did Upload From File: http://us.archive.gnewsense.org/gnewsense-metad/gnewsense-security/pool/updates/main/a/acpid/acpid_1.0.8.orig.tar.gz I did both of these ~4.5 hours ago and everything is still listed as queued (wget hasn't ever run to get mini.iso.) I will check the logs when I have access to the machine this evening to see what's going on and report back. But there is another problem with what you want to do. Analyzing 67 GB is going to take forever on a single machine. So choose your agents carefully. Thanks for the feedback. I was expecting it to take a while, although I had no idea /how/ long since this is my first in-depth experience with FOSSology. The machine has 8 CPUs so I am hoping that will help with the work, even if I have to break it up into pieces. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Scheduling failed for Agent agent_unpack
I've since done two things early this morning in response to Mark Donohoe's message. 1. I seleced Upload From URL and gave it this: http://archive.gnewsense.org/gnewsense-metad/installer/3.0~bet a1/i386/netboot/mini.iso I also downloaded this file and did Upload From File: http://us.archive.gnewsense.org/gnewsense-metad/gnewsense-secu rity/pool/updates/main/a/acpid/acpid_1.0.8.orig.tar.gz I did both of these ~4.5 hours ago and everything is still listed as queued (wget hasn't ever run to get mini.iso.) I will check the logs when I have access to the machine this evening to see what's going on and report back. Is the scheduler running? Mary ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
Re: [FOSSology] Scheduling failed for Agent agent_unpack
Is the scheduler running? Just to follow up with everyone, I was stupid forgot to restart the scheduler last night after I killed it to run /usr/lib/fossology/fossology-scheduler -t -L stdout. It is running now, and I'm going to take my ~67GB, break it up, and feed it to FOSSology via the Upload From URL option. Thanks to everyone for their assistance in identifying the root cause for the Upload From Server problem. I'll be looking forward to the next version. ___ fossology mailing list fossology@fossology.org http://fossology.org/mailman/listinfo/fossology
