Stupid questions but we are sure that it is named logging.conf (the same
name as the logconf variable) and that it is readable?
Dennis
Justin Hartman wrote:
> Hi Dennis
>
> The logging.conf file is in the /hdd2/jobstream/ folder along with the
> python script. I haven't modified the logging.conf file at all -
> should i?
>
> Regards
> Justin
>
> On 1/29/07, Dennis Kubes <[EMAIL PROTECTED]> wrote:
>> Justin,
>>
>> Thanks for the update. I will update the script and the wiki to be able
>> to run this from a clean, no previous fetches run. Currently it did
>> assume that there were at least some previous fetches, crawldb, and
>> segments to go with it.
>>
>> As to your error, I think it is looking for the logging.conf file. Is
>> that file in the same directory as the JobStream.py script? In the top
>> of the logging file there is a section called formatters like this:
>>
>> [formatters]
>> keys=simple
>>
>>
>> Dennis Kubes
>>
>> Justin Hartman wrote:
>> > Hi Dennis
>> >
>> > This is a great contribution and I personally thank you for making it
>> > available to the community.
>> >
>> > I am having a little difficulty getting it to work and possibly you
>> > can provide some assistance in what I'm doing wrong?
>> >
>> > A little background first:-
>> > I'm running the python script in the following location:
>> > /hdd2/jobstream/JobStream.py
>> > My master directory is: /hdd2/nutch/master
>> > My backup directory is: /hdd2/nutch/backup
>> >
>> > My config in JobStream.py is as follows:-
>> >
>> > Line 55 to 60 configured as:
>> > class JobStream:
>> > nutchdir = "/home/nutch/nutch"
>> > masterdir = "/hdd2/nutch/master"
>> > backupdir = "/hdd2/nutch/backup"
>> > log = logging.getLogger("jobstream")
>> >
>> > Line 377 onwards configured as:
>> > def main(argv):
>> > # set the default values
>> > resume = 0
>> > execute = 0
>> > checkfile = "jobstream.stop"
>> > logconf = "logging.conf"
>> > jobdir = "/hdd2/jobstream"
>> > nutchdir = "/home/nutch/nutch"
>> > masterdir = "/hdd2/nutch/master"
>> > backupdir = "/hdd2/nutch/backup"
>> > dfsdumpdir = "/hdd2/nutch/dump"
>> > tempdir = "/hdd2/nutch/temp"
>> > splitsize = 500000
>> > fetchmerge = 3
>> >
>> > All the above paths are correct and have been created and the master
>> > and backup directories contain zero data and have been created for
>> > usage of the python script.
>> >
>> > When executing JobStream.py -e for the first time I got an error
>> > telling me it could not find various directories within the master
>> > directory so I injected the URLs into the /hdd2/nutch/master
>> > directory.
>> >
>> > This solved my initial error however now I have this error (below) and
>> > not sure what to do about it:
>> >
>> > /usr/bin/python2.4 /hdd2/jobstream/JobStream.py -e
>> > Traceback (most recent call last):
>> > File "/hdd2/jobstream/JobStream.py", line 465, in ?
>> > main(sys.argv[1:])
>> > File "/hdd2/jobstream/JobStream.py", line 444, in main
>> > logging.config.fileConfig(logconf)
>> > File "logging/config.py", line 76, in fileConfig
>> > File "/usr/lib/python2.4/ConfigParser.py", line 511, in get
>> > raise NoSectionError(section)
>> > ConfigParser.NoSectionError: No section: 'formatters'
>> >
>> > Do you have any ideas?
>> >
>> > Regards
>> > Justin
>> >
>> > On 1/29/07, Dennis Kubes <[EMAIL PROTECTED]> wrote:
>> >> It is up on the wiki at the following location.
>> >>
>> >> http://wiki.apache.org/nutch/Automating_Fetches_with_Python
>> >>
>> >> It has also been added to the front page.
>> >>
>> >> Dennis Kubes
>> >>
>> >> Andrzej Bialecki wrote:
>> >> > Dennis Kubes wrote:
>> >> >> We have a python script with logging which fully automates the
>> >> >> fetching and updating process, not the invert links or the indexing
>> >> >> process. If anybody wants a copy, send me an email and I will send
>> >> >> you a copy.
>> >> >>
>> >> >> We are currently working on a more in-depth framework for
>> automating
>> >> >> these types of job streams in python but that is not complete yet.
>> >> >>
>> >> >> Andrzej, do you think this is something we should post to the wiki?
>> >> >
>> >> > Sure, if it's ok for you to release it I'm sure many people would
>> find
>> >> > it useful.
>> >> >
>> >>
>> >
>> >
>>
>
>
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
