Repeated Words
Hello, How do i read a simple file and look for "repeated words" in each line, writing out the words i have found and the numbers of line they were found? eg: File ==> Test.Dat sample line of text. this line follows another line. This is the last line. The program should report: Repeated Word(s): 'line' on Line 2. Any Sample code? TIA, H3li0 __ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/
Re: Repeated Words
Hi Helio, A similar question came up on the list. Please refer to the thread http://archive.develooper.com/beginners%40perl.org/msg00334.html and you should find an (at least good portion of) answer to your question. Cheers, Kevin On Wed, Apr 25, 2001 at 04:22:17AM -0700, Helio S. Junior ([EMAIL PROTECTED]) spew-ed forth: > Hello, > > How do i read a simple file and look for "repeated > words" in each line, writing out the words i have > found and the numbers of line they were found? > > eg: > File ==> Test.Dat > sample line of text. > this line follows another line. > This is the last line. > Writing CGI Applications with Perl - http://perlcgi-book.com -- "Families is where out nation finds hope, where wings take dream." -- G.W. Bush, LaCrosse, WI 10/18/2000
Re: Repeated Words
Helio S. Junior ([EMAIL PROTECTED]) wrote: > Hello, > > How do i read a simple file and look for "repeated > words" in each line, writing out the words i have > found and the numbers of line they were found? Well, that sounds like a homework to me, thus no sample code... Think about using either a hash or an array for storing the previous line's words. When using a hash, you can easily look up if the word in the current line has been there and report it. When you use an array, do a 'perldoc -f grep' to find out how to look for words in there. Do a 'perldoc perlvar' to find out how to get the current input's line number. Mike -- If we fail, we will lose the war. Michael Lamertz | [EMAIL PROTECTED] / [EMAIL PROTECTED] Nordstr. 49 | http://www.lamertz.net 50733 Cologne| Work: +49 221 3091-121 Germany | Priv: +49 221 445420 / +49 171 6900 310
Re: Repeated Words
--- "Helio S. Junior" <[EMAIL PROTECTED]> wrote: > Hello, Hi =o) > How do i read a simple file and look for "repeated > words" in each line, writing out the words i have > found and the numbers of line they were found? > > eg: > File ==> Test.Dat > sample line of text. > this line follows another line. > This is the last line. > > The program should report: > > Repeated Word(s): 'line' on Line 2. I was tempted to use something like $line =~ /(\b\w+\b).*\1/o; to find repeats, but don't -- it only reads the first repeated word, and gets more complex to fix after that. Instead, try doing it manually: open DAT, "Test.dat" or die $!; my $ln = 0; foreach my $line () { chomp $line; $ln++; my %hit = (); foreach my $word (split /\W+/o, $line) { $hit{$word}++ } foreach my $word (keys %hit) { print "Repeated Word(s): '$word' on Line $ln.\n" if $hit{$word} > 1; } } close DAT; =Test.dat sample line of text. this line follows another line. This is the last line. and foo and foo and foo. = You could even use this to tell you how many times the word appeared on the line by adding $jit{$word} to the printed line, etc. This could be condensed, but it works. __ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/
Re: Repeated Words
I believe if you add the (g)lobal modifier and optionally(i), Paul's line of code may work: $line =~ /(\b\w+\b).*\1/ogi; This is also addressed a little differently in Programming Perl, page 149 Steven Spears (905)-405-0955 [EMAIL PROTECTED] Paul , o.com> [EMAIL PROTECTED] cc: 04/25/01 Subject: Re: Repeated Words 10:31 AM Please respond to Hodges --- "Helio S. Junior" <[EMAIL PROTECTED]> wrote: > Hello, Hi =o) > How do i read a simple file and look for "repeated > words" in each line, writing out the words i have > found and the numbers of line they were found? > > eg: > File ==> Test.Dat > sample line of text. > this line follows another line. > This is the last line. > > The program should report: > > Repeated Word(s): 'line' on Line 2. I was tempted to use something like $line =~ /(\b\w+\b).*\1/o; to find repeats, but don't -- it only reads the first repeated word, and gets more complex to fix after that. Instead, try doing it manually: open DAT, "Test.dat" or die $!; my $ln = 0; foreach my $line () { chomp $line; $ln++; my %hit = (); foreach my $word (split /\W+/o, $line) { $hit{$word}++ } foreach my $word (keys %hit) { print "Repeated Word(s): '$word' on Line $ln.\n" if $hit{$word} > 1; } } close DAT; =Test.dat sample line of text. this line follows another line. This is the last line. and foo and foo and foo. = You could even use this to tell you how many times the word appeared on the line by adding $jit{$word} to the printed line, etc. This could be condensed, but it works. __ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/
Re: Repeated Words
--- [EMAIL PROTECTED] wrote: > I believe if you add the (g)lobal modifier and optionally(i), Paul's > line of code may work: > > $line =~ /(\b\w+\b).*\1/ogi; I didn't actually test it -- but wouldn't it still miss interlaced words? like: it might, it might Would it see that "might" was repeated as well? I think the first one would get thrown away by the .* in the pattern __ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/
Re: Repeated Words
At 10:48 AM 4/25/2001, you wrote: > > I believe if you add the (g)lobal modifier and optionally(i), Paul's > > line of code may work: > > > > $line =~ /(\b\w+\b).*\1/ogi; You don't need /o, because there are no variables in the pattern, so Perl will compile the regex at compile time anyway. I guess it doesn't hurt, though. I looked through some stuff for items like this, and found something in the Perl Cookbook that almost fit the bill. I modified it *very* slightly so that it can catch punctuation between words. Take a look at the comments and you'll know what to pull out if you don't want that behavior. And before anyone goes thinking I'm a regex guru, I'm not. I really literally copied 99% of this out of the Perl Cookbook. (BTW, the three guys who wrote it are *really* good people to copy off of. :-) ) I know Perl kinda well, but of all it's features, I'm weakest on re's. Anyway, here it be. while (<>) { while ( m{ \b # Start at a word boundary (\S+) # \S means non-whitespace who's # beginning and end are alphanums # if you put \b's around it. \b # until you hit another boundary \W?# match a non-word char ( \s+# Then some space \1 # Then what we matched in (\S+) above \b # with a boundary after it ) }xig# eXtended (Comments), case Insensitive # Global ) { print "Dupe word '$1' at line $.\n"; } } Thank you for your time, Sean.