I think you need to pipe the text through linux utility -> uniq
and to be more neat, you can pipe it again through another linux utility i.e -> sort

Regards, 
---------
Muhammad Faisal Rauf Danka

Chief Technology Officer
Gem Internet Services (Pvt) Ltd.
web: www.gem.net.pk
voice: 92-021-111-GEMNET

Vice President
Pakistan Computer Emergency Responce Team (PakCERT)
web: www.pakcert.org

Chief Security Analyst
Applied Technology Research Center (ATRC)
web: www.atrc.net.pk
voice: 92-21-4980523 92-21-4974781 

"Great is the Art of beginning, but Greater is the Art of ending. "

------BEGIN GEEK CODE BLOCK----
Version: 3.1
GCS/CM/P/TW d- s: !a C++ B@ L$ S$ U+++ 
P+ L+++ E--- W+ N+ o+ K- w-- O- PS PE- Y- 
PGP+ t+ X R tv+ b++ DI+ D G e++ h! r+ y+
------END GEEK CODE BLOCK------


--- [EMAIL PROTECTED] wrote:
>i have a friend who has a access database that needs some work done on it. 
>one thin in particular is that it needs to have all duplicate entries 
>removed. i had him export the database into a comma deliniated file. the 
>list has 50,000+ entries. now what i need to do is find a script or tool 
>that will search the text file for duplicate (or sometimes multiple) 
>entries and remove all but one of those entries. 
>
>i've searched freshmeat all day and google as well. i'm not finding 
>anything. another friend said it could be done in perl or php but i know 
>neither of them.
>
>does anyone know of a tool that could do this? the text file must remain 
>in it's current state so it can be imported back into the database.
>
>thanks for any and all help!
>
>-- 
>\||/   j    a    c    k      w    a    l    l    e    n     
> 00  
>  >        - chasing monsters with a butterfly net
> -     
>                                      http://www.monkeypantz.net/awf
>Key fingerprint = B52F 989C 899F B2AB EFF5  451A E7F3 A10D 7A23 4D7B
>sub  1024g/6ED2DCD4 2001-09-21
>
>
>
>
>
>_______________________________________________
>Redhat-list mailing list
>[EMAIL PROTECTED]
>https://listman.redhat.com/mailman/listinfo/redhat-list

_____________________________________________________________
---------------------------
[ATTITUDEX.COM]
http://www.attitudex.com/
---------------------------

_____________________________________________________________
Promote your group and strengthen ties to your members with [EMAIL PROTECTED] by 
Everyone.net  http://www.everyone.net/?btn=tag



_______________________________________________
Redhat-list mailing list
[EMAIL PROTECTED]
https://listman.redhat.com/mailman/listinfo/redhat-list

Reply via email to