Rich Shepard wrote: > On Tue, 16 Aug 2011, Rich Shepard wrote: > > >> This will work for all completely duplicated lines. I'll need to see how >> many remain that vary in one or more columns ('fields') such as the >> parameter, lab_id number, or qa_qc. >> > > I had manually cleaned up a bunch of lines so the souce file had 12,119 > lines. AFter running uniq the output file has 8,605 lines, about 1/3 fewer. > > The need to remove almost duplicates/triplicates, based on the same values > in three columns regardless of the rest of the contents, remains. > > Rich > Rich Shepard If this a one off (once and done), one might consider a spreadsheet sorted on the three columns in question ... a little rough around the edges on the down and dirty side, but it should work. If this is going to be ongoing, one might consider (G)AWK ... a glance at the "content" supplied suggest that the values might be delimited by either a tab or space, and that there may not be any spaces within a field? Otherwise one might have to treat the data as fixed length records ... is that possible? Hope this helps Regards Fred James
_______________________________________________ PLUG mailing list PLUG@lists.pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug