here's a perl script that will pull out the lines with a 2006 date in
field position 19. (#18 when zero indexed.) The new file created is
called "washer-2006.tsv". ("tsv" stands for "tab separated values".)
If the script can't find a date formatted as expected in column 19, it
prints an error message on standard out... so the error won't wind up
in your new data file. let me know if it's useful. ok? also, if you
wind up with a new file that's still too big, another script can
probably be written to pare down washer-2006.
good luck.
- Anna
perl -we '
$cnt=0;
while($l=<>) {
$cnt++;
@r=split(/\t/,$l);
if($r[18] !~ m/(\d{2})\/(\d{2})\/(\d{4})/) {
print STDERR "error: row# $cnt has unexpected date format\n";
}
else {
if($3==2006){
print $l;
}
}
}' washer > washer-2006.tsv
On Wed, Jan 24, 2007 at 10:08:28AM -0800, Dennis Bagley wrote:
> Hey Ana,
>
> Attached is a snippet of the file that I also sent to James Washer.
> I need to extract all records that have a date within 2006 in Field # 19
> That will give me a file small enough to use my regular tools with
> (along with being a superset of the records I need) which can
> pare down to my exact needs.
>
> Thanks
>
> Dennis
>
> Ana wrote:
>
> On Tue, Jan 23, 2007 at 04:04:36PM -0800, Dennis Bagley wrote:
>
>
> Hey folks,
>
> I have an ascii database containing approximately 190,000 records.
>
> I need to import the data into a dbms, trim out fields I do not need and
> extract about 10,000
> records.
>
> I keep bumping into a 64K record limitation - is there an open source
> solution available
> that won't require 3 weeks of studying to use?
>
>
> one idea: post more information here and let this list (aka, the people
> on the list) generate a script that will do all the work for you. 10 to
> 20 lines of perl would probably do it, depending on the complexity of
> the problem. (1 or 2 lines might do it, but I think homey don't play
> that game no more.)
>
> - Ana
>
>
> _______________________________________________
> RLUG mailing list
> [EMAIL PROTECTED]
> [2]http://lists.rlug.org/mailman/listinfo/rlug
>
>
>
> References
>
> Visible links
> 1. mailto:[email protected]
> 2. http://lists.rlug.org/mailman/listinfo/rlug
> 12 001-020-01 1 1 03943 KINGS ROW 1005 BREAZEALE,
> RONNIE D & SARA J 0 0 0 2030 PARKWAY DR
> RENO NV 89502 02639963 01/14/2002 PELLETT,MARK N
> 02174400 01/28/1998 WINDERMERE 1 LT 1 BLK A
> WINDERMERE 1 3176 01/14/2002 3B 021 PELLETT,MARK N 0
> 01/28/1998 3NTT 021 74000 03/01/1988 2D 021
> 900000 01/01/1986 2MQC 021 0
> 0
> 0 0 0
> Townhse End 0 TWO 1 1 1123 FAIR R020
> 002 STUD FRAMED 2 1 1 7 1 FA
> PLYWOOD 0 BUILT-UP ROC WOOD 0 0
> GARA 253 0 1984 1984 0 0
> 021 MF14 0.001 Ac MUNI WATER MUNI SEWER PAVED
> 2001-2006 2006 Reappraisal 2006 NR 50000 47710
> 97710 17500 16698 0 0 34198 0 2005 37673 44456
> 82129 13186 15560 0 0 28746 0 2004 29899
> 44163 74062 10465 15457 0 0 25922 0 08/19/2005
> GKB \\CAMA\AssessPro\washoe2000\sketch\0\12001.PCX 0
> 1 A 2120
> 03943 2994 184272 420R N
> SRDV
> 19 001-020-02 1 1 03945 KINGS ROW 1005 RAY & HALL
> CHARITABLE FNDTN, D & SIBYL 0 0 0 PO BOX
> 863 WILLISTON FL 32696 03218918 05/24/2005
> AMITY,RONALD J 02582641 08/06/2001 WINDERMERE
> 1 LT 2 BLK A WINDERMERE 1 266000 05/24/2005 2D 021
> AMITY,RONALD J 126000 08/06/2001 2D 021 SCHWEER,DAVID J 0
> 08/06/2001 3NTT 021 SCHWEER,DAVID J 95000 06/19/1992 2D
> 021 87500 03/01/1986 2D 021 900000
> 01/01/1986 2MQC 021 0
> 0 0 Townhse Ins 0 TWO 1
> 1 1444 FAIR R020 003 STUD FRAMED 2 2 1
> 11 1 FA PLYWOOD 0 BUILT-UP ROC
> 0 0 GARA 400 0 1984 1984
> 0 0 021 MF14 0.001 Ac MUNI WATER
> MUNI SEWER PAVED 2001-2006 2006 Reappraisal
> 2006 NR 50000 58220 108220 17500 20377 0 0 37877
> 0 2005 37673 54171 91844 13186 18960 0 0
> 32146 0 2004 29899 53703 83602 10465 18796 0 0
> 29261 0 08/19/2005 GKB
> \\CAMA\AssessPro\washoe2000\sketch\0\19001.PCX 0 2
> A 2120
> 03945 2994 283638 420R N SRDM
> 20 001-020-03 1 1 03947 KINGS ROW 1005 WRIGHT, FAEZEH
> F 0 0 0 3947 KINGS ROW RENO
> NV 89503 02166904 12/31/1997 CHK 05/17/1994
> WINDERMERE 1 LT 3 BLK A WINDERMERE 1 111700
> 12/31/1997 2D 021 104800 05/17/1994 2D 021
> 89900 09/01/1990 2D 021 85000 06/01/1988
> 2D 021 80500 04/01/1986 2D 021 0
> 0
> 0 0 Townhse Ins 0 TWO
> 1 1 1518 FAIR R020 003 STUD FRAMED 2 1
> 1 8 1 FA PLYWOOD 0 BUILT-UP
> ROC 0 0 GARA 400 0 1984
> 1984 0 0 021 MF14 0.001 Ac MUNI
> WATER MUNI SEWER PAVED 2001-2006 2006
> Reappraisal 2006 NR 50000 58500 108500 17500 20475 0
> 0 37975 0 2005 37673 54407 92080 13186 19042 0
> 0 32228 0 2004 29899 53909 83808 10465 18868 0
> 0 29333 0 08/19/2005 GKB
> \\CAMA\AssessPro\washoe2000\sketch\0\20001.PCX 0 3
> A 2120
> 03947 2994 20 420R N POQ
> 21 001-020-04 1 1 03949 KINGS ROW 1005 BUCHANAN,
> ROBERT K & BEVERLY J 0 0 0 3949 KINGS ROW
> RENO NV 89503 02438507 04/13/2000
> 02409534 12/27/1999 WINDERMERE 1 LT 4 BLK A
> WINDERMERE 1 0 04/13/2000 123500
> 12/27/1999 2D 021 92000 08/13/1993 2D 021
> 82500 02/01/1987 2D 021 900000 01/01/1986
> 2MQC 021 0
> 0 0 0 Townhse
> Ins 0 TWO 1 1 1471 FAIR R020 003
> STUD FRAMED 2 2 1 10 1 FA
> PLYWOOD 0 BUILT-UP ROC 0 0
> GARA 400 0 1984 1984 0 0 021
> MF14 0.001 Ac MUNI WATER MUNI SEWER PAVED
> 2001-2006 2006 Reappraisal 2006 NR 50000 58390
> 108390 17500 20436 0 0 37936 0 2005 37673 54315
> 91988 13186 19010 0 0 32196 0 2004 29899
> 53810 83709 10465 18833 0 0 29298 0 08/19/2005
> GKB \\CAMA\AssessPro\washoe2000\sketch\0\21001.PCX 0
> 4 A 2120
> 03949 2994 21 420R N POQ
> 22 001-020-05 1 1 03951 KINGS ROW 1005 BLONDE SHELTERS
> LLC 0 0 0 880 CROCKER RENO
> NV 89509 03069865 07/19/2004 PHELPS,LAMOIN & DEBORAH A
> 01852872 11/30/1994 WINDERMERE 1 LT 5 BLK A
> WINDERMERE 1 190000 07/19/2004 2D 021 PHELPS,LAMOIN &
> DEBORAH A 0 11/30/1994 93035
> 02/01/1993 2D 021 90000 08/01/1990 2D 021
> 80500 05/01/1986 2D 021 900000 01/01/1986
> 2MQC 021 0 0
> 0 Townhse Ins 0 TWO 1 1
> 1518 FAIR R020 003 STUD FRAMED 2 1 1 8
> 1 FA PLYWOOD 0 BUILT-UP ROC
> 0 0 GARA 400 0 1984 1984 0
> 0 021 MF14 0.001 Ac MUNI WATER MUNI
> SEWER PAVED 2001-2006 2006 Reappraisal 2006 NR
> 50000 58500 108500 17500 20475 0 0 37975 0
> 2005 37673 54407 92080 13186 19042 0 0 32228 0
> 2004 29899 53909 83808 10465 18868 0 0 29333 0
> 08/19/2005 GKB
> \\CAMA\AssessPro\washoe2000\sketch\0\22001.PCX 0 5
> A 2120
> 03951 2994 259191 420R N POM
> 23 001-020-06 1 1 03953 KINGS ROW 1005 DUPREE, MARK &
> MERRILY P 0 0 0 3953 KINGS ROW
> RENO NV 89503 02872114 06/18/2003 LYONS,ELLEN R
> 01498097 07/31/1991 WINDERMERE 1 LT 6 BLK A
> WINDERMERE 1 158000 06/18/2003 4BV 021 LYONS,ELLEN R
> 102000 07/31/1991 2D 021 88179 03/01/1990 2D
> 021 87000 04/01/1987 2D 021 82500
> 04/01/1986 2D 021 0
> 04/21/2003 GVY 0 0 0
> Townhse Ins 0 TWO 1 1 1471 FAIR
> R020 003 STUD FRAMED 2 2 1 10 1 FA
> PLYWOOD 0 BUILT-UP ROC 0
> 0 GARA 400 0 1984 1984 0 0
> 021 MF14 0.001 Ac MUNI WATER MUNI SEWER PAVED
> 2001-2006 2006 Reappraisal 2006 NR 50000 58390
> 108390 17500 20436 0 0 37936 0 2005 37673
> 54315 91988 13186 19010 0 0 32196 0 2004 29899
> 53810 83709 10465 18833 0 0 29298 0 08/19/2005
> GKB \\CAMA\AssessPro\washoe2000\sketch\0\23001.PCX 0
> 6 A 2120
> 03953 2994 226903 420R N LPQ
> 24 001-020-07 1 1 03955 KINGS ROW 1005 SAMMARTANO,
> LAVALA TR 0 SAMMARTANO TRST 0 0 3955 KINGS ROW
> RENO NV 89503 03094799 09/08/2004
> SAMMARTANO,LAVALA 01722624 10/18/1993
> WINDERMERE 1 LT 7 BLK A WINDERMERE 1 0 09/08/2004 3BGG 021
> SAMMARTANO,LAVALA 0 10/18/1993
> 98500 08/31/1993 2D 021 97000 09/01/1990 2D
> 021 124656 09/01/1988 3B 021 92500
> 07/01/1987 2D 021 0
> 0 0 Townhse Ins 0 TWO 1
> 1 1623 FAIR R020 003 STUD FRAMED 3 3 0
> 11 1 FA PLYWOOD 0 BUILT-UP ROC
> 0 0 GARA 400 0 1984 1984
> 0 0 021 MF14 0.001 Ac MUNI WATER
> MUNI SEWER PAVED 2001-2006 2006 Reappraisal
> 2006 NR 50000 62440 112440 17500 21854 0 0 39354
> 0 2005 37673 58088 95761 13186 20331 0 0
> 33517 0 2004 29899 57596 87495 10465 20159 0 0
> 30624 0 08/19/2005 GKB
> \\CAMA\AssessPro\washoe2000\sketch\0\24001.PCX 0 7
> A 2120
> 03955 2994 263436 420R N POQ
_______________________________________________
RLUG mailing list
[email protected]
http://lists.rlug.org/mailman/listinfo/rlug
