Re: [U2] Speeding sort
I was thinking of a similar solution, but the op said that they have lots of
disks space, so I am thinking why not just add the field from the other file
into the your reporting file.
David A. Green
(480) 813-1725
DAG Consulting
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of dennis bartlett
Sent: Thursday, October 18, 2012 8:10 PM
To: U2 Users List
Subject: Re: [U2] Speeding sort
however the absolute fastest would be to add an itype to FILE-1 that
accesses FILE-2 SORT_VALUE [you're already prepared to do multiple read on
2 files ]
OTH_FILE_SORT_KEY
001 I
002 TRANS('OTHER_FILE', SOME_KEY, '', 'X') where SOME_KEY is the
value you were saving from your example below and using as key to the second
file
FANCY_ITYPE
001 I
002 OTH_FILE_SORT_KEY : '*' : SOME_KEY
etc
{you could combine these two - separated for clarity}
The index on FILE-1 remains THIS_FILE_FIELD
then
Step 1
-
SELECT FILE WITH *THIS_FILE_FIELD*='VALUE' SAVING UNIQUE FANCY_ITYPE
NO.NULLS
-- gives you a list that is complete (ie doesn't need to access the other
file again)
-- the combo key doesn't detract from the unique because the other-file
accesses are done using SOME_KEY
-- the SAVING is doing the ITYPE work, using a reduced list, thereby saving
more processing
Step 2
-
Your final step is to sort a list -- altho this represents another step, it
does not involve reprocessing a file, and therefore should be very quick.
A sort on the saved list will provide the result you were looking for...
and the field you ultimately want saved { SOME_KEY } is extractable from
FANCY_ITYPE.
Bring on the flames...
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Speeding sort
How are they set for memory? Why not setup a memory drive (or tempfs in linux)
at least it should
Speed up your disk access quite a bit.
George
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of David A. Green
Sent: Friday, October 19, 2012 10:08 AM
To: 'U2 Users List'
Subject: Re: [U2] Speeding sort
I was thinking of a similar solution, but the op said that they have lots of
disks space, so I am thinking why not just add the field from the other file
into the your reporting file.
David A. Green
(480) 813-1725
DAG Consulting
-Original Message-
From: u2-users-boun...@listserver.u2ug.org
[mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of dennis bartlett
Sent: Thursday, October 18, 2012 8:10 PM
To: U2 Users List
Subject: Re: [U2] Speeding sort
however the absolute fastest would be to add an itype to FILE-1 that
accesses FILE-2 SORT_VALUE [you're already prepared to do multiple read on
2 files ]
OTH_FILE_SORT_KEY
001 I
002 TRANS('OTHER_FILE', SOME_KEY, '', 'X') where SOME_KEY is the
value you were saving from your example below and using as key to the second
file
FANCY_ITYPE
001 I
002 OTH_FILE_SORT_KEY : '*' : SOME_KEY
etc
{you could combine these two - separated for clarity}
The index on FILE-1 remains THIS_FILE_FIELD
then
Step 1
-
SELECT FILE WITH *THIS_FILE_FIELD*='VALUE' SAVING UNIQUE FANCY_ITYPE
NO.NULLS
-- gives you a list that is complete (ie doesn't need to access the other
file again)
-- the combo key doesn't detract from the unique because the other-file
accesses are done using SOME_KEY
-- the SAVING is doing the ITYPE work, using a reduced list, thereby saving
more processing
Step 2
-
Your final step is to sort a list -- altho this represents another step, it
does not involve reprocessing a file, and therefore should be very quick.
A sort on the saved list will provide the result you were looking for...
and the field you ultimately want saved { SOME_KEY } is extractable from
FANCY_ITYPE.
Bring on the flames...
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] Speeding sort
Stupid question of the day on unidata 7.2.x I build indexes all over to speed queries - I have plenty of disk to burn. However does having an index help when sorting on a field that's not part of the selection criteria? For example we often do this dance between two files: SELECT FILE WITH FIELD='VALUE' SAVING UNIQUE SOME_KEY NO.NULLS SELECT ANOTHER_FILE BY SORT_FIELD I know indexing FIELD helps the first question, but does having an index on SORT_FIELD help the second? -- Jeffrey Butera, PhD Associate Director for Applications and Web Services Information Technology Hampshire College 413-559-5556 http://www.hampshire.edu http://www.facebook.com/hampshirecollegeit ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Speeding sort
I can't say definitively, but I would expect the answer to be 'no'. On Thu, Oct 18, 2012 at 6:39 AM, Jeffrey Butera jbut...@hampshire.eduwrote: Stupid question of the day on unidata 7.2.x I build indexes all over to speed queries - I have plenty of disk to burn. However does having an index help when sorting on a field that's not part of the selection criteria? For example we often do this dance between two files: SELECT FILE WITH FIELD='VALUE' SAVING UNIQUE SOME_KEY NO.NULLS SELECT ANOTHER_FILE BY SORT_FIELD I know indexing FIELD helps the first question, but does having an index on SORT_FIELD help the second? -- Jeffrey Butera, PhD Associate Director for Applications and Web Services Information Technology Hampshire College 413-559-5556 http://www.hampshire.edu http://www.facebook.com/**hampshirecollegeithttp://www.facebook.com/hampshirecollegeit __**_ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/**mailman/listinfo/u2-usershttp://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Speeding sort
No -Original Message- From: Jeffrey Butera jbut...@hampshire.edu To: U2 Users List u2-users@listserver.u2ug.org Sent: Thu, Oct 18, 2012 5:39 am Subject: [U2] Speeding sort Stupid question of the day on unidata 7.2.x I build indexes all over to speed queries - I have plenty of disk to burn. However does having an index help when sorting on a field that's not part of the selection criteria? For example we often do this dance between two files: SELECT FILE WITH FIELD='VALUE' SAVING UNIQUE SOME_KEY NO.NULLS SELECT ANOTHER_FILE BY SORT_FIELD I know indexing FIELD helps the first question, but does having an index on SORT_FIELD help the second? -- Jeffrey Butera, PhD Associate Director for Applications and Web Services Information Technology Hampshire College 413-559-5556 http://www.hampshire.edu http://www.facebook.com/hampshirecollegeit ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Speeding sort
however the absolute fastest would be to add an itype to FILE-1 that
accesses FILE-2 SORT_VALUE [you're already prepared to do multiple read on
2 files ]
OTH_FILE_SORT_KEY
001 I
002 TRANS('OTHER_FILE', SOME_KEY, '', 'X') where SOME_KEY is the
value you were saving from your example below and using as key to the
second file
FANCY_ITYPE
001 I
002 OTH_FILE_SORT_KEY : '*' : SOME_KEY
etc
{you could combine these two - separated for clarity}
The index on FILE-1 remains THIS_FILE_FIELD
then
Step 1
-
SELECT FILE WITH *THIS_FILE_FIELD*='VALUE' SAVING
UNIQUE FANCY_ITYPE NO.NULLS
-- gives you a list that is complete (ie doesn't need to access the other
file again)
-- the combo key doesn't detract from the unique because the other-file
accesses are done using SOME_KEY
-- the SAVING is doing the ITYPE work, using a reduced list, thereby saving
more processing
Step 2
-
Your final step is to sort a list -- altho this represents another step, it
does not involve reprocessing a file, and therefore should be very quick.
A sort on the saved list will provide the result you were looking for...
and the field you ultimately want saved { SOME_KEY } is extractable from
FANCY_ITYPE.
Bring on the flames...
___
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
