Dear Andrea,
you write:
a few tenth of sky maps, each 600 MB = some TB.
a few thousand sources, lat/long for each = let's say 10k = 1 MB.
Sounds to me, like you should turn your problem around:
- Distribute your sources.
- work on chunks of your sky maps files.
- search for sources fully covered
and longitude
Thanks,
I posted this question on StackOverflow:
http://stackoverflow.com/questions/17617654/running-hadoop-for-processing-sources-in-full-sky-maps
Regards,
Andrea Zonca
data to hadoop? for each
source I have a reference to the full sky map, latitude and longitude
Thanks,
I posted this question on StackOverflow:
http://stackoverflow.com/questions/17617654/running-hadoop-for-processing-sources-in-full-sky-maps
Regards,
Andrea Zonca
