I've cc'ed dev@ on this since it's more of an appropriate list.
What seems the easiest way to me would be to convert the IDs to positive and add version and changeset information. You might be able to do this within the tool you use to convert FileGDB to OSM, or post-process. I have intended to write a tool that will take a .osm file and output the same .osm file, just with all IDs multiplied by -1. You could then load the data with osmosis and update the counter for the node ID if you need to do any subsequent changes. Because it's an initial load there's no existing data to conflict with so osmosis is likely the fastest way. The bad news: I've converted FileGDB data to .osm XML and found it takes up about 10 times the room on disk, or about 400 GB for your data. I just completed an import into an apidb of the planet which is under half the size of your data and it took nearly a full month on a RAID0 array of four 7200 RPM drives. The completed database was about 1 TB. Unless you have lots of extremely fast drives or slower drives and lots of time, I suggest you consider if there might be a way to do what you intend to do without loading the data into an apidb. Dealing with a dataset the size of the planet is bad enough. Yours is larger and will take even longer. From: Benoit Pronovost [mailto:bpronov...@gmail.com] Sent: Monday, June 04, 2012 8:15 AM To: t...@openstreetmap.org Subject: [OSM-talk] Initial population of data in OSM I've installed a local OpenStreetMap architecture for a project I'm currently working on. We need to load a very large amount of data into our OpenStreetMap database. The original data is stored in a FileGDB that is around 40Gb. It has been transfered into the OSM structure (with negative ids and no version, changeset,etc tags). My question is what would be the ideal way to procede to load this initial set of data? I'v tried bulk_upload.py which takes about 1 hour to process 1/30000 of the data. Either my local API 0.6 needs to be tuned or it is not made to handle such important data updates. Osmosis requires that their be changeset and version tags which do not exist in the data I currently have. Writing directly to the database without passing through the API would possibly speed up the process, but I've not found a way to do this. Thank you in advance for your advice.
_______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
