Hi Debbie, Importing old data - anything from before a server first begins exporting to the database - is a one-time-only process. It works back through everything until it reaches the 'oldest date to collect' specified in Server Settings, and then stops permanently. So simply deleting existing old data, I'm assuming from the Maintenance Tasks page, won't cause it to re-import. For that, you'll need to start with a completely fresh database.
Here's what I would recommend: 1. Before anything else, make sure you're using a good data retention policy. Initially, everything is set to 'Default', i.e. interfaces use the same policy as the device they belong to, devices use the same policy as the maps they are on, and maps use the server-wide policy from the Reports Server pane in Server Settings. So unless you've overridden the policy for specific maps/devices/interfaces, you can just change the server-wide policy and it will affect everything. By default, the server-wide policy is set to the built-in '24 Hours' policy, which keeps just one day's worth of data. Click the Configure button on the Reports Server pane in Server Settings to bring up the IMDC web interface home page. Click 'Change Settings' for InterMapper Database, then bring up the 'Policies' tab. This is where you create and edit retention policies. To answer your question about how much data to keep, I generally recommend a policy like this: Keep original data for: 1 week 5-Minute samples for: 1 month Hourly samples for: 1 year Daily samples for: forever It's really a question of how much detail you want in your reports, and how far back you want that detail. Raw data takes up most of the space in the database, and it's rarely used for reporting purposes. So unless you have a specific need to see the individual data points, I wouldn't keep that for more than a few days or a week. For the rest, you just need an idea of what you'd like to do with the data. If you're reporting on utilization on a particular link, but you only care about the daily averages, and if you want to report on that going back 5 years, then you'll want to keep your daily samples for at least 5 years. The 5-minute & hourly you could get rid of much sooner. It's a balance; you want to keep just enough data to get useful reports, but not more than that, or else it will affect disk usage and performance. In general, the database is fine with 2500 devices using the retention policy I gave above, though if you have a lot of interfaces you may want to send an email to support so we can take a look and give some further recommendations, since that could be at the upper end of what we currently support. 2. Having created a policy and saved changes, the new policy should show up in IM, including in the server-wide policy drop-down in Server Settings, and you can switch to it. 3. Stop the IMDC service, then delete the existing database folder, which can be found in C:\Program Files\InterMapper\dwf\config on Windows and /usr/local/imdc/config on all other platforms. When you start IMDC again, it will create a fresh database. The next time your IM server connects, since it's a fresh database, it will re-import your old data, applying the new retention policy as it does. This process can take a while; several days if you have years' worth of data store in IM. The easiest way to know when it's done is to click 'Configure' from the Reports Server pane in server settings, and check the status on the IMDC web interface home page. If it says that it's requesting data, or that an old data request is pending, then it's still going. Once it has finished, all you'll see is 'Next import (new data) scheduled for...' interspersed with some activity every 30 seconds as it stays up-to-date. The best way to see what's there is to use Reports. The drop-down is intended more for refining an existing selection, i.e. adding or removing devices to it. An easier way to get an initial selection is to right-click on a device in IM, and click the new 'Reports' item in that menu. That will bring up reports with that specific device selected. You can also do this from the device list, and select more than one device to have them all selected. Then choose the data you'd like to report on, i.e. Response Time, Utilization, etc. from the Dataset drop-down. Now the time-range selection in the lower-left will become available. Bring that up, and it will let you pick a specific date range to report on. The little calendar-picker shows only the dates for which data is available; everything else is greyed out. If the database is still working its way through old data, the available dates will slowly extend back (when you refresh the page) as more data is gathered into the database. I hope that helps, David ----------------------------------------- David Schnur - Dartware, LLC [email protected] -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Debbie Fligor Sent: Friday, April 08, 2011 12:32 PM To: InterMapper Discussion Cc: Debbie Fligor Subject: [IM-Talk] moving old data into IMDC we spun up IMDC and played a bit last summer on a separate server (after our very first test overloaded the main monitoring server trying to export all the old data), but didn't have enough disk on our dev system to hold all our data. We got the new disk, I deleted the old data (or at least I told it to and it said it did) and tried to re-import our few years worth of old info (we have some charts that go back at least 3 years) with the "Gradual" setting. IMDC is 5.4.2, the server that is reporting is on 5.3.8. So far, it's clear that some data is making it there, I have "Created" dates of May 2010 for some devices/ports, and last seen dates of today. but I also have traffic stats for at least April 2010, even though the interface I have stats for was "created" in May 2010. All the Maps seem to be there now, although that took a while. The process (both times) has left me with a few questions that I didn't find in the online docs: How do I tell when it's done getting everything it can get off the server? How do I tell what is in there in a very generic way? The database was so big that the free SQL stuff I downloaded last summer would spin forever on generic "list this table" type queries. my husband's pretty good at SQL (shameless plug: he wrote the O'Reilly "Using SQLite" book that came out last fall) and he could barely get it to tell me anything about what all was in there (ie date of earliest record, list of all devices known, etc.). How much data is "too much?" I'd love to be able to use the saved chart data that's 3-5 years old in trending reports, and then go down to keeping ~ 1.5 years worth of data, but I don't see where the chart data is going (uptime, traffic stats on 5 minute intervals, etc). I've played a little bit with Intermapper Reports, but the drop-down style it uses doesn't seem very practical for a server with 58 maps and 2489 devices (I can't even guess the number of interfaces) Thoughts, ideas, and pointers to Docs I didn't find are all appriciated. .+-j·!÷¬Ûiÿü0Âf¢ªÜ+Þr¿{^®f©¥êíjY8ÒX¬µÖ«·«yÊ&ý:.˱Êâmë צj)m íz³¦«M©d¡÷åË]j»pj·¢
