Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning
For the past three weeks, I've been having my Information Store crash. The crash happens at roughly the same time every Sunday morning. Event Type:Error Event Source:MSExchange Common Event Category:General Event ID: 4999 Date: 12/20/2009 Time: 3:44:30 AM User: N/A Computer: EX-CAS Description: Watson report about to be sent to dw20.exe for process id: 9192, with parameters: E12, c-RTL-AMD64, 08.01.0393.001, MSExSearch, unknown, unknown, System.OutOfMemoryException, 0, unknown. ErrorReportingEnabled: False It appears to be the Search service apparently causing the issue. The only info I can find on the web that looks similar states that it's likely an iFilter causing the problem, however I haven't installed any extra iFilters or changed anything since for at least a month before this started happening... If it's a iFilter problem, I suppose it could be a bad item that's causing the crash, but how would I go about hunting down what item? But then why would it happen every Sunday at about the same time, within an hour? Does the Search service try to re-index non-indexed items on a regular basis? During that period, I'm not aware of any extra processes running No backups, no database maintenance... Anyone have any other thoughts what might be causing the Search to crash the IS, or can at least have a suggestion how an can go about troubleshooting further? Joe Pochedley Network Telecommunications Manager Fives North American Combustion, Inc. email: joe.poched...@fivesgroup.com v: +1 216-206-5505 f: +1 216-641-7852
RE: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning
Oops, someone called just as I was getting ready to send my message... Exchange version: 2007, SP1, RU9 From: Joe Pochedley [mailto:joe.poched...@fivesgroup.com] Sent: Monday, December 21, 2009 10:12 AM To: MS-Exchange Admin Issues Subject: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning For the past three weeks, I've been having my Information Store crash. The crash happens at roughly the same time every Sunday morning. Event Type:Error Event Source:MSExchange Common Event Category:General Event ID: 4999 Date: 12/20/2009 Time: 3:44:30 AM User: N/A Computer: EX-CAS Description: Watson report about to be sent to dw20.exe for process id: 9192, with parameters: E12, c-RTL-AMD64, 08.01.0393.001, MSExSearch, unknown, unknown, System.OutOfMemoryException, 0, unknown. ErrorReportingEnabled: False It appears to be the Search service apparently causing the issue. The only info I can find on the web that looks similar states that it's likely an iFilter causing the problem, however I haven't installed any extra iFilters or changed anything since for at least a month before this started happening... If it's a iFilter problem, I suppose it could be a bad item that's causing the crash, but how would I go about hunting down what item? But then why would it happen every Sunday at about the same time, within an hour? Does the Search service try to re-index non-indexed items on a regular basis? During that period, I'm not aware of any extra processes running No backups, no database maintenance... Anyone have any other thoughts what might be causing the Search to crash the IS, or can at least have a suggestion how an can go about troubleshooting further? Joe Pochedley Network Telecommunications Manager Fives North American Combustion, Inc. email: joe.poched...@fivesgroup.com v: +1 216-206-5505 f: +1 216-641-7852
RE: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning
Have you tried resetting the search index files? I don't believe anything will trigger a full index crawl, this could be something to do with online maintenance given the time it's happening. From: bounce-8769238-8066...@lyris.sunbelt-software.com [mailto:bounce-8769238-8066...@lyris.sunbelt-software.com] On Behalf Of Joe Pochedley Sent: 21 December 2009 15:12 To: MS-Exchange Admin Issues Subject: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning For the past three weeks, I've been having my Information Store crash. The crash happens at roughly the same time every Sunday morning. Event Type:Error Event Source:MSExchange Common Event Category:General Event ID: 4999 Date: 12/20/2009 Time: 3:44:30 AM User: N/A Computer: EX-CAS Description: Watson report about to be sent to dw20.exe for process id: 9192, with parameters: E12, c-RTL-AMD64, 08.01.0393.001, MSExSearch, unknown, unknown, System.OutOfMemoryException, 0, unknown. ErrorReportingEnabled: False It appears to be the Search service apparently causing the issue. The only info I can find on the web that looks similar states that it's likely an iFilter causing the problem, however I haven't installed any extra iFilters or changed anything since for at least a month before this started happening... If it's a iFilter problem, I suppose it could be a bad item that's causing the crash, but how would I go about hunting down what item? But then why would it happen every Sunday at about the same time, within an hour? Does the Search service try to re-index non-indexed items on a regular basis? During that period, I'm not aware of any extra processes running No backups, no database maintenance... Anyone have any other thoughts what might be causing the Search to crash the IS, or can at least have a suggestion how an can go about troubleshooting further? Joe Pochedley Network Telecommunications Manager Fives North American Combustion, Inc. email: joe.poched...@fivesgroup.com v: +1 216-206-5505 f: +1 216-641-7852
RE: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning
Do you mean by deleting the Index files off the drive (which would then force a complete re-index)? I've thought about that, but it would be nice to better figure out which DB's I need to re-index instead of having to do them all. Is there a logging level that can be increased to tell me better which DB the service was working in when it crashed? I've got over 1.5 TB worth of DB's, and I'd rather not set my server re-indexing everything. It would be especially annoying if upon deleting the indexes, the service still crashed and didn't get me any further along figuring out what's causing the crash or why. Joe P From: Sobey, Richard A [mailto:r.so...@imperial.ac.uk] Sent: Monday, December 21, 2009 10:36 AM To: MS-Exchange Admin Issues Subject: RE: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning Have you tried resetting the search index files? I don't believe anything will trigger a full index crawl, this could be something to do with online maintenance given the time it's happening. From: bounce-8769238-8066...@lyris.sunbelt-software.com [mailto:bounce-8769238-8066...@lyris.sunbelt-software.com] On Behalf Of Joe Pochedley Sent: 21 December 2009 15:12 To: MS-Exchange Admin Issues Subject: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning For the past three weeks, I've been having my Information Store crash. The crash happens at roughly the same time every Sunday morning. Event Type:Error Event Source:MSExchange Common Event Category:General Event ID: 4999 Date: 12/20/2009 Time: 3:44:30 AM User: N/A Computer: EX-CAS Description: Watson report about to be sent to dw20.exe for process id: 9192, with parameters: E12, c-RTL-AMD64, 08.01.0393.001, MSExSearch, unknown, unknown, System.OutOfMemoryException, 0, unknown. ErrorReportingEnabled: False It appears to be the Search service apparently causing the issue. The only info I can find on the web that looks similar states that it's likely an iFilter causing the problem, however I haven't installed any extra iFilters or changed anything since for at least a month before this started happening... If it's a iFilter problem, I suppose it could be a bad item that's causing the crash, but how would I go about hunting down what item? But then why would it happen every Sunday at about the same time, within an hour? Does the Search service try to re-index non-indexed items on a regular basis? During that period, I'm not aware of any extra processes running No backups, no database maintenance... Anyone have any other thoughts what might be causing the Search to crash the IS, or can at least have a suggestion how an can go about troubleshooting further? Joe Pochedley Network Telecommunications Manager Fives North American Combustion, Inc. email: joe.poched...@fivesgroup.com v: +1 216-206-5505 f: +1 216-641-7852
RE: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning
I wouldn't worry about re-indexing, I've got 8TB of databases and it's almost second nature to reset indexing for a few databases every couple weeks (a persistent issue that we've got open with PSS, I won't bore you with the details). So long as your Exchange server isn't trying to be a web, file and AD server all at once I can't imagine you'll run into performance issues, but ymmv. But yes, that's what I mean. In fact, resetting indexes one by one (with an appropriate gap to ensure a full crawl takes place) might help you identify which db is causing the problem. Similarly, this might take a while if it's really only on a Sunday morning, but you could disable indexing on a database-by-database case and wait for it to not crash :) You can try upping the event logs - it should be fairly clear which ones are indexing related. From: bounce-8769548-8066...@lyris.sunbelt-software.com [mailto:bounce-8769548-8066...@lyris.sunbelt-software.com] On Behalf Of Joe Pochedley Sent: 21 December 2009 20:43 To: MS-Exchange Admin Issues Subject: RE: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning Do you mean by deleting the Index files off the drive (which would then force a complete re-index)? I've thought about that, but it would be nice to better figure out which DB's I need to re-index instead of having to do them all. Is there a logging level that can be increased to tell me better which DB the service was working in when it crashed? I've got over 1.5 TB worth of DB's, and I'd rather not set my server re-indexing everything. It would be especially annoying if upon deleting the indexes, the service still crashed and didn't get me any further along figuring out what's causing the crash or why. Joe P From: Sobey, Richard A [mailto:r.so...@imperial.ac.uk] Sent: Monday, December 21, 2009 10:36 AM To: MS-Exchange Admin Issues Subject: RE: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning Have you tried resetting the search index files? I don't believe anything will trigger a full index crawl, this could be something to do with online maintenance given the time it's happening. From: bounce-8769238-8066...@lyris.sunbelt-software.com [mailto:bounce-8769238-8066...@lyris.sunbelt-software.com] On Behalf Of Joe Pochedley Sent: 21 December 2009 15:12 To: MS-Exchange Admin Issues Subject: Event 4999, MESxSearch - OutOfMemoryException, Every Sunday Morning For the past three weeks, I've been having my Information Store crash. The crash happens at roughly the same time every Sunday morning. Event Type:Error Event Source:MSExchange Common Event Category:General Event ID: 4999 Date: 12/20/2009 Time: 3:44:30 AM User: N/A Computer: EX-CAS Description: Watson report about to be sent to dw20.exe for process id: 9192, with parameters: E12, c-RTL-AMD64, 08.01.0393.001, MSExSearch, unknown, unknown, System.OutOfMemoryException, 0, unknown. ErrorReportingEnabled: False It appears to be the Search service apparently causing the issue. The only info I can find on the web that looks similar states that it's likely an iFilter causing the problem, however I haven't installed any extra iFilters or changed anything since for at least a month before this started happening... If it's a iFilter problem, I suppose it could be a bad item that's causing the crash, but how would I go about hunting down what item? But then why would it happen every Sunday at about the same time, within an hour? Does the Search service try to re-index non-indexed items on a regular basis? During that period, I'm not aware of any extra processes running No backups, no database maintenance... Anyone have any other thoughts what might be causing the Search to crash the IS, or can at least have a suggestion how an can go about troubleshooting further? Joe Pochedley Network Telecommunications Manager Fives North American Combustion, Inc. email: joe.poched...@fivesgroup.com v: +1 216-206-5505 f: +1 216-641-7852
