Hello guys,
Last week we noticed that the IO on our database server was extremely high. It didn't take long to find out that it was the OperationsManagerDW database that causing the issue. Upon further investigation it seemed the Alert.Alertstage table was causing this since it was processing some very expensive queries.
After emptying it (around 180million rows) the IO was back to normal. Most of the records were due to a self-made monitoring script that checks for services so we've disabled it however the events just keep coming back even if we keep deleting them. it almost seems it's processing a huge backlog from somewhere (all from the same day 3/11/2013) or that it's stuck in looping the same event over and over.
Where do I start looking? is there some other table where the backlog is stored so I can delete the source or is there something else wrong? Or do I keep deleting the redundant events until they've caught up? But that could take a while since every 50 rows the Time Raised increases only half a second :-) ...
Thanks!