Hi Guys,
This is my issue: We have recently has an issue with our SCOM 2012 SP1 Datawarehouse where the alert stage table was growing out of control (this was resolved). While checking out the heath of the DW I noticed that there were a number of monitors that had a large number of state changes 2k+ and looking into it further it seems that there are a couple of monitors which have a high number of state change events.
So I have run the following query on the OpsMgr DB:
select distinct top 50 count(sce.StateId) as NumStateChanges, m.MonitorName, mt.typename AS TargetClass from StateChangeEvent sce with (nolock) join state s with (nolock) on sce.StateId = s.StateId join monitor m with (nolock) on s.MonitorId = m.MonitorId join managedtype mt with (nolock) on m.TargetManagedEntityType = mt.ManagedTypeId where m.IsUnitMonitor = 1 group by m.MonitorName,mt.typename order by NumStateChanges desc
and get the result:
NumStateChanges MonitorName 14151 OpsMgr.2012.Self.Maintenance.Check.Data.Warehouse.DB.Hourly.Aggregation.Count.3State.Monitor 14151 OpsMgr.2012.Self.Maintenance.Check.Data.Warehouse.DB.Daily.Aggregation.Count.3State.Monitor 4822 _769983AD_9EB2_4526_B5FD_0559A373B300_ 4208 Microsoft.Linux.RHEL.6.LogicalDisk.PercentFreeSpace.Monitor 3217 _ef4a8515_1cc0_48a9_aa12_92ba059a6eae_ 2718 Microsoft.HPUX.11iv2.PhysicalDisk.AverageDiskTransferTime.Monitor 2456 Microsoft.SystemCenter.DataWarehouse.ManagementServer.AlertCollectionWriterState.FirstChanceState 2349 Microsoft.SystemCenter.WebApplicationTest.WebTestProbe.Monitor 2237 Microsoft.SystemCenter.HealthServiceModules.AlertSubscriptionDSModule.AcknowledgementPerformanceMonitor 2225 Microsoft.SystemCenter.DataWarehouse.ManagementServer.PerformanceDataMaintenanceState.DedicatedMaintenance.FirstChanceState 2218 Microsoft.SystemCenter.DataWarehouse.Deployment.ComponentGetProcess.FirstChanceState 2216 Microsoft.SystemCenter.DataWarehouse.Synchronization.Configuration.StateReadState.FirstChanceState 2214 Microsoft.SystemCenter.DataWarehouse.Synchronization.HealthServiceOutage.StateReadState.FirstChanceState 2212 Microsoft.SystemCenter.DataWarehouse.Synchronization.MaintenanceMode.StateReadState.FirstChanceState 2212 Microsoft.SystemCenter.DataWarehouse.Deployment.ReportDeploymentState.FirstChanceState 2212 Microsoft.SystemCenter.DataWarehouse.Synchronization.DomainSnapshot.StateReadState.FirstChanceState 2212 Microsoft.SystemCenter.DataWarehouse.Discovery.DataSet.FirstChanceState 2211 Microsoft.SystemCenter.DataWarehouse.ManagementServer.AlertDataMaintenanceState.DedicatedMaintenance.FirstChanceState 2210 Microsoft.SystemCenter.DataWarehouse.Synchronization.ManagedEntity.StateReadState.FirstChanceState 2210 Microsoft.SystemCenter.DataWarehouse.Synchronization.MonitorInitialState.StateReadState.FirstChanceState 2210 Microsoft.SystemCenter.DataWarehouse.Discovery.StandardDataSet.Alert.FirstChanceState 2210 Microsoft.SystemCenter.DataWarehouse.Synchronization.TypedManagedEntity.StateReadState.FirstChanceState 2210 Microsoft.SystemCenter.DataWarehouse.Synchronization.Relationship.StateReadState.FirstChanceState 2208 Microsoft.SystemCenter.DataWarehouse.Discovery.StandardDataSet.Event.FirstChanceState 2208 Microsoft.SystemCenter.DataWarehouse.Discovery.StandardDataSet.State.FirstChanceState 2207 Microsoft.SystemCenter.DataWarehouse.ManagementServer.StateDataMaintenanceState.DedicatedMaintenance.FirstChanceState 2207 Microsoft.SystemCenter.DataWarehouse.Deployment.DataWarehouseDeployedManagementPackListRequestState.FirstChanceState 2206 Microsoft.SystemCenter.DataWarehouse.Discovery.StandardDataSet.Performance.FirstChanceState 2206 Microsoft.SystemCenter.DataWarehouse.ManagementServer.EventDataMaintenanceState.DedicatedMaintenance.FirstChanceState 2203 Microsoft.SystemCenter.HealthServiceModules.AlertSubscriptionDSModule.QueryPerformanceMonitor 2203 Microsoft.SystemCenter.HealthServiceModules.AlertSubscriptionDSModule.ConfigurationMonitor 2203 Microsoft.SystemCenter.HealthServiceModules.AlertSubscriptionDSModule.AvailabilityMonitor 2195 Microsoft.SystemCenter.DataWarehouse.Synchronization.Configuration.DataWriteState.FirstChanceState 2192 Microsoft.SystemCenter.DataWarehouse.Deployment.DataWarehouseDeployedManagementPackListRequestState.RecoveryState
So as you can see that DW monitors have a large number of state changes and this looks to be happening every 2 minutes. But when I look at the Health explorer for the DW windows server, it does not experience this.
I cent see any errors in the Operations manager log on the DW server.
So looking at the DW Alert stage table, it still continues to grow.
How can I resolve this?
Thanks Martin