OVM_STATISTIC Table Growing abnormally on OVM Manager 3.4. OVM Manager Stopped Working.
Symptoms:
OVM Manager login is very slow.
OVM manager login failed and repeatedly asking login credentials.
OVM Manager "Server and VM Statistics" menu not responding.
AdminServer.log showing event,
####<2017-12-07T08:16:39.005-0500> <Warning> <com.oracle.odof.util.PersistUtils> <ovmmgr.<domain.name> <AdminServer> <[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <a5fed4f7-6dfb-4b3c-befd-ed62a6f7823c-0000055a> <1478524599005><BEA-000000> <Error executing query: select c from CertificateEntity c where c.fingerprint = :fingerprint
javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
Error Code: 0
Call: SELECT ID, CERTIFICATE, FINGERPRINT, VERSION, USER FROM OVM_USER_CERTIFICATE WHERE (FINGERPRINT = ?)
bind => [1 parameter bound]
Query: ReadAllQuery(referenceClass=CertificateEntity sql="SELECT ID, CERTIFICATE, FINGERPRINT, VERSION, USER FROM OVM_USER_CERTIFICATE WHERE
(FINGERPRINT = ?)")
at org.eclipse.persistence.internal.jpa.QueryImpl.getDetailedException(QueryImpl.java:378)
at org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery(QueryImpl.java:260)
at org.eclipse.persistence.internal.jpa.QueryImpl.getResultList(QueryImpl.java:469)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2280)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2196)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2174)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1575)
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:255)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)
Caused By: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
Error Code: 0
Call: SELECT ID, CERTIFICATE, FINGERPRINT, VERSION, USER FROM OVM_USER_CERTIFICATE WHERE (FINGERPRINT = ?)
bind => [1 parameter bound]
Query: ReadAllQuery(referenceClass=CertificateEntity sql="SELECT ID, CERTIFICATE, FINGERPRINT, VERSION, USER FROM OVM_USER_CERTIFICATE WHERE (FINGERPRINT = ?)")
Reason:
Disk is full on OVM manager Server, and OVM_STATISTIC.ibd of OVMM database is growing too large
# du -hs /u01/app/oracle/mysql/data/ovs/OVM_STATISTIC*
16K /u01/app/oracle/mysql/data/ovs/OVM_STATISTIC.frm
62G /u01/app/oracle/mysql/data/ovs/OVM_STATISTIC.ibd <--- this file
# mysql ovs -u ovs -p -S /u01/app/oracle/mysql/data/mysqld.sock
mysql> select count(*) from OVM_STATISTIC;
+----------+
| count(*) |
+----------+
| 97127984 |
+----------+
Solution:
Truncate the huge OVM_STATISTIC table as workaround.
# mysql ovs -u ovs -p -S /u01/app/oracle/mysql/data/mysqld.sock
mysql> truncate table OVM_STATISTIC;
there is no reported/known issue or side effect for truncating the table.
But need to analyze the root cause of abnormal DB growth if this issue is repeating.
Hello, Has any known issue or side effect been noticed after publication?
ReplyDelete