Content Marketing Platform (CMP) Library: Asset Visibility in CMP DAM
Incident
Report for Optimizely Service
Resolved
This incident has been fully resolved, and the service has been operating normally since the fix was implemented.
The root cause was traced to an anomalous use-case for a particular CMP instance, where our assets database faced an unusual amount of load (500k asset lineage entries for a particular asset) causing delays in data replication in our Elasticsearch instances which resulted in uploaded assets not being visible, as well as slowness/504 errors in our asset public API endpoints.
We have taken corrective actions and are implementing additional preventive measures to reduce the likelihood of similar incidents occurring in the future.
The root cause was traced to an anomalous use-case for a particular CMP instance, where our assets database faced an unusual amount of load (500k asset lineage entries for a particular asset) causing delays in data replication in our Elasticsearch instances which resulted in uploaded assets not being visible, as well as slowness/504 errors in our asset public API endpoints.
We have taken corrective actions and are implementing additional preventive measures to reduce the likelihood of similar incidents occurring in the future.
Monitoring
We have deployed a targeted fix addressing the reported issue, and the impacted functionality should now be operating as expected.
Our teams are actively monitoring system performance to ensure full stability. We will provide additional updates as further insights become available.
Our teams are actively monitoring system performance to ensure full stability. We will provide additional updates as further insights become available.
Update
Our engineering teams remain fully engaged and are continuing a focused investigation into the issue.
Update
We are continuing to investigate this issue.
Investigating
We are currently investigating an issue affecting the visibility of newly uploaded assets within the CMP DAM for a subset of customers.
Our engineering teams are actively working to identify the root cause and restore full functionality as quickly and safely as possible.
We will continue to provide updates on this page as more information becomes available.
Our engineering teams are actively working to identify the root cause and restore full functionality as quickly and safely as possible.
We will continue to provide updates on this page as more information becomes available.
This incident affected: Content Marketing Platform (Library).