Degrading performance of staging operations on oss.sonatype.org
Incident Report for Sonatype
Resolved
We found that a timeout value had been rolled back to an incorrect value during yesterday's migration process. That has been corrected, and our monitors indicate that staging operations are no longer stacking up, and most seem to be completing in under a minute.
Posted Mar 27, 2020 - 15:26 EDT
Monitoring
We just completed a restart to migrate to scaled up infrastructure. We're monitoring to see if staging operations performance improves.
Posted Mar 27, 2020 - 13:53 EDT
Update
It appears that opening/closing/releasing staging repositories is being handled fine by oss.sonatype.org, but drops are being batched up and taking longer than the default Maven timeout.
Posted Mar 27, 2020 - 12:45 EDT
Identified
We will need to issue a series of restarts for some scaling and cleanup operations. You may see 504 errors replaced by 405s and 502s. If at all possible, you may want to consider pausing your deployments for the next hour.
Posted Mar 27, 2020 - 12:13 EDT
Investigating
The restart seems to have provided only temporary relief. Staging operations look like they're still stacking up, and users are reporting 504 timeouts.
Posted Mar 27, 2020 - 11:49 EDT
This incident affected: Central (OSSRH).