Skip to content

Commit

Permalink
Describe the latest hub.fio outage
Browse files Browse the repository at this point in the history
Signed-off-by: Andy Doan <[email protected]>
  • Loading branch information
doanac committed Feb 19, 2020
1 parent 0a23abf commit 97f92a5
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 0 deletions.
13 changes: 13 additions & 0 deletions 2020-02-18-hub-fio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## hub.foundries.io Disruption

### Timeline of Events

* **20:00 UTC** - CI jobs start to fail that are doing `docker push hub.foundries.io...`
* **20:05 UTC** - Our Docker registry(hub.foundries.io) has been running unchanged for 74 days.
* **21:00 UTC** - A test server on home network was set up and pointed to the same Google Storage bucket. This did not suffer from the same blob upload errors we were seeing in production.
* **21:15 UTC** - We conclude this has to be something internal to Google.
* **22:00 UTC** - Things are magically working again.

### What Happened

According to status.google.com, there have been no service issues with GCS. Clearly something was wrong and was fixed.
1 change: 1 addition & 0 deletions index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ on our [Slack channel](https://join.slack.com/t/foundriesio/shared_invite/enQtNT

### Previous Outages:

* [2020-02-18 hub.foundries.io disruption](2020-02-18-hub-fio.md)
* [2019-11-14 hub.foundries.io outage](2019-11-14-hub-fio)

0 comments on commit 97f92a5

Please sign in to comment.