-
Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
-
We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.
AWS outages currently, anyone else impacted?
- Thread starter Squarehard
- Start date
yeah same here. i do wonder what this means for that side of it.
for companies like disney, netflix, and etc. I think it is up to those companies to implement their failover i.e. if their website is hosted at us-east-1 and it goes down they need to implement their failover in the other regions. that's how it works for us.
but for amazon's own stuff.. lol i have no clue.
Yep. Needed to start up some instances to test a few services in our environment and haven't done a damn thing today.
As a DevOps tech lead at a medium size company, this is why I always object to cloud only compute. I'm always an advocate of hybrid on premise and cloud bursting. Don't put all of your eggs into one basket. Also if you're using a baseline level compute, on premise is pennies on the dollar for 24/7/365. Bursting is where it is economical because then you don't have to procure, power, and maintain peak compute forever.
Oooooh. That's why Amazon kept auto-loading more and more items at the bottom of the page, preventing from being able to click on customer service (where I went because chat was failing to load on the app).
It's common sense, but I swear every 2 years someone in upper management sees a slide show taunting how the cloud is inexpensive, scales virtually indefinitely, there's no procurement time, and I suspect the real perk it is inline with company direction of downsizing IT honestly because of mismanagement.
I have to explain each time that cloud 24/7/365 is orders of magnitude more expensive and requires substantial investment from R&D to move all software tooling into cloud friendly technologies. We moved to git just a few years ago.
It's easier now that I'm at a much higher level but when I was like level 2 software engineer they didn't believe me and would spend months evaluating the cloud and we're like yeah you're right. Lol
My favorite (read: least) thing about this, is all the numbskulls that come to the engineering team after an "outage" that effects our deliverables, and are always asking, "how can we protect ourselves from this in the future?" "What can we do to make sure we don't see this kind of outage again?" Yes, Diane, our 200 person company is capable of recursively and seamlessly implementing enough redundancy to keep ourselves protected from an outage that has impacted the largest companies in the world. Oh, you meant without increased costs? Of course you did.
Always have a DR in place or active-active architectures with other regions.
Surprised Netflix is having issues the way they do micro-services etc... they always said they were active-active-active between 3 regions in AWS.
For the on-prem believers, it's all about use cases. If you don't need to scale, build global web services and do AI/ML models, sure stay on-prem or try to build those services on-prem(good luck).
Surprised Netflix is having issues the way they do micro-services etc... they always said they were active-active-active between 3 regions in AWS.
For the on-prem believers, it's all about use cases. If you don't need to scale, build global web services and do AI/ML models, sure stay on-prem or try to build those services on-prem(good luck).
All the apps are in git with ci/cd pipelines, should be easy right?
I think the reason why the issues today impacted so many customers is due to the fact that their 'Global' services (like STS) map to us-east-1. We operate in us-west-2 and were still dealing with random API failures across various services we utilize.
I wonder if that was why it was taking me 3+ hours to download builds at work? I already have issues since my team is on the other side of the world.
Ended up going and seeing Ghostbusters though so that's a win.
Ended up going and seeing Ghostbusters though so that's a win.
While this is somewhat true, diversifying across multiple cloud vendors or hell, even across multiple AWS regions would have mitigated impact of US-East-1 problems. The issue is that for some strange reason (i.e. $$) even larger companies put all their eggs into single region basket... which is nuts.
Oh for sure it messed everything up. Had some packages that was supposed to be delivered to my work this afternoon that I ordered two days ago, and the status still shows that it has arrived at the local Amazon facility at 4am this morning and no other updates.