http://finance.yahoo.com/news/Amazons-Cloud-Crashed-allthingsd-2328149185.html?x=0&.v=1

Just a brief synopsis the amazon EC2 cloud was hit with problems early this morning. From what I've read thus far via news and the Amazon web services monitor it seems to go something like this:

>Unexpected outage to one of their network backbones
>All the services simultaneously tried to mirror themselves to other nodes, which inundated the control node and the rest of the backbones
>Due to unresponsiveness multiple reboots and mirrors started kicking into action faster than standby systems coming online could cope.
>Outage stage is mostly over, but you can't launch any new instances, run backups, or use some of the API's.

Reddit, Foursquare, and several others are down. All were primarily hosted at the N. Va. facility.

4/21/2011 2:15:34 PM

and all the while PSN is down as well, Coincidence?........


yeah probably

4/21/2011 2:18:24 PM

4/21/2011 2:22:00 PM

PSN is still down?


That makes sense. I was up playing Civilizations 'till around 2AM and got some message about losing connection after midnight.

But the PSN website servers have been down for days - I was trying to look up my ID from my psp and couldn't get on.

4/21/2011 2:55:41 PM

ahhhh, explains why Ballpark isn't working either

4/21/2011 2:57:17 PM

^^

well that explains why it's been all fucked up every day when i've been trying to watch netflix.

haven't tried to play any CoD, so i didn't know what was up.

4/21/2011 3:12:21 PM

^&^^^http://www.engadget.com/2011/04/21/playstation-network-down-for-a-long-count-whats-up-sony/
http://kotaku.com/#!5794265/is-the-entire-playstation-network-down

[Edited on April 21, 2011 at 3:14 PM. Reason : links]

4/21/2011 3:14:24 PM

http://www.ps3news.com/PS3-Hacks-JailBreak/Video-Anonymous-Hacker-Group-Declares-War-on-Sony-Over-PS3/

rumors circulating that it could be Anonymous workin' hard again

4/21/2011 4:18:30 PM

is PSN definitely on EC2 or is this just speculation since they are both down today??

4/21/2011 4:30:09 PM

I know the point is to attack the ps3 but its not like psn is a paid service(yeah I know they have a premium level but who pays that) so really this is hurting users more than sony at this point

[Edited on April 21, 2011 at 4:32 PM. Reason : ^ should be completely unrelated]

4/21/2011 4:30:35 PM

I want them to be related, because I trust Amazon will have things back up and we'll probably never hear of a problem like this again for a looooong time. But I don't think they are.

PSN is said to be having problems at the japanese side of things - the mothership datacenter that houses any and everything.

Either way 2 days seems outrageous.

4/21/2011 4:43:47 PM

It’s kind of neat how the automated system “panicked” in response to the outage to a point where it made things worse… it’s like the cloud form of an allergic reaction.

4/21/2011 8:30:31 PM

^ there's actually been quite a bit of research on cascading failures and cascade-based attacks on this type of system-- some dating back before cloud computing was even a buzzword

http://globalguerrillas.typepad.com/globalguerrillas/2004/05/cascading_syste.html
http://chaos1.la.asu.edu/~yclai/papers/PRE_02_ML_3.pdf

4/21/2011 9:55:24 PM

skynet is taking over

4/21/2011 10:07:31 PM

looks like it's that time again

6/15/2012 12:28:55 AM

http://status.aws.amazon.com/

6/15/2012 7:52:08 AM

I was just reading an article yesterday that NASA has moved some of their stuff to AWS. That just makes me sad

6/15/2012 8:54:07 AM

What makes me sad is that they're using Xen.

6/16/2012 4:12:57 PM

what other hypervisors do you control the source that can scale that big?

6/16/2012 6:02:38 PM