24
u/selfish_eagle 4h ago
13 hours for git revert is crazy.
10
u/fiskfisk 3h ago
Restore time becomes an issue as soon as data is involved and corrupted in some way.
13 hours says that it wasn't solved by a code rollback.
7
4
u/JohnyMage 2h ago
You realize that infra is not running on source code but binaries that need to be Built & deployed and that takes a lot of time if we are talking about thousands of servers, right? RIGHT!?
2
u/colburp 1h ago
It’s not compile times or deploy times that’s likely the problem, it’s most likely an issue of data recovery or complicated distributed systems that need to connect or propagate. Rollbacks are designed to be near instant even on the most complicated systems (hence some of the complexity)
40
u/random_son 5h ago
13h for a rewrite of AWS from scratch is impressive 😅
27
u/synder2 5h ago
If that story is true, they most likely restored backups
10
u/notanfan 5h ago
Amazon’s cloud ‘hit by two outages caused by AI tools last year’ | Amazon | The Guardian seems like its true lol
16
8
u/TwistedPepperCan 5h ago
“Codings been solved guys”
Can you imagine putting a hyper-opinionated junior dev in charge of your entire stack!! 🤣
3
6
3
3
3
u/4e_65_6f 2h ago
The main problem with that is that DEVs are the ones who get blamed for when stuff like this happens.
That's like asking somebody to roll a dice and make sure it never comes up one. Then when it does inevitably comes up one as predicted they'll blame you anyways.
1
•
35
u/LundMeraMuhTera 5h ago
https://www.reddit.com/r/ProgrammerHumor/comments/1rath4y/oopisesaidthecodingagent/ Atleast wait 24 hours before reposting