GitHub Incident with Git Operations, Pull Requests and Actions

https://www.githubstatus.com/incidents/djlmxz2zd0j7

127•gagan2020•1h ago

Comments

geooff_•1h ago

Actions being down is one thing - I can't even land a PR.

vitosartori•1h ago

PR? I can't even push haha

laurent123456•1h ago

I can't even create issues (error 500)

gagan2020•1h ago

Same here. I wanted to push something on Github and its was giving 500. This is most severe issue in recent times.

balanceiskey15•1h ago

GitHub added new webhook IP ranges this morning, curious if that's related.

gagan2020•57m ago

That might be the cause but let's see.

There 4 updates contains webhook mention in 2 of the updates.

"Webhooks is experiencing degraded performance"

iercan•58m ago

first time seeing "git push" on github throwing an "Internal Server Error"

cowlevel•56m ago

gitea, forgejo, gitlab CE, and ssh still working fine

hermannj314•32m ago

Exactly. I'm running forgejo in a container, pushing backups to a private S3 storage, and sent the forgejo team a small liberapay payment.

Maybe 30 minutes to get it all setup and then I moved on with my life.

user43928•30m ago

I never noticed downtime with public GitLab. It's also generally better than GitHub and Actions.

A self-hosted enterprise GitLab instance in Azure, on the other hand, has had outages and performance degradation quite a few times.

cowlevel•22m ago

Of course. Azure is the reason GitHub is down so often, too.

alexellisuk•56m ago

I thought it was odd that we'd not had one of these for a few weeks.

A few hours ago GH pages just would not publish at all.. and the status page showed as fine, I guess it's now cascaded into a proper event.

cowlevel•22m ago

There was one yesterday.

malfist•20m ago

Not had one for a few weeks? There was an outage on Monday (as usual)

caaqil•56m ago

Who even runs GitHub at this point?

dam_jackalopes•53m ago

I just work here man.

Kudos•53m ago

Copilot.

herpdyderp•49m ago

According to themselves, 225 million people.

https://github.com/about

esafak•49m ago

Who's the commanding officer here, soldier?

freakynit•55m ago

Spent 5 minutes debugging why my builds were failing... then recalled github is not google when it comes to availability... checked status.github.com ... and voila.. there it is.

embedding-shape•35m ago

I'm surprised people still rely so much on GitHub. Anything I want to (have to*) be able to deploy on a moments notice, for hotfixes and what not, was moved away from Github like a year ago, because of the horrible uptime. Did others not get the memo yet?

It's not getting better, if anything it's getting worse. Best time to get off GitHub was yesterday, the second-best day to get off is today. Tangled, Codeberg or self-hosted Forgejo (my approach) or Gitea are all good alternative solutions here.

pixelesque•54m ago

One of the PRs I eventually managed to open 8 mins ago had a page flash up a generic error page containing a badge error of: "Unexpected token '<', "<!DOCTYPE "... is not valid JSON"

aarestad•52m ago

99% of the time, this is a server returning a generic 500 HTML response to an HTTP call that was expecting a JSON response.

aleksandrm•51m ago

"Rewrite GitHub in Rust. Make no mistakes."

jeremyjh•49m ago

codefloe.com is still up

freakynit•47m ago

So, "git push" is down ... how are github engineers gonna push the fix now?

whalesalad•44m ago

sudo git push

pkthunder•44m ago

Make no mistakes.

tripleee•37m ago

in vim over ssh of course.. agents can use vim right?

dodslaser•25m ago

Yeah, but they keep trying to use Emacs commands...

C-x M-c M-unfuck-git

bearjaws•36m ago

rsync and sftp, like god intended

redanddead•28m ago

Like the olden days

ChuckMcM•

oslem•45m ago

Color me shocked

nstott•33m ago

This is so frustrating. I wish it were easier to migrate away from this tool

Brian_K_White•45m ago

holy crap push is a new low

acclay•44m ago

I'm tired, boss.

rapind•42m ago

Based on the comments in this thread, you'd think that Reddit was down.

icantevenhold•8m ago

Complaining that HN is becoming Reddit is imo even worse than posting Reddit-level comments - those are at least sometimes funny

iambenm•6m ago

"Please don't post comments saying that HN is turning into Reddit." [0]

[0] https://news.ycombinator.com/newsguidelines.html

schrodinger•2m ago

This isn't that imo... do you think it violated the spirit of that rule?

DaSHacka•2m ago

I suspect it'll only become more common as Reddit enshittifies further and further. The platform is basically unusable for anyone, thus its suitability as a 'containment zone' for the redditor phenotype goes down with it.

I hope BlueSky makes a Lemmy-style Reddit alternative, just something to satiate people's social media-ified forum fix.

rvz•42m ago

The last time GitHub went down was...yesterday.

GitHub is never going to improve and at this point. You might as well say that the AI chatbots, Tay and Copilot are maintaining the platform and are running it into the ground. No CEO of GitHub to go to as well.

It is time to self host instead of using something as broken as GitHub as I predicted 6 years ago.

time0ut•42m ago

Well it is a Wednesday.

I noticed because SSO was down. They don't even report on that as far as I can tell.

coliveira•41m ago

That's what you get when the bulk of open source is now depending on Microsoft.

knowknow•40m ago

Embarrassing that a multi-billion dollar corporation that other billion dollar corporations depend on seems to be fine with breaking their clients workflows. Was github always this shoddy or has something changed to cause this many outages?

TavsiE9s•39m ago

I mean... they got acquired by Microsoft.

cowlevel•24m ago

Almost 10 years ago

embedding-shape•34m ago

They used to run their own data centers, and while the unicorn (their error page back in the day) was visible sometimes, it wasn't nearly as bad as things are today.

And of course, Microsoft is saying that none of this downtime is at all related to them moving everything to Azure, and also at the same time they'll fix all this downtime by finishing moving everything to Azure.

I wouldn't hold my breath here.

mjr00•31m ago

> has something changed to cause this many outages?

AI coding. Which can be interpreted one of two ways:

* The generous way, which is that Github is so overloaded with massive volumes of AI codebases and AI-driven automation that they're hitting a scale they never anticipated; or

* The not-so-generous way, which is that Github itself was one of the first companies to push everyone to AI code as much as possible, which has lead to an eventual breakdown of the stability of the system, as the people responsible for it no longer understand how it actually works, leading to production outages once every few days.

MadrasTh0rn•36m ago

Love it

gwbas1c•36m ago

What's frustrating is that:

1: The status site says that pull requests are working,

2: A PR I'm working on right now is missing commits,

3: The contact support page errors out.

bossyTeacher•36m ago

This is becoming way more frequent than a monthly event. Does anyone know why GitHub keeps having incidents?

christophilus•33m ago

It’s mentioned on most threads, but their traffic and usage is way up due to AI / agentic programming.

cowlevel•23m ago

Because it's vibecoded and because it's running on Azure

tripleee•34m ago

Three days in a row now right?

HenkieTenkie•34m ago

Another case of tactical downtime.

sleepybrett•33m ago

i see they started letting the github engineers ship code again after that shitshow last month.

user-•28m ago

I can attest to the outage starting around 11:05, as that’s when my GitHub auth for Tailscale SSH broke. Of course, the next 10 minutes of troubleshooting and checking GitHub’s status page showed “all systems operational,” so I assumed it was something on my end. Then, it started working again.

And while I was typing this comment it just blipped again. Atleast this time I know why.

I feel dumb for setting up tailscale like 7 years ago using the github auth, time to figure out how to just move to email on that account.

honeycrispy•23m ago

Funny, yesterday git operations had an unreported outage too. I couldn't pull changes for ~10 minutes.

odo1242•17m ago

To be fair to GitHub, outages are pretty short (~40 minutes), and they rarely impact critical parts of the service, nor do they affect one’s ability to work (git offline)

slowin•11m ago

Not being able to push is pretty critical.

lenerdenator•10m ago

I'm not able to get code pushed, meaning I can't address PR feedback, meaning I can't get things approved, meaning code isn't getting merged to main and thus a ticket isn't getting closed. Sounds pretty critical to me.

OrderlyTiamat•7m ago

40 minutes is an eternity when you can't push to a gitops repo with a PRD issue.

mschuster91•5m ago

... and that is (yet another example) why pipeline driven gitops is a really darn bad idea.

m2ger•6m ago

git push ending up with Internal Server error is similar to save file ending up with disk full error. yes, it can work after 40 minutes, and one can go to lunch and retry later, but here in Europe the troubles started at the end of a (normal) workday.

acedTrex•15m ago

Im so grateful that coding is solved

esafak•14m ago

This is such an opportunity for competitors. I hope they don't squander it.

hdgvhicv•8m ago

Must be a Y in the day

udave•7m ago

Hope it survive the next few years of codification of all of humanities unfinished cheap ideas.

mococa•3m ago

Again?

GitHub Incident with Git Operations, Pull Requests and Actions

Shipping JPEG XL in Chrome

A font recreated from photographs of classic Commodore 64 keycaps

Nobel Prize in Chemistry 2026 to Henri B. Kagan and Kenso Soai

Show HN: A walkable 3D art history museum built from Wikipedia

Animated ASCII Art for Web Pages

Google Playground

Navier–Stokes Lost in Translation

AI-assisted proof of optimal packing for 11 squares

Write Like It's 1866: LLMs Relearn Telegraphese

Mallet Head Angle

Anti-Patterns in Software Blogging

Wood Tape (2004)

Across the Globe, People Increasingly Say Social Media Is Harming Democracy

Rust's derive often implies inline

Reverse Engineering of the M-VAVE FM-1 Pocket Synthesizer Firmware

Thank You Indonesia for the Clean Air

Show HN: Procinsh – A 3D Linux process inspector

How many GPUs is 1M/B/T tokens?

AI firm HUMXN offers free plumbing and HVAC service in Minnesota to train robots

Show HN: AstroHelm – Use your phone camera to aim a telescope or telephoto lens

Strands Decider 2B: a small, open-source, decision model

ShinyHunters Extorted Boeing Spin-Off Prior to Arrests

Visa, Mastercard, Major Banks Facing New Litigation over 'Anticompetitive' Fees

God of War on PSP, recompiled to WebAssembly and running in the browser

Sharing AI progress in mathematics

SynthID Detector

Mistral Large 4

House with 15m underground tunnels for sale for 300k

Paramount, Warner Bros Formally Merge, Form Giant Mountain of Disastrous Debt

GitHub Incident with Git Operations, Pull Requests and Actions

Shipping JPEG XL in Chrome

A font recreated from photographs of classic Commodore 64 keycaps

Nobel Prize in Chemistry 2026 to Henri B. Kagan and Kenso Soai

Show HN: A walkable 3D art history museum built from Wikipedia

Animated ASCII Art for Web Pages

Google Playground

Navier–Stokes Lost in Translation

AI-assisted proof of optimal packing for 11 squares

Write Like It's 1866: LLMs Relearn Telegraphese

Mallet Head Angle

Anti-Patterns in Software Blogging

Wood Tape (2004)

Across the Globe, People Increasingly Say Social Media Is Harming Democracy

Rust's derive often implies inline

Reverse Engineering of the M-VAVE FM-1 Pocket Synthesizer Firmware

Thank You Indonesia for the Clean Air

Show HN: Procinsh – A 3D Linux process inspector

How many GPUs is 1M/B/T tokens?

AI firm HUMXN offers free plumbing and HVAC service in Minnesota to train robots

Show HN: AstroHelm – Use your phone camera to aim a telescope or telephoto lens

Strands Decider 2B: a small, open-source, decision model

ShinyHunters Extorted Boeing Spin-Off Prior to Arrests

Visa, Mastercard, Major Banks Facing New Litigation over 'Anticompetitive' Fees

God of War on PSP, recompiled to WebAssembly and running in the browser

Sharing AI progress in mathematics

SynthID Detector

Mistral Large 4

House with 15m underground tunnels for sale for 300k

Paramount, Warner Bros Formally Merge, Form Giant Mountain of Disastrous Debt

GitHub Incident with Git Operations, Pull Requests and Actions

Comments