frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

GPT 5.6 Sol is the best "vision" model OpenAI ever released

https://blog.roboflow.com/openai-gpt-5-6/
109•plurby•2h ago•58 comments

Incident with Github.com

https://www.githubstatus.com/incidents/zkxwbgr0cnmx
342•kevcampb•50m ago•269 comments

Show HN: Sokoban AI Solver

https://mkornreich.me/projects/sokoban/
24•enjoyyourlife•1h ago•12 comments

Qwen 3.8 27B is excellent, but it defaults to overthinking things

https://simonwillison.net/2026/Aug/16/qwen-38-27b/
654•bilsbie•14h ago•309 comments

Ask HN: Alternatives to GitHub

28•dhruv3006•32m ago•12 comments

GitHub down again? no PR access

87•yodon•54m ago•26 comments

My Ten Years in No Man's Sky

https://nmsspot.com/2026/08/09/my-ten-years-in-no-mans-sky/
25•blakespot•4d ago•7 comments

How Go detects struct copies with sync.noCopy

https://func25.dev/posts/go-sync-nocopy/
32•func25•4d ago•26 comments

Stripe to Buy OpenRouter for $7B

https://www.bloomberg.com/news/articles/2026-08-16/stripe-nears-deal-to-buy-ai-firm-openrouter-fo...
67•FinnLobsien•1h ago•26 comments

On A.I. regulation and messaging

https://twitter.com/DarioAmodei/status/2088758816376807762
165•jacquesm•12h ago•324 comments

Anthropic's 'watermark' text adulteration in Claude is a perversion of writing

https://daringfireball.net/2026/08/anthropics_watermark_text_adulteration_in_claude_is_a_perversi...
519•ropbear•16h ago•466 comments

Mexico Crackdown on Coastal Development Underway

https://yucatanmagazine.com/mexico-crackdown-on-coastal-development/
40•untiledsource•2h ago•6 comments

Cialis is an erectile dysfunction drug. Could it also help you live longer?

https://www.npr.org/2026/08/17/nx-s1-5928263/cialis-viagra-tadalafil-longevity-heart-health
32•brandonb•1h ago•11 comments

Show HN: Desktopcolors.com – A museum for solid background colors of classic OS

https://desktopcolors.com
71•vlowrian•6h ago•32 comments

How I developed an Am29000 C compiler and web browser

https://nanochess.org/am29000_c_compiler_web_browser.html
6•nanochess•17h ago•0 comments

A third world engineer responds to “RISC-V: They should have known better”

https://rvembedded.com/blog_post/12/
570•Narishma•21h ago•292 comments

We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility

https://www.404media.co/we-tracked-a-shipment-of-rare-books-it-ended-at-an-amazon-ai-training-fac...
19•amarcheschi•47m ago•14 comments

Linear algebra done right

https://linear.axler.net/
156•the-mitr•9h ago•57 comments

Reticulum – Decentralized Mesh Network

https://reticulum.network/
173•sudo_cowsay•14h ago•59 comments

Claude: System Prompts

https://platform.claude.com/docs/en/release-notes/system-prompts
717•tosh•1d ago•271 comments

Tell HN: GitHub Is Experiencing Degraded Performance

79•SpyCoder77•56m ago•23 comments

AGI-64 Brings Sierra Adventures to the Commodore 64

https://meanhamster.com/news/agi-64-brings-sierra-adventures-to-the-commodore-64
115•erickhill•12h ago•15 comments

The Mysterious Syndrome Destroying Endurance Athletes

https://www.outsideonline.com/health/training-performance/running-empty/
64•cwwc•2d ago•45 comments

How do I permanently disable random Google Photos popup to backup photos? (2024)

https://support.google.com/photos/thread/256212140/how-do-i-permanently-disable-google-photos-pop...
180•dt3ft•3d ago•124 comments

Build a Stratum 1 PTP Grandmaster on a Budget

https://opscode.io/posts/ptp-grandmaster-cm4-sr1723u10/
33•malcolmfrazier•4d ago•9 comments

Who Owns Commodore? The Retro PC Brand Still Exists, but a Lot Has Changed

https://www.bgr.com/2233625/who-owns-commodore-retro-brand/
19•theanonymousone•2h ago•3 comments

Rhombus 1.1 is now available

https://blog.racket-lang.org/2026/08/rhombus-v1.1.html
107•spdegabrielle•13h ago•30 comments

Gakutensoku

https://en.wikipedia.org/wiki/Gakutensoku
57•benbreen•3d ago•10 comments

SIMD in the 90s: Programming Intel's Pentium MMX

https://pikuma.com/blog/programming-intel-pentium-mmx-simd
134•ibobev•4d ago•60 comments

Applying a photosynthetic process to treat “dry eye”

https://www.science.org/content/blog-post/taking-tip-plants-eyes
64•gumby•13h ago•24 comments
Open in hackernews

Incident with Github.com

https://www.githubstatus.com/incidents/zkxwbgr0cnmx
320•kevcampb•50m ago

Comments

lol768•49m ago
Issues seems to be completely broken, just getting unicorns.

Interestingly the API still works for creating an issue (but webhooks weren't fired).

root-parent•49m ago
As sure as it is Monday....
Maxion•48m ago
Welp, workday over in Europe early!
zeafoamrun•45m ago
Who are you kidding, nobody has been in since the start of August!
augustohp•48m ago
On the bright side, they were pretty fast on the Incident Declaration this time.
beardedetim•46m ago
We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?"

No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it.

What's everyone else using? Self hosted Gitlab? Gitea?

drno123•45m ago
Self hosted Gitea
newswasboring•43m ago
When you self host, what happens with your compute costs? CI can be super expensive depending upon your test pipeline. Do you end up spending more than you would with GitHub actions for example?
Maxion•42m ago
Huh? The VPS boxes have a fixed monthly fee
applidhumanco•34m ago
He is likely referring to the CI runners. Gitea has their own version of Actions runner. It is very much usable, though currently inferior to GitHub Actions (not always a 1:1 replacement, even though they aim for it to be): https://docs.gitea.com/usage/actions/

Cost as always depends on your usage, but it will likely be less than your current cost of GitHub Actions.

JoshStrobl•
xet7•46m ago
1) Issues showed unicorn. Reloading page.

2) Issues showed Not found.

Anyway, GitHub Actions for building new release has progress. Good enough.

dhruv3006•46m ago
how long can they survive like this?
datakan•25m ago
Microsoft has thrived like this for 40 years
brookst•45m ago
It's getting to where we should just post when it's working fine, to reduce noise.
phatfish•45m ago
If you are required to use a Microsoft tool is Azure DevOps now a better choice? Our Azure hosted instance rarely (if ever) has had an outage that effects our ability to work.
chilmers•37m ago
The worry I would have is, is anybody at MS still working on it? It is barely updated anymore:

https://azure.microsoft.com/en-us/updates?filters=%5B%22Azur...

That's not necessarily a bad thing for stability, but long-term I would not be surprised if it gets EOL-ed, or radically shrunk to just a basic Git mirroring service.

lionkor•32m ago
As someone who maintained a product that was, until recently, on Azure DevOps, stay away. Like, really really far away. There's nothing positive I can say about it.
aleksandrm•45m ago
Can anyone shed some light why GitHub is having so many problems lately?
allanmacgregor•44m ago
Its all vibes man
biennvops•43m ago
Totally not because of their movements to Azure.
cybrox•42m ago
That certainly doesn't help but I'd imagine their traffic in general spiked massively with agentic interactions as well?
Retr0id•43m ago
I think the most charitable theory is that they're seeing rapidly growing usage (more users churning out higher volumes of code, via LLMs), and are struggling to scale to meet demand.

Alternative theories include "azure sucks" and "they're vibecoding everything now".

dtech•43m ago
They say it's the skyrocketing load because of AI tooling

Many people suspect it's the year-long migrating to Azure and all their products instead of the things they have been using.

thinkingtoilet
rvz•44m ago
As I said 4 days ago GitHub is in chronic operational decline which is killing trust in hosting repositories [0]

You can use as many AI agents as you want (it won’t solve the problem), but with that many outages ever since getting bought by Microsoft, it is more than enough time to give them to change. They don’t care and there is no CEO of GitHub to help either.

A self hosted instance would have much better uptime than GitHub and GitHub Actions at this point as I predicted 6 years ago. [1]

[0] https://news.ycombinator.com/item?id=49276571

[1] https://news.ycombinator.com/item?id=22867803

khvn26•44m ago
I had a lot of goodwill for GitHub but I think today is the tipping point.

Looking at a unicorn page, I feel this lingering hope that it's transient (like it usually was in the old days) but my mind reassures me it's probably going to be a long full outage again.

The hope is dead.

stingraycharles•41m ago
I’m willing to give them a break as I’m assuming they have a lot of scaling problems due to the influx of LLM assisted coding.

But maybe I’m wrong?

benrutter•38m ago
I'm really curious how other codeforges are faring. I only use codeberg and github - codeberg seems to have less outages, especially for a much smaller organisation, but they've recently removed support for LLM projects which could be linked.

Anyone here have data on whether the likes of gitlab have been impacted in the same was as github?

sznio•22m ago
my personal forgejo instance has had 100% availability for the last two years.
koalp•18m ago
I love codeberg but it also have outages and is sometimes quite slow.

But I’m more lenient with codeberg because they are a non-profit and as you say, they are banning vibe-coded project because of the cost to infrastructure. In the meantime, GitHub is encouraging vibe-coding, multiplication of the source code, and genAI discussions.

blinky88•44m ago
This is getting ridiculous. Issues, PRs, actions... nothing's working
biennvops•42m ago
Interestingly enough if you access the PR number directly it still works.
majewsky•39m ago
Not for me.
imstil3earning•37m ago
But the `Files changed` tab still 500's so the direct PR link is not worth much
Maxion•36m ago
Probably cached data and the listing search is not?
orangeisthe•38m ago
gh CLI works for me
duxup•18m ago
Not for me. Unicorn error.
esskay•43m ago
Githubs a great advert as to why you shouldn't use Azure. What an utter mess its become since they took over. Their argument that its scaling issues due to increased traffic just makes them and Azure look incompetent.
MrFurious•43m ago
Oh, shit, here we go again...
allanmacgregor•42m ago
So whats the alternative? I'm thinking I'll migrate my company to something self hosted, but what about personal projects?
warmwaffles•32m ago
For your personal projects, do they _need_ to be on a remote git server with a full GUI?
tempaccount420•42m ago
Why haven't we all moved off GitHub yet?

Or let me ask a more useful question: if we were to move, then to where? Ideally somewhere new (not Gitlab)

mrkaye97•39m ago
I think the clear reason is even with their unreliability, the cost of migrating off of GitHub for _most_ places is not worth it, and so companies don't / won't (yet)
ArcHound•30m ago
This bugs me so much. Management decides what is reasonable cost of service and the employees have to deal with all bullshit coming from that product. And you can't really escape it. Maybe the real moat was enterprise deals we made along the way.
mrkaye97•23m ago
Could be - I’d also argue the time cost to the engineers to switch is pretty significant too, regardless of the monetary cost
ndriscoll•35m ago
Ideally something like IPNS to have a mutable ref based on (repo, pubkey, branch) to resolve a commit hash, a DHT for who has what commit hashes, and then just p2p transfers. You could still have a github or gitlab for issues and CI, but code (and maybe issues) could all be p2p like git is already designed for. Your forge could also act as a peer for code cloning, of course.
bhouston•
peri-cl•41m ago
Ah, so that's what the agent meant by summarizing a documentation page as "Unicorn!" I'd thought it was a very colorful hallucination (it's just GitHub's (unicorn-themed) failure page).

    Gathered details on embedding request syntax from
        Unicorn! · GitHub
I was querying docs to figure out how to set up RAG to query docs locally. Alas.
abluecloud•29m ago
i'm surprised the agent didn't take into consideration the http status code
JoshStrobl•39m ago
So glad I moved my OSS org mostly over to a self-hosted Forgejo + Woodpecker CI. I get that we are operating on a completely different scale but I swear the GitHub outages are far too regular than they ought to be.
Maxion•39m ago
> We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available

Well it's 100% down for me, so does this mean 20% of all users are fully down?

lionkor•33m ago
It means it doesn't count towards their uptime at all, I guess, because all their uptime stats are very high for a service that is down more than anything else I use
Maxion•32m ago
> Pull Requests is experiencing degraded performance. We are continuing to investigate.

> Posted 1 minute ago. Aug 17, 2026 - 13:58 UTC

It also took them 20 minutes to realize that peoples requests to /pull/xx are not working :D

chromehearts•38m ago
And I just wondered why I couldn't update my nix flake lol
SpyCoder77•38m ago
[dupe] https://news.ycombinator.com/item?id=49330597
ape4•38m ago
Will our hug of death now make githubstatus.com crash
layer8•33m ago
I was vaguely remembering that there’s http://githubstatusstatus.com, but it isn’t very helpful unfortunately.
anoplus•37m ago
How did Github become so famous for outages? It is almost like a routine. I haven't used other code repo hosting / CI/CD pipelines platform for last decade, so maybe it's just the nature of this business, I don't know.
vanillax•31m ago
By switching to Azure....
jemiluv8•37m ago
Awwww maaaan. I was just reading an upstream patch on the erpnext framework. I was going to do their copy the pr to patch my erpnext v15
melezhik•35m ago
If someone needs lightweight self hosted git server with ci pipelines via general programming languages - consider DSCI - http://deadsimpleci.sparrowhub.io
Arubis•34m ago
This is starting to feel like Twitter. For a variety of reasons, we had a centralized place where everyone of a particular set of persuasions could connect, and this had outside benefits for the community as a whole. That place is becoming untenable, and with the loss of goodwill and stability, the community’s started to dissipate. But there isn’t one obvious transition candidate, so the diaspora is finding itself spread across a bunch of disparate places and services. The stated need they satisfy (source control with a web interface and some technical features, like pull requests with reviews) will be fulfilled. But those emergent features like a core community and default expectation of where you can find someone will fade. And that is a very real loss for all of us.
noosphr•30m ago
Luckily we already have a distributed version control system that can take over github: https://fossil-scm.org/home/doc/trunk/www/index.wiki

I suppose people can use git too if they ever figure out how to send emails from a dedicated client.

AlexandrB•25m ago
> Luckily we already have a distributed version control system that can take over github

Surprised that this wasn't just a link to Git. The biggest benefit of Git over predecessors was that it enabled decentralization. Github re-centralized the common case for the sake of convenience.

jeremyjh•19m ago
Yes but on Earth every project needs a canonical main branch. Most also need a way to manage and review changes to that branch that all project members have direct access to.
ieie3366•34m ago
To everyone who is angry: calm down. Github’s servers are constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code

Unless you have a emergency hotfix (you don’t), go hit the gym or walk outside.

If a work tool going down triggers you enough mentally to start angrily ranting online, it’s a sign you need to chill out and focus more on your health

delduca•31m ago
> increased something like 50x due to LLM

This should not be an excuse since they probably can use AI to fix it /partially sarcasm

eddieroger•31m ago
For the amount of money enterprises are paying to GitHub, there is a reasonable (and contractual) expectation of uptime. I don't think my boss would find it to fun if I bailed work to hit the gym just because GitHub was down.
iamcoder18•20m ago
Yes, but GitHub enterprise has different uptime (https://us.githubstatus.com/posts/dashboard) and actually does have an SLA.
AgentK20•30m ago
what if I do have an emergency hotfix for a production outage though ;-;
askonomm
wren6991•31m ago
At this point, they should notify us on days that it's up
bhouston•31m ago
What is the best Github alternative that has something like Github Actions? What is the easiest to switch to? I do not want to switch but it is having too much of an impact on me.

It is time to give up on Github. This is just too much.

lionkor•24m ago
* GitLab

* Forgejo

Those are my favorites. Both you can self-host for your company, including CI/CD.

nlightcho•20m ago
I am building an MIT-licensed drop-in compatible alternative to GitHub Actions at github.com/automata-ci/automata. Very early in development but it already runs some of our internal workflows.
mariocesar•18m ago
GitLab is probably the most reasonable alternative. I migrated and it took me some weeks to adapt to the new CI, but for my use case it's basically equivalent in features.

I started with the cloud version, then moved to self-hosted because it wasn't a big effort for our team. We also reduced our monthly invoice by about 50% after the move.

vanillax•31m ago
This is what happens when you try to run on Azure.
s_dev•29m ago
I recall reading years ago that cloud services were expected to run with a reliability of 3 or 4 '9's and that if they didn't competing services would quickly overtake them in adoption. The industry was supposed to be that cut throat.

Has big tech reached a similar status like banks in that they are "too big to fail" i.e. when they do fail we all just look the other way and say: "well everyone else is out too". Didn't someone recently calculate that GitHub is running at 95%? For comparison the Irish Rail service which is not reliable has 80% of it's trains run on time.

This seems absurd and really challenges a lot of ideas I had about big tech and cloud infrastructure. GitHub seems to have remained the dominant player relative to GitLab etc.

akmarinov•26m ago
Competing services can't afford the data storage to compete with Big tech
Galanwe•25m ago
That canot happen if the market becomes monopolistic, with every bigtech out there buying every startup worth a penny. The erosion of anti trust in the US created this monstrosity.
sarjann•28m ago
You would think for a company that has close to a monopoly of developer mind share wrt git that they'd maybe put in the bair minimum to get several 9s.
perks_12•28m ago
The LLM code influx is not the problem. Microsoft mismanagement is: https://damrnelson.github.io/github-historical-uptime/
MallocVoidstar•7m ago
GitHub's historical uptime page is fake, they backfilled it with 100% uptime going back to 1996: https://www.githubstatus.com/uptime?page=200
leishman•26m ago
I don't understand why Github hasn't solved this problem with pricing updates. My understanding is they are getting hammered with LLM generated code growing their traffic by over an order of magnitude. So why not rate limit non-paying users and charge for whatever scarce resource is being consumed that is causing them to constantly fall-over? This seems like a basic economics problem.
noahbp•22m ago
Because any price at all will immediately cause users to shift to another platform, and GitHub's value is that it is _the_ place to put your code on the internet.
phtrivier•21m ago
They also sell the machinery to produce the slop (through OpenAI at least).

If they make it harder to push slop on github, people might treat it as an incentive to generate less slop. Less tokens sold. Bad. Very bad.

The github paying customers are not going to move away.

Plus, it's probably genuinely hard to scale. And they can't throw compute at the problem - all compute is already going to the LLMs.

BryanBeshore•17m ago
They would benefit from a HackerNews-esque karma system.
Waterluvian•15m ago
The people with the spreadsheets have determined the value of this loss leader to be too enormous to shut it off.
DataDive•26m ago
Maybe they should charge a little more per action, and the onslaught would stop.

Seems like a simple solution.

z3ugma•25m ago
You know, I think Fossil with its integrated issue management right in the repo is actually a perfect match for the LLM era. Maybe today is the day I migrate to using a Fossil forge with GitHub becoming increasing untenable.

Anyone else using Fossil?

robin_reala•21m ago
Biggest problem with Fossil is immutable history. That’s a design choice, and I respect them for that, but it’s not the right choice for me.
ebfe1•25m ago
It's DNS issue, i'm telling ya!
fzumstein•24m ago
Funny enough, I received a marketing email from GitHub during the outage with the following subject: "What if every pull request you started actually shipped"
Aachen•19m ago
Huh, they usually do? I'm not a full-time developer though so I'm surely biased towards doing the obvious quick fixes or already-discussed things, but discussing how to solve a problem before throwing something at the wall seems like the normal thing to do for me

Or is their point about abandoned projects where the dev never clicks release after merging, or even never merges?

__vivek•22m ago
Go touch grass
hmokiguess•19m ago
I'm not mad, I'm just disappointed.
rglullis•18m ago
I'm self-hosting Gitea and Woodpecker CI for least 6 years, alongside my own docker and pypi registries. Adding x86 and ARM runners, it costs me circa $30/month to run the whole thing, and I am reasonably sure that I could make the whole thing scale at a lower cost per user than what Github charges.

I am seriously, 100%, contemplating the idea of offering consulting for migrating companies and projects out of Github. Given that I am fully employed, I'd only be able to do this on the weekends. Could I get a show of hands to get an idea of potential customers?

herpdyderp•17m ago
Sounds interesting but I'm just going to do the same thing as you (self host).
amrit_mirch•17m ago
i dont even know why people not using alternatives like gitlawb
OtomotO•16m ago
I have one client that relies on github. Just one, out of dozens I've worked with over the years.

It's just a constant pain in the ass.

Wherever I have anything to say I prefer ANY other gitforge over this piece of shit.

herpdyderp•16m ago
I've actually started loving GitHub down days because it means I get to spend time migrating away from them! (moving to self hosted Gitea)
sharts•12m ago
Question to those who work at Github…what the heck is going on? Why?
bogzz•11m ago
The GitHub failure unicorn image always brings me back to the good old days of trying out new Android ROMs. That unicorn used to be the logo of the Android Open Kang Project, which used to be my favorite ROM.

I wonder how it made it onto GitHub.

koito17•10m ago
I remember a few years back when there was 2 major outages per month.

Thankfully this is encouraging me to stop working overtime for now and enjoy the rest of my night

imnes•10m ago
GitHub, Charles Schwab, Microsoft Teams, Copilot, New York State, AWS, Progressive, Azure, Microsoft Store, Cloudflare, ADP, Google Sheets, E-Trade, Fidelity... Did Mythos break out of it's container?

https://downdetector.com/

DrScientist•8m ago
Never quite understood why large enterprises use github as core infrastructure.

Sure use it to collab externally, use it if you can't justify running your own infrastructure, but why are large companies choosing to place code externally - which is one secret leak away from compromise?

__vivek•8m ago
Someone tried loading the Bun Zig-to-Rust PR again.
standwportugul•5m ago
Funny thing about the status page is that you can't find any record of last Thursday's outage. It's all green across every service.
karel-3d•5m ago
Is that even news at this point?

Yes, github now has issues. Github is always having issues.

36m ago
Just get some x86 and ARM runners off netcup, manage Woodpecker CI + Woodpecker Agent. Generally pretty hands-off once you get it set up. That plus a cheap VPS running Forgejo with postgres off-site backups, you are looking at a small fraction of the cost compared to per-seat GitHub pricing for orgs + Action Minutes.
elric•44m ago
> We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?"

Did you have the conversation "do we need to move from $whatever CI/CD to GitHub Actions?" beforehand? What were the arguments in favour?

beardedetim•36m ago
It's a ~year old product that started on GH. So this is the first time this team has had to think through "okay, so this isn't as reliable as it was at $oldjob."
Macha•35m ago
Not the same company but similar discussions here:

- It was believed that GitHub would be more reliable than our self hosted Gitlab CI runners. This has proven to be very false.

- It was believed it would be easier to write tooling. I’m not on the dev tooling team, so can’t confirm if they found it true or not, but from the outside I see a lot more “Github can’t do that” than I did “Gitlab can’t do it”

Normal_gaussian•7m ago
Those are both hilarious to me. In my last FTE role we had the exact opposite beliefs - we wanted to leave GitLabs increases in pricing and move to annual sub, but the reliability of self-hosted runners and the flexibility (and clarity) of the CI system was fantastic.
qweqwe14•43m ago
Self-hosted GitLab and self-hosted GitLab Runners (Omnibus Docker image)
nerdypepper•41m ago
> What's everyone else using? Self hosted Gitlab? Gitea?

tangled.org!

classified•38m ago
I'm completely untangled. From GitHub.
lsuresh•38m ago
We use self-hosted Github runners, but it's a moot point if all of Github is mostly down as is the case right now. :\
c2403•37m ago
Have you considered Codeberg?
sscaryterry•30m ago
Different set of ideologies completely, not an alternative.
6031769•24m ago
An alternative set of ideologies is part of what makes it ... an alternative!

I mean, that plus better uptime.

Macha•7m ago
I remember in 2024 passing over codeberg as the host for some of my personal projects because of the uptime. I don’t think Codeberg’s uptime has gotten any better but despite that it’s getting a lead over GitHub now.
layer8•35m ago
Self-hosted GitLab at work, self-hosted Gitea at home.
speleding•33m ago
If your app is not too large you can do CI/CD perfectly fine from your laptop. Put it in a docker container if you want to ensure it is perfectly reproducible. You still need Github as the origin repo to pull from, but the "git part" of GitHub seems to be (almost) always up and running. (We use "capistrano" for this)
cesarb•11m ago
> If your app is not too large you can do CI/CD perfectly fine from your laptop.

Not if you want to make sure it compiles and works on Windows (which you don't have) or macOS (which you don't have and can't emulate). That's something github gives for free, and for me, it's the only reason to stay on github for small/tiny free software projects (at least as a secondary host).

latchkey•31m ago
blacksmith.sh just raised tens of millions on the core premise that self-hosting CI/CD is hard [0].

they've had multiple outages in the last week, so i guess they are proving themselves correct.

in practice, giving ssh/sudo to AI (codex) and telling it to setup ephemeral self-hosted runners on your own host VM (qemu), takes about 30 minutes and works extremely well. it bangs out all the shell scripts. moved my own business over to it. zero problems.

you might even find it is kind of neat too. you can customize your VM with software pre-installed. speeds up your builds quite a bit.

[0] https://www.blacksmith.sh/blog/blacksmith-seed-announcement

badgersnake•31m ago
TeamCity
•
42m ago
Microsoft simply can not make software anymore. I worked at S&P Global and when you work at a global-mega corp, you quickly see why they lose the ability to create anything and just acquire companies to grow.
everfrustrated•41m ago
Lately? GitHub has always been bad with downtime. It hasnt got worse just more visible as people do more CI now than before.
grim_io•40m ago
AI-Agents are hammering GitHub.

Whatever the actual increase in productivity may be, the commits, clones, pushes, action runs, etc. are skyrocketing.

consumer451•40m ago
Copy/paste of a comment I posted a couple months ago:

I am not trying to make any excuses for GitHub, however I recall one tweet that made my eyes bug-out:

> Yup, platform activity is surging. There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.)

> GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week.

https://x.com/kdaigle/status/2040164759836778878

https://xcancel.com/kdaigle/status/2040164759836778878

rand_r•37m ago
That's fine, but rate limiting is a solved problem at this point. They need to rate-limit free accounts and be realistic about their limits instead of letting the system fail.
Maxion•33m ago
Probably can't do that easily due to shitty engineering decisions in the past
4lx87•26m ago
Or inept managers.

Manager: Why is the site down!?

Engineer: Because it can’t handle this level of traffic. We can rate limit to the level we can handle to keep the service running.

Manager: Okay, let’s do that!

Engineer: Okay. Rate limiting at N req/min.

Manager: I’m getting complaints we’re denying service to customers!? We can’t rate limit our users if they want to use it. Increase the rate limits.

I’ve had this conversation more than once in my career.

chilmers•27m ago
Free (and non-free) accounts are already rate limited. The likely problem is that individual accounts/repos rarely hit those limits, instead the explosive demand is due to a massive growth in the number of small, individual projects being creatd.
perrygeo•39m ago
Github's, COO Kyle Daigle, claimed that the number of commits has increased 14x since last year. So shear scale has something to do with it. I don't know of too much software already operating a planetary scale that could easily absorb a 14x bump in a year.
slowin•13m ago
GitHub is pretty tiny compared to something like YouTube and Instagram. Both of which have much better uptime.
MontagFTB•36m ago
Their infrastructure is buckling under increased demand from AI generated code.

Imagine the software development lifecycle is a river blocked by several dams- necessary processes that also slow the water’s volume and speed. Writing the code was a very big dam upstream, and successive dams didn’t need to handle a lot of water. With that dam busted wide open, the torrent of code is inundating downstream processes. GitHub happens to host many of them.

Earlier this year the GitHub COO shared a post on Twitter with explosive numbers they were facing. I cannot seem to find it at the moment. (Edit: shared in other replies to this thread.)

slowin•15m ago
I mean, that’s what they say but it doesn’t pass the sniff test. Which other major service struggles so much with basic scaling? They’re not serving streaming video, they’re not a real-time system. Any problems scaling should have been fixed years ago. Especially with all of Microsoft’s resources behind them.
sunaookami•31m ago
People will say LLMs but the issues with GitHub long predate LLMs. It all started when Microsoft acquired them.
ShinyLeftPad•26m ago
Microsoft bought it, https://damrnelson.github.io/github-historical-uptime/
Maxion•38m ago
They've done the typical sillicon valley sillyness of having 10 000 engineers over-engineer a product so that there's way too many features built on top of what should be a simple tool and are now paying the price. Literally every larger US company makes this mistake.
nicoburns•38m ago
I saw an HN comment a few weeks back claiming their traffic has increased ~10x.
ayewo•10m ago
Yup.

The original source was Kyle Daigle, GH COO.

- https://x.com/kdaigle/status/2040164759836778878

- https://xcancel.com/kdaigle/status/2040164759836778878

rvz•34m ago
Well, it has been 8 years since the acquisition and they have had access to Azure infrastructure to keep up with the scale and yet, it is still struggling due to free compute thanks to Actions.

They need to start getting rid of the free tiers for GitHub Actions and actually charge appropriately or increase prices for it.

kedihacker•33m ago
It should be embarrassingly paralel I don't get how they can't figure out
blanched•29m ago
What should? Presumably if it were easy and solved their most visible problem then they would have done it already?
worble•31m ago
I have little sympathy that they're buckling over a problem they helped create.
bamboozled•29m ago
Exactly.
bamboozled•30m ago
Isn't all the LLMs and AI supposed to be "solving these problems" because all the engineers are now 10x ? Not being sarcastic but I mean, this is what we were told and sold...by the likes of...Microsoft.
slowin•19m ago
Services like Facebook have orders of magnitude more activity than GitHub and manage much better uptime.
sharts•6m ago
Even Claude could have caught and addressed most of these scalability issues long ago. This can’t be due to load or capacity. This is something else.
bjackman•37m ago
I deployed Forgejo on my homelab yesterday. Now GitHub is just a free backup service.

I guess they're quite happy with that TBF, it doesn't seem like they're that interested in being a git forge anyway.

fmbb•31m ago
They should replace the unicorn image with a horse or at least a zebra during ongoing incidents.
30m ago
I do not need it to be meaningfully different than Github, it sounds like that introduces additional complexity. I just want Github that works.
ndriscoll•28m ago
The complexity is in locating what is already a distributed database item (git is fundamentally already designed that way), just using a protocol meant for that. Basically just support `git clone` with e.g. IPFS URIs.
datakan•26m ago
Its hard to get lemmings to change course.
duxup•17m ago
For us the uptime issues have been annoying but not consequential enough to justify the work / risk of moving.

I suspect even poor uptime for many services is a cosmetic problem for most users.

ShinyLeftPad•6m ago
Who's we? Your company?

Otherwise there's no "we", Git is decentralized. Everyone can use whatever they want.

Sanzig•12m ago
Gitea (and its popular fork, Forgejo) are very lightweight. You can easily stand up an instance on a cheap VPS.
skydhash•8m ago
It's on the project to decide which is canonical and export a read only accessible version. Change review only requires a communication medium (the patches are mostly plain text) and that can be done in a lot of way. As for merging those changes, it's an organizational issue.
poisonborz•14m ago
Github and other providers before proved that git itself is just a small part of managing code, there's a lot more needed for a full workflow, and it's meaningful for it all to be tightly coupled.
MoreQARespect•16m ago
There are plenty of alternatives for git hosting. The last thing which needs to be migrated before we get an exodus off this vibe coded monstrosity is reputation (i.e. github stars) which are inherently sticky.
ryandrake•11m ago
Git doesn't even need to be "hosted" anywhere. That's the point of it being distributed. What needs to be hosted somewhere are all the barnacles (like issue tracking, code reviewing, CI, and so on) that have grown out around git, that people and businesses have become dependent on.
seanc•6m ago
Which is why they cleverly moved the central story of a software change from the commit to the PR. Commits migrate automatically with the repo. But migrating PR descriptions and comments takes engineering effort.
dimgl•25m ago
Twitter is unstable? I can't remember the last time it had an outage.
BryanBeshore•23m ago
~2007-12?!
sunaurus•22m ago
Are you potentially in the US? I think the experience is very different globally.

Ever since the Twitter -> X rebranding, every time I open Twitter links, I have a very high chance of hitting a "Something went wrong" type page, with a retry button that just does nothing. It's been like this for ages. I barely open Twitter links anymore as a result.

obeattie•17m ago
I think they may mean Twitter's 'Fail Whale' era – late 2000s/early 2010s, when it regularly fell over under load.
duxup•20m ago
And like Twitter most people will keep using it anyway?
OtomotO•15m ago
Just in the US. I have ONE European client... ONE of dozens who use GitHub.

EVERY other client I've ever worked with used their own gitforge or at least bitbucket...

nixgeek•13m ago
Selection bias then as the majority of European companies are for sure not hosting their own Forgejo or GitForge.
OtomotO•11m ago
I've asked around in my circles, because I am genuinely interested: Everyone not working in startup space IS hosting their own instances.

Hell, even I do it.

altmanaltman•14m ago
I mean if such a place is of such vitality and importance not only to its participants but society as a whole, how do you contrast that with such a pessimistic attitude? Surely people will move on, things will emerge and dissolve but the core process (again, given by the funadmental seriousness you pose in your comment) should be safe no? The free market will ensure such a void gets filled when it becomes a void. So far Github hasn't really seen any exodus that would warrant such a fear that the community is going to disappear into thin air.
icepush•13m ago
I think that, exactly how 95% of the people that left Twitter eventually returned, we will see a similar outflux of people from Github and the vast majority of them will quietly be back in a few months.
xnorswap•10m ago
That was a political decision, this is an operational one. A lot of companies have become dependent on a working github and effectively have to down tools when it is down.
mort96•8m ago
Do you have numbers to back that statement up? I don't necessarily doubt it, but I haven't seen anything to support it.

Note that raw user numbers won't really cut it; if 1 million people left Twitter, it's possible that 500 000 of those people joined and 500 000 new people joined; the user numbers would look like everyone returned, but really only 50% did in this hypothetical.

hliyan•9m ago
"Protocols, not platforms" should have remained the philosophy of the Internet. Alas, we are here.
bewareofscams•5m ago
Fuck the community, nobody need that, especially in version source control. Thinking otherwise is just dweeb eternal longing for belong, or someone has some bullshit to sell to unsuspecting dirty masses.
•
30m ago
That's an overly optimistic view of things. For a lot of us GitHub is critical infrastructure, which if it goes down loses us and our customers money.
AlexandrB•26m ago
Boomer opinion but trusting third parties to be critical infrastructure, especially with no SLA in sight, will always end in tears. "The cloud" is very convenient, but its providers will never care about your infrastructure or your customers as much as you will.
BryanBeshore•25m ago
Truly a boomer opinion. Respect!
Maxion•23m ago
Older millennial here and I agree. If you rely on Github you should at least be have your processes such that you can work around it.
raffael_de•23m ago
+1 (as a millenial) ... especially given that setting up a git server for non-OSS company code isn't too much of a challenge really. also, no need to self-denigrate this reasonable opinion in preemptive obedience.
SideburnsOfDoom•20m ago
People are already saying things like "We need a plan B in case we urgently need to deploy a fix to production, and GitHub Actions is unavailable again".

But in general, it's not feasible to do everything in house.

And I don't think that GitHub is devoid of SLA: https://github.com/customer-terms/github-online-services-sla

TZubiri•18m ago
Boomer who has never had a business? Depending on third party suppliers is business as usual.
poisonborz•18m ago
How did this become a boomer opinion? It is proved truth thousand times a day. Not that you shouldn't use third parties - but in this industry you can shrink this exposure to the minimum, and have plan B for anything else.
aenis•15m ago
Boomer here as well, but I'd add that trusting your own org for critical infra usually also ends in tears. Most everything in IT involves failure, including in well designed systems designed by great engineers.

I worked for a few years in an exceedingly well capitalised place which ran everything in their own data centers, money no object, with a truck parked somewhere, ready to go, with a smaller version of our critical infra. We had a serious business-stopping outage once every 18 months or so, every time for fringe reasons one only learns about when trying to run a large data center. Its convenient to blame the cloud and pretend that self-hosting in private sector was so, so great with six nines.

otterley•9m ago
3P-maintained infrastructure is what makes civilizations work efficiently. We're not all digging our own wells, generating our own electricity, and burning or burying our own garbage.
Waterluvian•22m ago
I'd be curious what the statistics might actually be for people who are directly affected because their business is suffering vs. people affected because their employer's business is suffering.
polycaster•14m ago
For Germany it's roughly 1:11 if you go by the self-employment rate (~8% of the workforce).
6LLvveMx2koXfwn•11m ago
Most people's employment prospects are directly correlated with their employers ability to make money.
antaviana•22m ago
"It's just money. It's made up. Pieces of paper with pictures on it so we don't have to kill each other just to get something to eat". Jeremy Irons in movie Margin Call
faeyanpiraat•7m ago
You cant eat a quote
TZubiri•19m ago
And it's not just the user's fault, GH spent a considerable effort in marketing to position themselves as such, see Github Actions and similar junk.
datsci_est_2015•15m ago
It is interesting, how much money is being lost during this outage?

My significant other was just let go from their job as a scapegoat for an organizational error: 3 layers of failure - IC, manager, director, and the IC was let go. The error caused a 7 figure loss for the company that has 10 figures of revenue per year. The manager and director may not see any consequences, though the director will probably be forced out by end-of-year due to incompetence. The new executive has taken to firing employees much more eagerly than their predecessor, like some sort of Jack Welch acolyte.

Their firing has put a lot of things into perspective for me. Mostly, fuck "at-will" employment and its negative effect on the American social contract.

But also this "angry ranting" online that the original poster was referencing. Not everyone has the privilege to calmly respond to things that directly impact their livelihood.

raincole•29m ago
Or, you know, it's perfectly reasonable and natural to feel angry when a service you paid for gets worse over time.

> Unless you have a emergency hotfix (you don’t)

Oh, we do. Given the sheer number of users, it's almost guaranteed someone is on fire ever time GitHub is down. Statistics is a very charming branch of reality.

leishman•29m ago
> Github’s servers are constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code

Which means they have a pricing problem. Rate limit non-paying users.

anoplus•23m ago
Right. It is not an excuse and they were notorious for outages before LLM era. Also, LLM can serve Github engineers as well, so we are all playing in the same field.
cluckindan•23m ago
They already do
s_dev•28m ago
What if we're not angry and just perplexed? Do we calm down further or can we still ask questions?
Topfi•28m ago
No one ever has to commit anything time sensitive.
sausagefeet•28m ago
Actually, I do have an important fix for a deal we are trying to close that does need to go out. And I am paying GitHub to host this. Once or twice, I can see, but GitHub's SLA is getting worse than just hosting it myself, and that's the entire reason I pay GitHub.
world2vec•28m ago
Geez, sorry that the paid service my company paid for is down and I can't do my work.
jjice•27m ago
> Github’s servers are constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code

I sympathized with them when they said this a handful of months ago, but then I saw this [0] page that shows how it's been shot for years prior (which tracks with my memory).

I feel for the GH engineers that have to deal with this, especially the SREs. I also don't hate the downtime right now, as I'll make a cup of coffee and do something else. I will say though, I did have a hotfix a week or so ago during the Actions outage, which really was a pain.

You're right that getting angry and ranting isn't the right reaction here, but I do no give them the LLM load excuse. I don't give them an out for having awful uptime during work hours for a product we pay quite a bit for as an org.

[0] https://damrnelson.github.io/github-historical-uptime/

Topfi•23m ago
Github being under the CoreAI division probably also doesn't help the engineers prioritize addressing infrastructure issues and makes using LLM load as an excuse feel self-inflicted. Akin to feeling sorry when a pyromaniacs house burns down...
debarshri•22m ago
It looks like right after Microsoft acquisition this started going off.

This is the outcome of violating single responsibility principle in business.

twistedpair•15m ago
They _are_ the LLM coding agent vendor, and _owned_ by MS, the majority owner of OpenAI. How can you NOT consider that 20x+ scaling in your capaciticy roadmap projections, we're you are trying to get everyone to use these agents as part of your core OKRs?

Or, maybe your #1 IT priority was moving everything to Azure instead ;)

cesarb•27m ago
> go hit the gym or walk outside.

My mind immediately went to this classic xkcd: https://xkcd.com/303/

javier2•26m ago
While I agree in principle, Github with their CI is very critical to many parties.
sarjann•26m ago
This has to be rage bait, this is a critical piece of infrastructure for many people. Stuff going down can lead to deployments failing and as you mentioned in many cases emergency hotfix's.

The idea that we should be fine with this unreliability is just amazing. It's not a mental health issue to have problems when important infra fails.

john_strinlai•23m ago
>The idea that we should be fine with this unreliability is just amazing.

the comment doesnt say you should be "fine" with the unreliability.

they are saying people shouldnt get so emotionally worked up over it. which, while i wouldn't phrase it in the way the parent did, i agree with the direction of their point.

dimgl•26m ago
What if you, you know, have a job?
phtrivier•25m ago
I understand the need to level-head the discussion and calming everyone.

However, your "(you don't)" comment is not going to calm down all the people, who, you know, DO have a hotfix to push now, and DO have an angry customer that could not care less for which part of our infrastructure is breaking _their_ workflow.

The only things would calm everyone down is guarantee that Microsoft would be paying for _our_ SLA breach compensation. But they don't. And I don't think anyone is paying their GH bill with a prorata of the number of time the platform was actually available.

(I'm also aware that the wording of the contract probably clearly says that you should not use GitHub for anything critical, that Microsoft is only a small startup in their garage, that you can't credibly expect 90% uptime anyway, and that it's all the fault of LLM slop ! Bad LLM slop ! Also, please buy our LLMs to generate more slop, please.)

bhouston•25m ago
> To everyone who is angry: calm down. Github’s servers are constantly on fire

Is this rage bait? Isn't Google, Amazon, and all the other services in the world similarly impacted by LLM's? Isn't Claude, OpenAI, etc.? Why can they handle the load but not Github?

denysvitali•25m ago
Stop blaming LLMs: https://damrnelson.github.io/github-historical-uptime/
c2403•23m ago
In this case, perhaps they shouldn't allow AI slop or promote usage of AI agents...
madeforhnyo•21m ago
> due to LLM sloppers pushing large amounts of trash code

Which they encouraged by pushing Copilot down everyone's throat

slowin•21m ago
Well, they’re a critical piece of infrastructure with terrible stability. I’m in the process of migrating us off GitHub now.

I’m not a Meta fan but it’s interesting that they manage to keep their systems up with an order of magnitude more traffic. GitHub’s uptime is inexcusable.

AustinDev•20m ago
Alternatively, don't use GH for critical infrastructure they are no longer reliable, simple as.
TZubiri•20m ago
is the infrastructure of the free tier shared with that of paid users? That might be the issue. Otherwise if usage of paid users scales, then it would be fine.
john_strinlai•15m ago
>is the infrastructure of the free tier shared with that of paid users?

github enterprise is fully operational at the moment

nope1000•7m ago
At least enterprise EU is partially affected

https://eu.githubstatus.com/posts/dashboard

We cannot pull any actions images (or how it is called) for example.

neo_doom•19m ago
Agreed. This impatience culture is so toxic. Been down less than 30 mins.
anonymousab•19m ago
Allowing and accepting the LLM load is a willful choice they made and are making at the expense of their users, including their paying enterprise users.

They could easily tighten things up in that regard, and make a choice that is right for their main users at the expense of The MS corpo mandate/mission. It is a choice to do otherwise.

pamcake•12m ago
Allowing and accepting? More like encouraging and instigating.
constant_flux•16m ago
GH shouldn't have promoted LLM usage the way they have if they didn't have the infrastructure to support it. Regardless of how you feel about the code quality (which you have zero evidence of), if GH has made commitments to supporting broad LLM usage, they need to back that up with the proper hardware and without whatever fragile SDLC processes they have.
stetrain•15m ago
> as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code

Github and their parent company are active participants in pushing LLM-driven coding.

DharmaPolice•15m ago
This might be naïve but wouldn't the appropriate response be to reduce access/rate limit free/new accounts in order for service to be maintained for everyone else?
nerdix•14m ago
Wouldn't be surprised if we lost free private repos because of the slopocalypse. Though most don't seem to be shy about sharing their slop with the world so I'm not sure if that will actually increase stability.
sharts•11m ago
It has nothing to do with load.
otterley•7m ago
The CTO said otherwise: https://github.blog/news-insights/company-news/addressing-gi...
OtomotO•9m ago
You know, I am not employed.

I have little time in my day to spend with my daughter. But I also have responsibilities. And one client has decided to bet on github. It's the only client I've ever worked with who has their code on github. EVERY other client has hosted their own gitforge or used bitbucket.

So now I am not angry because some critical piece of US-american infrastructure is down all the time.

I am angry because instead of spending time with my daughter I have to work on this later, because there are due dates and "Well, fucking GitHub was down" ain't gonna cut it.

malfist•8m ago
Why do you get to tell me how I should feel?
kilroy123•7m ago
As a _paying_ customer, I disagree.
semiquaver•5m ago
Why would you take what they say at face value? I have my sources for the fact that this is basically a lie covering up deeper rot.