frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Ask HN: Is there a general, multi-PL programming task dataset?

1•quartztz•1y ago
Hello!

Being a student interested in PL design, I have had this idea floating around for a while: the gist is finding out what programming languages LLMs might be the most proficient in, to study their design choices and syntactic features with the goal of designing the perfect language for LLMs. This is, of course, gimmicky, but I entertained the idea for a while as a fun afterschool project.

The challenge is: what would be the best way to evaluate programming performance _in specific languages_? There are two main hypotheses here:

1. There are intrinsic syntactic/structural features that the transformer architecture is uniquely able to parse/reproduce/understand best, leading to higher quality code generated. For example: Lisp dialects make parsing code structure and blocks very easy, so one could assume an LLM can "understand their code better" 2. There is so much Python/JS out there that the question isn't even worth asking, and the performance in those will beat whatever other language you throw at it. This is probably not as much of a point thanks to newer transformer architectures but the question is still up.

I suspect the answer can be made somewhat interesting by considering performance relative to language popularity, but the ground question is: is there a general dataset containing different programming challenges, of varying difficulty, in multiple languages, with standard solutions? I couldn't find anything when I looked around, but I might have missed something obvious. It wouldn't be impossible to build a simple website to crowdsource, but I'm thinking that if I missed something obvious I'd rather find out early than late. Also, if you have any input on the project itself, I'd love to hear your ideas!

Comments

Someone•1y ago
> For example: Lisp dialects make parsing code structure and blocks very easy, so one could assume an LLM can "understand their code better"

I would expect the reverse: lisp has no syntactic sugar, making it harder for a LLM to glue code fragments together in a way that produces valid lisp code. Even guaranteeing that parentheses are correctly nested already can be a challenge.

As to a set of programs: they aren’t exactly what you’re looking for, but I would consider https://projecteuler.net (does not contain solutions, but searching for project Euler solutions” finds some) or https://benchmarksgame-team.pages.debian.net/benchmarksgame.

sargstuff•1y ago
Very open ended questions. Geeks for Geeks loosely organized around computer science topics of study : https://www.geeksforgeeks.org/

nit-pick details:

Ignoring hardware differences, "performance" comparisons can be based on differences between algorithm(s) used vs. how algorithm is implimented. For a given language, "algorithm implimentation performance" can be defined as the trade-offs on how a a given algorithm is implimented in a language (compared to other programming languages, but also easy use/flexibility based on 'language generation level -> https://www.geeksforgeeks.org/generation-programming-languag... )

----------------------

1) General computation language specialty 'modules' not withstanding; "languages" are built/optimised around core algorithmic concepts / anticipated area/concentration of targeted professional environment. aka opencl (gpu), R (statistics), Lisp (engineering design), C (OS level), sql (data selection), jasper reports, cobol (business), etc. Languages tend to be 'popular' because of the ecosystem provided around/for a given language.

snarky side note -> can always write a more standard language that compiles to an esolang & provide appropriate emacs/vim/sed/spacemacs ide support.: https://esolangs.org/wiki/Main_Page

  LLM's are very useful at curating information and recognizing/summarizing "statisical" relevance. aka apl is great for engineering mind set, not so good for business use cases aka cobal.  LLM might recognize a language for a given user that combines commonly used 'apl' aspecs of user and commonly used 'cobal' aspecs of user and recommend a language(s) with suitable commonalities for given user. 


2) Search engine topic 'coding challenges' 'algorithmic coding challenges' brings up many types of answers/sites for honing one's coding skills (various languages, beginner to expert, etc). Coding 'algorithms' vs. coming up with algorithm(s) to code is sort of a side aspect. Also differences in 'competition' challenges vs. 'technical challenges' (aka 512 c64 vs. 1 raspberry pi) ; vs. "computer science coding challenges" vs. 'computational genomic challenges'

     ?? how easy / hard based on 'profession' aka artist vs. software designer 20 years experience programming in scheme; environment -- NASA vs. google vs. insurance company.

   ?? from scratch : https://synoptek.com/insights/it-blogs/10-challenges-every-software-product-developer-faces/

   ?? based on industry standards ?? ; just trying to keep skills honed ??

T3 Code now supports multi-repo projects

https://github.com/pingdotgg/t3code/releases/tag/v0.1.12-nightly.20261121.5038
2•porterde•7m ago•1 comments

Show HN: Built a cheap and fast YouTube transcription API and MCP

https://getyoutubetext.com/youtube-transcript-api
1•mkapoor26•10m ago•0 comments

Riemann-Hypothesis

https://github.com/leanprover-community/riemann-hypothesis
1•zlu•11m ago•0 comments

Benchmarking Linux CUDA EP vs. Windows DirectML on Mobile GTX 1650 Ti(I5-10300H)

https://medium.com/@olesha-ai/bypassing-the-avx2-tax-part-6-same-hp-on-linux-cuda-wins-again-db4c...
1•olesha-ai•16m ago•0 comments

Dutch public prosecutor hacked Tata Steel for environmental investigation

https://www.techzine.eu/news/security/145955/dutch-public-prosecutor-hacked-tata-steel-for-enviro...
2•Brajeshwar•25m ago•0 comments

AMD "Zen 6" for Desktop Reportedly Pushed to Computex 2027

https://www.techpowerup.com/355030/amd-zen-6-for-desktop-reportedly-pushed-to-computex-2027
1•kristianp•27m ago•0 comments

Biochar and River Mud Recycling Cut Emissions in Traditional Silk Dyeing

https://bioengineer.org/biochar-and-river-mud-recycling-cut-emissions-in-traditional-silk-dyeing/
1•bryanrasmussen•31m ago•0 comments

ThinkBoard 0.9.0

https://vestigecounty.github.io/thinkboard/
1•sev_verso•33m ago•1 comments

My Monitor Was Cutting My Download Speed by 75%

https://blog.tymscar.com/posts/studiodisplayethernet/
2•soheilpro•33m ago•0 comments

Reports: US Agriculture Department, FBI Breached by Hackers

https://english.aawsat.com/world/5343919-reports-us-agriculture-department-fbi-breached-hackers
3•theanonymousone•34m ago•0 comments

What the World Trades. and with Whom?

https://trade.innerkore.com/
1•gagan2020•35m ago•0 comments

FBI reportedly investigating apparent hack of its criminal database

https://www.bbc.com/news/articles/cvze7de86pk3o
20•theanonymousone•36m ago•0 comments

Built for the End of the World and Beyond [video]

https://www.youtube.com/watch?v=8yULxRefZ0w
1•tosh•37m ago•0 comments

Show HN: Turn an Old Android Phone into a Customizable PC Control Deck

https://github.com/ersinkoc/phonebox
2•ishqdehlvi•41m ago•0 comments

Bending Spoons buys Mailchimp for $3.68B

https://techcrunch.com/2026/11/20/bending-spoons-buys-mailchimp-for-3-68-billion/
2•ms7892•41m ago•0 comments

Leaked Files Show Deutsche Bank Protected Russian Profits Despite Warnings

https://www.nytimes.com/2026/11/17/world/europe/deutsche-bank-russia-oligarchs-leaked-documents.html
5•doener•46m ago•0 comments

Ask HN: What is your current hardware set up?

1•dfdydx•46m ago•0 comments

GL.iNet Router and KVM Security Incident (Nov 20 2026)

https://www.gl-inet.com/blogs/blog/glinet-security-notice-2026-11-20
2•benchtobedside•48m ago•0 comments

Show HN: Stop applying to jobs using AI-generated resume

https://unmark.grandimam.com/
2•grandimam•49m ago•0 comments

Ask HN: Solo dev at an early-stage startup (4 YoE). How to land mid-level roles?

2•nasimstg•50m ago•0 comments

Verify OpenAI-Generated Content

https://openai.com/research/verify/
3•tosh•54m ago•1 comments

MRAM Basics: How Magnetic Memory Is Replacing Embedded Flash

https://www.allaboutcircuits.com/technical-articles/mram-basics-how-magnetic-memory-is-replacing-...
2•giuliomagnifico•56m ago•0 comments

Visible Autorouter for KiCad

https://github.com/imsaiguy/autorouter
2•l8rlump•58m ago•0 comments

Jenson's Charity to Invest in CoreWeave

https://www.bloomberg.com/news/articles/2026-11-19/jensen-huang-s-charity-committed-350-million-t...
2•donbox•1h ago•0 comments

An experiment with SIMD in Rust using fearless_simd

https://ankursethi.com/blog/an-experiment-with-simd-in-rust-using-fearless_simd/
2•Brajeshwar•1h ago•0 comments

Some thoughts on AI-assisted work

https://tmrw.bearblog.dev/some-thoughts-on-ai-assisted-work/
2•orbis-tertius•1h ago•0 comments

PostHog and Composability

https://sourceoftruth.substack.com/p/on-posthog-and-composability
1•alexdean•1h ago•0 comments

Echo Lantern Game

https://www.gamepix.com/play/echo-lantern
1•RENiXTech•1h ago•0 comments

Agentic Code Quality – By Addy Osmani

https://addyo.substack.com/p/agentic-code-quality
1•fagnerbrack•1h ago•0 comments

$350M Google Android Settlement Offers Cash for Background Cellular Data Use

https://www.classaction.org/news/350m-google-android-settlement-offers-californians-cash-for-back...
3•DeepLogin•1h ago•0 comments