why are they using software to detect software?
I can detect AI written prose in less than five seconds; I would expect a trained teacher to be able to do that as well.
Have you considered using your own words to express those thoughts?
As soon as someone yells "witch" you cannot disprove you're not one, and I've even had people put my handwritten comments through "AI detector" websites that "proved" they were AI (they weren't). It literally just highlighted two popular English phases.
LLMs were trained on sites like HN and Reddit, so now if you write like a HN or Reddit commentator, you sound like AI...
LinkedIn, OTOH....
If someone calls an article like this a "jeremiad" I know they're a human.
Just make it be what you want to say and how you want to say it. And when they come after you, shame them to the best of your ability or treat them like they are not there.
It wasn't someone who was primarily motivated by fear of the past that made it work the first time.
I've begun downvoting each and every entry that questions the authenticity of a comment or article.
I don't even bother if the claim is true or not. A text can be AI-generated and interesting, or human-written and dumb.
This will likely be valuable for AI skills too.
The schools simply don’t have the flexibility, agility, or frankly it seems motivation to adapt to what has already happened.
The ship has sailed; essay writing is no longer a viable form of assessment.
The idea to try to build a reliable AI detector is asinine, and fundamentally misunderstands how any of this works now, let alone the very obvious trend-lines.
Stop with the lazy half-baked solutions, get your head out of the sand, rethink the whole curriculum. This is an emergency, we needed to be urgently attending to this years ago.
This has nothing to do with Public School in particular. This is impacting private and university education too.
Of course it is. In person, with an unseen prompt/question. By hand or not doesn’t really matter as we can airgap or just monitor via software when in class.
If now teachers abdicate this judgment to a software, students should be allowed to abdicate their duties to a computer as well.
I've noticed I write a lot different because of combative online arguments. I have a problem.
So much of my communication is directed to people who don't want to hear me or understand me. So I've become very punchy and repetitive, trying to hammer home ideas that people are either unable or unwilling to understand.
I need to find ways to talk to people who want to hear and understand me.
It's hard to find other people who actually want to hear and understand though. People have different interests, and even when people appear to be working towards the same goal, they often aren't; like a boss who just won't understand the bad news, because it's easier to ignore the problem.
I'm told blogging works for some. I don't really know how you build an audience, though, and it's hard to keep going (first-hand experience) without one.
For example, I abhor talking about modern politics. If it’s election season and I’m being asked to cast a vote or take some other specific civic action, then I understand it’s my civic duty to understand the situation and make a decision accordingly and I do.
But if it’s March and there’s really nothing specific I can do as a result of this particular conversation, I would probably also be in your camp of the “unwilling”. I would much rather chat about something else, or nothing at all.
I'm also assuming you're referring to in-person communication. If it's online communication, all bets are off. It's unlikely you're having a linear conversation and these days you're probably not even talking to a person.
I find that a little faith goes a long way here: assume that you have a higher audience and speak to them accordingly.
Don't let the loud ones confuse you: normal, reasonable people (with normal, reasonable thoughts, just like yours) might not always reply, but they also read you.
The result is that a ton of web forum/social-media posting would, in any other context, be fairly poor writing (even if it's otherwise got no problems) simply because of the the extra crap and contortions required to minimize garbage posts by poor readers who are, themselves, allowed to post to the same medium.
This is in addition to, though not wholly separate from, the tendency toward combativeness in online posting.
If they don't want to listen, why waste the time?
> So I've become very punchy and repetitive, trying to hammer home ideas that people are either unable or unwilling to understand.
If they don't want it, why stuff it down their throats? Aren't they allowed to have their own ideas?
Ask more questions. It takes work when dealing with smart people who think beyond the question you asked, adding their own context, and then replying with a different question. But those are the people who are willing to engage with you. Statements without questions can be ignored, and people who engage with different questions than the ones that you asked can be safely ignored as those who don't want to engage.
The cure to a purely adversarial conversation is educated curiosity. The educated part being being able to differentiate the threads that will lead down a tribalistic path vs those that will lead down an exploratory one.
More important than all of the above, is knowing when to walk away. It's barely a majority, but that barely majority "want" to waste your time. Ignore their DOS attempt, and save your time for people who want to engage, fairly. The fairly part being the most important.
Honestly, I lean towards shaming educators who do that. If you can't detect the whiff of LLM with your own senses, then it has been used properly and shouldn't be faulted. If that premise invalidates your assignment, change the assignment. It's not as if you're assigning this work to test the basic mechanics of writing (grammar, sentence/paragraph structure, parallelism, whatever) — I mean, how much of that did you consciously try to teach? My recollection is, not an awful lot; and I can only imagine it's gotten worse since I was in K-12 (and I went to pretty darn good K-12).
But wouldn't this apply to any cheating method? I don't think educators would be able to tell the difference between using a calculator, getting answers from previous tests, resubmitting assignments, etc.
> using a calculator
Students who are at a level where they'd be learning to do the computations a calculator does, shouldn't have graded homework. And even at that level, real mathematics is more than just computation.
> getting answers from previous tests
Decades ago, my teachers and professors knew advanced tricks for this, like "not just reusing the test questions from last year". Sometimes they even changed the constants in math questions between sections of the class.
Reading previous tests (including correct answers) was never considered cheating, or even slightly unethical, in my education. In fact, one of our professors had this party trick of working through all the answers for a past-year exam (perhaps multiple of them; I can't recall the details, but certainly much faster than students were expected to work things out under exam conditions) in the space of a single lecture, near the end of the course. Students were meant to see this and learn from it (as well as be impressed).
> resubmitting assignments
Why would you ever not notice this?
>Students who are at a level where they'd be learning to do the computations a calculator does, shouldn't have graded homework. And even at that level, real mathematics is more than just computation.
So, a math level less than Real analysis shouldn't have graded homework?
>Decades ago, my teachers and professors knew advanced tricks for this, like "not just reusing the test questions from last year".
Math is not the only subject. For an English class, what constant would you change so that students get a comparable exam (especially if you are going to do this between sections in the same corhort)?
>resubmitting assignments
Students are not stupid, and obviously would not resubmit an assignment for the same teacher. However, there is a significant overlap between classes, so certain assignments should be retooled for other assignments.
The discourse around "cheating" with these products has always been a mistake. We should have characterized them less as "cheating machines" and more as "expediency machines." Because once you're invested in describing students as having academic dishonesty issues rather than skill issues, you've made it an administrative problem. You never come back from that.
For mine, we lost the issue long ago when accountability culture won. We should never have bothered with the idea that "mechanics, grammar, and proofreading" should be part of a "rubric" that "assessed outcomes" for "good writing." We should have just said "we don't care if you don't think this is worthwhile, because your time is worth nothing." The last two years of student labor certainly suggests this.
In my experience educators no longer use AI detectors given the risk of false positives. But some work is obviously lazy AI content. When that happens, educators talk to the student to see if they understand what they wrote.
Teachers cope with more in person writing, oral presentations, defense of what’s been written.
If you think out it the pre-AI computing generation is itself anomalous for having ubiquitous access to efficient human-only writing tools. We probably wrote more than previous generations. Early Internet / blogging culture bears this out.
Teachers are being hamstrung on curriculum. The districts enter into contracts that require the use of certain programs for certain amounts of time. We've known for decades (if not a century) that direct instruction works [1] but you can't sell devices, platforms and consulting services that way.
We're literally at the point in education we were in the 1950s when the health benefits of nicotine in your Q zone were lighting up the airwaves.
And generative AI means it's all but impossible to have take home writing assignments. But hey this is another opportunity to sell AI or cheating detection software, that's often just an em-dash detection [2].
We have a generation that gets to college quite possibly having never written a book. social promotion through grades and the constant distraction of electronic devices in classroom settings. I don't even necessarily blame the parents entirely either because we've constructed a society where 2 people need 5 jobs to make ends meet.
And while all this is going on we have a coordinated and well-funded effort to defund public education and move government funds to private schools based on the failing public education that's failing because we defunded it. This is usually backed up by some baloney study that shows charter shcool produce better results that really comes down to charter schools being able to be selective with enrolments while public schools cannot be. Plus we mingle in special education kids into public education because those programs got defunded too.
And really that's just a bunch of already affluent people who want a tax break for doing somethign they were going to do anyway: send their kids to private schools so they don't have to mingle with the poors and aren't taught inconvenient things like human reproduction, critical thinking and self-determination.
And after all of that we just end up teaching kids how to pass standardized tests.
[1]: https://marginalrevolution.com/marginalrevolution/2018/02/di...
[2]: https://medium.com/@brentcsutoras/the-em-dash-dilemma-how-a-...
Did not this self censorship process started decades ago? There are certain answers expected in academia, arguing for anything else would get you in troubles. Not using “devoid” seems pretty minor inconvenience.
For me biggest wtf is why students are still expected to write graded essays, and to keep this make believe it is somehow useful and applicable skill.
In short it’s a good way measure thinking.
Glad to see some schools and teachers teach how to use them well, rather than ban them outright.
How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term?
All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They can prepare as they wish, use AI, and even cheat on the homework, but there will be a revelation at the end of the term.
That final test can be proctored, monitored, audited to ensure that whatever words are used are indeed the student's own words. The resulting grade depends on that, and that alone.
The approach of continuous assessment, which to me always seemed suspect and ripe for abuse, was completely broken by the AI tools that are now available.
Do you only learn when you’re being graded?
As a general rule when changing complex systems, you sacrifice what you aren’t trying to optimize. If you make a random change to a car without consideration for gas mileage it’s very likely to reduce gas mileage.
(The other side of that contract is, kids are not merely attending schools to learn, but to earn a degree that carries some degree of prestige)
A grade, on a single proctored test, is a crude metric, but at least it would be a brutally fair one.
There are things you learn from spending several days structuring a 20-page argument that you will not learn (and cannot assess) from oral examination or a 5-paragraph essay written in a blue book.
That knowledge will show up in the blue book vis-a-vis the other exam candidates.
I would prefer not to be confrontational here, but I am having a hard time imagining that you've deeply considered the pedagogy of how to teach and evaluate students on squishy skills like this.
Knowing a bunch of facts about something is a world apart from structuring a compelling in-depth argument about it.
I love this idea. And if a student is having a really bad day, or their dog just died, or they have bad cramps, or they have a hard time dealing with the intense stress of your entire grade being decided in one exam... well, those loser students can just fuck right off.
Accommodations are part of the fabric already. It doesn't seem inconceivable that we couldn't deal with them in exceptional circumstances in a similar way to how it's done today.
... well then, why not use those same protections (proctoring, monitoring, auditing) in continuous examination?
Accommodations are real and necessary, but applied at the end.
(Experimental sciences are an exception)
A better approach is to rethink what we assess and how we assess it. Research shows that the design of assessments plays an important role in academic integrity. Assignments that require original thinking and regular engagement can reduce incentives to cheat and improve learning outcomes.
https://www.sciencedirect.com/science/article/abs/pii/S22119...
If the only remedy is monitored end of term exams, so be it.
Of course, creating good exams is difficult, but you have to do that either way.
Sure, it requires more resources, but it really shouldn't require much more:
- We've had multiple exams before AI, and I don't see how AI makes it any harder. Obviously these are closed-book
- Schools should already be banning phones in class (and colleges have insane tuitions, they can afford more exams)
- The students who go out of their way to cheat - as long as they're a minority, let them. Why not? Either they'll fail later in life, or they didn't need to learn the material because they're pathological fakers (even if you won and forced them to learn the material, they'd probably still fake their way out of using it). I doubt you need much proctoring to ensure that most students don't cheat, since most of the smart students are generally smart enough to know that actually learning is important.
Meanwhile, downsides of one exam:
- Disadvantages students who get overly stressed about particularly important exams, or have a particularly bad day on the exam
- Many students will blow off the (ungraded) assignments and put off actually learning until the end
- Less graded content (especially if the exam isn't overly long, which would disadvantage some students)
This reminds me a bit of that. AI writing is—in many ways—objectively very good, but that doesn’t matter if no one thinks you wrote it. AI writing is boring exactly because it is consistent and like any art form people want to see something original.
If your school uses software to detect AI writing, that's a problem with the quality of your school. The people choosing that software are too stupid to be running a school. The software isn't going to get any better.
The problem isn't that AI detection doesn't work. State of the art in this field is pretty solid. The only issue is that it's probabilistic, so it sometimes fails, and when it does, we have nothing else in situations where you actually want to know if someone put in the work.
So what are you proposing, exactly? That we run a large-scale experiment of "let's see what happens if children don't actually need to learn to do thinking and writing on their own"? The reality is that without some form of compulsion, most kids would rather play video games / scroll through TikTok all day. Or that we move to a vastly more resource-intensive model where every kid is given personalized instruction and watched 1:1?
That's what fortunetellers do. The problem isn't guessing correctly about AI content in writing. The problem is false positives. That's what puts it in the same category is predictive policing scam software. And fortunetelling.
(This is roughly the same problem as evaluating software that only does an approximation of what it claims to do.)
(Aside: AI-based variations on this theme are in the early stages of proliferating across our society. They're being developed by many people using this forum and being sold to our schools, businesses, governments, and other organizations with little regard to whether they actually do what they claim.)
My institution subscribes to TurnItIn's AI detector. The documentation is quite clear that the system is tuned in a manner that produces a significant number of false negatives and minimizes false positives. They also state that they don't report anything under "20% AI-generated" content.
So the marketing I've seen is intended to reassure skittish administrators that the software is not going to generate false accusations.
That being said, I have no idea whether the marketing claims are true. The software is a black box.
On a side note: the fixed-pattern essay thing seems to be an American invention, or at least popularized by the American education system.
These kinds of things are novel to us and deserving skepticism, but become just the world we live in to them.
It turns out to be built into the training data. The diffusion model just doesn't have many references of naked people not embedded in porn tropes, so it autocompletes porn.
Online moderation of generated images have the same weird incentive. Since real people seldom film themselves having sex, a naked person not having sex is a red flag for a possible real person, and gets moderated more strongly.
So in the new world, well written sentences are a handicap and nudity is generally accompanied by an exchange of fluids.
But, the article's focus on writing "worse" for AI detectors misses what is important. Trying to distinguish humans from machines does not develop student capability. In fact, it's a fleeting technique because AI writing styles will vary and improve over time.
Paracompact•2h ago
I think grading in general can be stymying for students' motivation and creative drives.
idontwantthis•2h ago
Onavo•1h ago
carcabob•1h ago
I wish brevity and linguistic precision were taught more, as well. Miscommunication due to ambiguity is one of the biggest causes I see for confusion or heated arguments.