Why not open book + AI use exams, because that's what students will have in their careers?
Tools allow traversal of poorly understood, but recognized, subskills in a way that will make one effective in their job. An understanding of the entire stack of knowledge for every skill needed is an academic requirement born out of a lack of real world employment experience. For example, I don't need to know how LLMs work to use them effectively in my job or hobby.
We should stop spending so much time teaching kids crap that will ONLY satisfy tests and teachers but has a much reduced usefulness once they leave school.
I never need to "fall back" to the principles of multiplication. Multiplying by the 1s column, then the 10s, then the 100s feels more like a mental math trick (like the digits of multiples of 9 adding to 9) than a real foundational concept.
We know, because we taught computers how to do both. The first long multiplication algorithm was written for the Colossus about 10 minutes after they got it working.
The first computer algebra system that could manage variable substitution had to wait for Lisp to be invented 10 years later.
https://www.sigcis.org/files/Haigh%20-%20Colossus%20and%20th...
The limitation seems to have been physical rather than logical.
Oxford and Cambridge have a "tutorial" system that is a lot closer to what I would choose in an ideal world. You write an essay at home, over the course of a week, but then you have to read it to your professor, one on one, and they interrupt you as you go, asking clarifying questions, giving suggestions, etc. (This at least is how it worked for history tutorials when I was a visiting student at an Oxford college back in 2004-5 - not sure if it's still like that). It was by far the best education I ever had because you could get realtime expert feedback on your writing in an iterative process. And it is basically AI proof, because the moment they start getting quizzed on their thinking behind a sentence or claim in an essay, anyone who used ChatGPT to write it for them will be outed.
If they are trade schools, yes teach React and Node using LLMs (or whatever the enabling tools of the day are) and get on with it.
And the library, and inter-library loan (in my case), and talking to a professor with a draft...
And it did teach and evaluate skills I’ve used me entire career.
For that, the student must have internalized certain concepts, ideas, connections. This is what has to be tested in a connectivity-free environment.
Faking intelligence with AI only works in an online-exclusive modality, and there’s a lot of real world circumstances where being able to speak, reason, and interpret on the fly without resorting to a handheld teleprompter is necessary if you want to be viewed positively. I think a lot of people are going to be enraged when they discover that dependency on AI is unattractive once AI is universally accessible. “But I benefited from that advantage! How dare they hold that against me!”
I get the same "you won't always have a calculator with you" vibes from 90s teachers chiding you to show your work when I hear people say stuff like this.
Plus all about capability to actually retain whatever you ask from the model...
It's more likely I will not have paper and writing implements than not having a calculator.
Besides, most people have room for fast arithmetic or integrals; fast arithmetic would be more useful, but I'm not putting the time in to get it back.
Challenge accepted. One possible solution: https://github.com/RonSijm/ButtFish
Making them open book + AI would just mean you need “larger” questions to be as effective a test, so you’re adding work for the graders for basically no reason.
Because I was demonstrating that I understood the material intrinsically, not just knew how to use tools to answer it.
And, as someone who got paid minimum wage to proctor tests in college, I couldn't keep a straight face at this:
> The most cutting-edge educational technology of the future might very well be a stripped-down computer lab, located in a welcoming campus library, where students can complete assignments, by hand or on machines free of access to AI and the internet, in the presence of caring human proctors.
I think the author's leaning heavily on vibes to do the convincing here.
I have no idea what you're trying to express in your comment, so who's using vibes?*
Were you triggered by the word "caring?" A waiter usually cares that the people they're serving have an enjoyable meal. It doesn't mean that they love them, it means that they think the work of feeding people is purposeful and honest (and theirs.)
-----
[*] It's certainly not in the words; I don't know what made you angry about "joy," I don't know why you think the author does not teach writing skills in "communications," I don't know why the fact that you went through multiple drafts in writing school papers is relevant or different than anyone else's experience. Maybe that's over now. Maybe I actually don't care if you use AI for your second and further drafts, if I know you can write a first draft.
Drafting and redrafting a cumulative course paper, as well as iteratively honing a thesis, is a writing skill.
I would argue it as important than demonstrating recall and interconnection of the material. It is being lost if long-term work on papers is being replaced with 3-hour blue-book essays.
That is why I thought it was relevant. That's it.
I guess what I'm asking is, how did AI shift the status quo for in class exams over, say, Safari?
A common mode I have seen is phone in lap, front-facing camera ingests an exam page hung over the edge of the desk. Student then flips the page and looks down for the answer.
Administrations won't allow it because they just don't care enough. It's a pain dealing with complaining parents and students.
In any case, cheating has existed since forever. There is nothing much new about cheating in in-class exams now with AI than before without.
AI is transformative here, in toto, in the total effect on cheating, because its the first time you can feasibly "transfer" the question in with a couple muscle-memory taps. I'm no expert, but I assume there's a substantive difference between IDing someone doing 200 thumb taps for ~40 word question versus 2.
(part I was missing personally was that they can easily have a second phone. same principle as my little bathroom-break-cheatsheet in 2005 - can't find what's undeclared, they're going to be averse to patting kids down)
Currently, it's been a place for acquiring skills but also a sorting mechanism for us to know who the "best" are... I think we've put too much focus on the sorting mechanism aspect, enticing many to cheat without thinking about the fact that in doing so they shortchange themselves of actual skills.
I feel like some of the language here ("securing assessments in response to AI") really feels like they're worried more about sorting than the fact that the kids won't be developing critical thinking skills if they skip that step.
Maybe we can have
When I started out (and the original Van Halen was still together), blue book exams were the norm in humanities classes. I've had narrow experience with American undergrad classes the past 25 years, so I don't have a feeling for how things have evolved.
Why replace a system that generally works well with one that introduces additional potential problems?
Online instruction / learning can work for some people, and that's good.
I don't understand how anyone ever thought that an online exam could be made secure. There's just no way to ensure that the person who registered for the course is the one taking the exam when you don't control anything about the hardware or location, when students have a wide variety of hardware that you must support, and any attempt at remove video monitoring of the exam immediately runs into scalability and privacy issues. Like, even if you're watching a video of the person taking the online exam, how do they prove that they didn't just hook up an extra keyboard, mouse and (mirrored) monitor for person #2 to take the exam for them while they do their best to type and/or mouse in a convincing way?
It also doesn't help that you periodically get students who will try to wheedle, whinge, and weasel their way into an online exam, but then bomb the in-person exam (it's so strange and totally unrelated that they really, really wanted to take an online exam instead of in-person!).
Ok, I'll stop ranting now :)
That being said, the whole experience had an impact on my generally optimistic view of human nature.
There's a whole system for this, it already works very well if people actually wanted to make online exams work. Of course it's not "social distancing" so it didn't help with covid.
Our interview usually starts with them breathlessly reading from a script out of the corner of their eye. I'm ok with notes to make sure you hit some high points about yourself even in person. Nervousness shouldn't disqualify a talented person. But with the coding part I've gotten exasperated and started asking these senior candidates to share their screen and do a fizz buzz exercise live in a text editor in the first few minutes. If they struggle I politely end the interview on the 15.
One candidate cheated and it was interesting to watch. In the time I sent the message in Zoom and them sharing their screen, just a few seconds, they had either queried or LLM-ed it on their phone or another computer, had someone off screen or in the same room listening and sharing the answer on another monitor or something else. Whatever it was they turned their head slightly to the side, squinted a bit and typed the answer in Java. A few syncopated characters at a time. When asked what modulo was they didn't know and couldn't make any changes to it. It was wacky. In retrospect I think it was them reading the question out loud to an LLM.
I'm waiting for the candidate who has someone behind them with the same shirt on pretending to be their arms.
These are the absolute worst.
You're taking people out of their comfort zone (highly customized IDE like JetBrains / VSCode / Vim) which cause them to lose shortcuts and decently working intellisense. Yes, my TypeScript in my projects is configured in such a way that I get way more information from the compiler than the standard config. After all, you're testing my ability as a software engineer, not a code monkey, right?
In this very uncomfortable place there is no way of asking questions. Yes, sometimes stuff is ambiguous. I rather have someone who asks questions vs someone who guesses and gets it right.
The testing setup is horrible too. No feedback as to what part of the tests fail, just... fail.
No debugger. No way of adding log messages. When was the last time you've been in that situation at your workplace?
All under the pressure of time, and additional stress from the person that they really NEED a new job.
Oh, and when you use compiled languages, they're way slower than say TypeScript due to the compilation phase.
And then even when your score (comprised of x passed tests and y failed tests) is of passing grade there is a manager out there looking at how many times someone tabbed outside of the window/tab?
Where am I supposed to look up stuff? Do you know all of this information by heart: https://doc.rust-lang.org/std/collections/struct.BTreeMap.ht...
Which reminded me that one time I used a function recently stabilized, but the Rust version used was about 8 versions behind. With that slow compilation cycle.
/sigh.
Everyone says this over the years, even before AI, and I've never felt it made the slightest difference in how they rate me.
Grading a stack of blue books is a "just kill me now" brutal experience. A majority of cognitive effort is just finding the page for the next problem to grade, with finding the answer another delay; any programming language with this kind of access latency would stay a "one bus" language. So of course professors rely on disinterested grad students to help grade. They'll make one pass through the stack, getting the hang of the problem and refining the point system about twenty blue books in, but never going back.
With stapled exams one problem per page one can instead sort into piles for scores 0-6 (if you think your workplace is petty, try to imagine a sincere conversation about whether an answer is worth 14 or 15 points out of 20), and it's easy to review piles.
When I had 200 linear algebra exams to grade at once, I'd scan everything, and use my own software to mark and bin one question at a time, making review passes a pleasure. I could grade a 10 question final in one intense sitting, with far more confidence in the results than team grading ever gave me.
Also, second stapled exams with prescribed spaces for answers. So much time wasted just looking for where the answer is.
It’s still a hard problem though. If the students have the laptops outside of the testing site, they can load cheating materials on them, or use them to smuggle questions and answers out if the test is used in multiple class sections. You realistically will not lock down a laptop students can take home sufficiently that some people won’t tamper with it.
Otherwise you have to have enough laptops to get each student a wiped and working machine for every test, even with lots of tests going on. And students need to be able to plug them in unless the batteries are rigorously tested and charged, but not every classroom has enough outlets. And you need to shuttle the laptops around campus.
Then you need a way to get the student work off the laptops. You probably want the student to bring the laptop up when done and plug in a USB printer. Anything else, like removable media, and you have to worry about data loss or corruption, including deliberate manipulation by students not doing well on the exam, and students claiming what got graded wasn’t what they meant to hand in. And you still have to worry about students finding a way to deliberately brick their laptops and the inevitable paper jams and other hardware failures, especially an issue when students need to leave on time to get to another class.
So you need systems that are cheap but reliable, tamper-resistant but easy to diagnose and maintain, robust against accidental data loss but easy to reliably erase, able to export data and install security updates without letting students surreptitiously input data, and heavily locked down while easy to use for students with a wide variety of backgrounds, training, and physical abilities.
I've never seen anyone attempt blue books for anything else.
First, its interesting to see a situation where a new technology INCREASED the cost of something.
Grading a stack of blue books is a "just kill me now" brutal experience.
When I had 200 linear algebra exams to grade at once
We're probably not going to get that, though, and instead the cost increases, the funding stays the same, so the quality will decline to compensate.
So it goes...
Now we'll see the reverse, with students arguing that they can't handwrite effectively and need more time or whatever in order for exams to be fair. Hopefully handwritten exams will become the norm from grade school onward, so that this complaint will be rendered moot.
Given the absolutely wild increases in tuition, administrations should have massive resources to bring to bear to solving this and other problems.
The reasons they don't feel more like excuses than legitimate explanations.
https://research.collegeboard.org/media/pdf/Trends-College-P...
Published tuition has gone up, but grant aid has matched it, making the net cost at 4-year public institutions to be flat, or even down slightly over the 10 year period. The same applies at private 4-year institutions, large increase in nameplate price, matched by large increase in grant aid, actual net tuition flat.
Expenditure data also show that they are not spending significantly more. See the chart at the end of this page, which gives expenditures in real, per-student dollars. They are up, a little less than 20% over 10 years, but half of that is increases in hospital costs of their associated health care systems, which have nothing to do with tuition. The rest is a mix of various activities of universities, much of which are not tuition-funded.
https://nces.ed.gov/programs/digest/d23/tables/dt23_334.10.a...
On the few occasions (3) I took one, I ended up "punching through" the paper every single time. I tend to write with high pressure and the paper quality is atrocious. Twice, the tares were so bad the book was partially de-bound.
On both occasions, when presented with a "torn/destroyed book" I had to show the proctor the "issues" and then, very carefully, hand-copy over everything into a new book in their presence--absolute PITA.
sarchertech•6h ago
When I took physics we had weekly 3.5 hour lab sections. That should be enough for most CS assignments.
SoftTalker•6h ago
girvo•4h ago