You can't rely on asking the customer. When they're upset (they often are in these calls), they'll lean towards the negative regardless.
I don't know how well these AIs evaluate, but if they're even a little bit good, it makes sense to use it to screen for outliers, then have a human listen to those outliers and judge.
Patient or customer? I even struggle with that, but I guess that’s what people are in a privatised healthcare system.
If you outsource that work to customers/patients, you'll end up with the car dealership model, where the sales rep begs you to give a 10 on every single question including on the interior design so they don't get fired.
That's the part most of this discussion misses. Supervisors exist for a reason. Congrats on your flat org structure, you fucked up an important feedback channel.
It’s somewhat the point of democracy to maintain a limit on rent seekers’ and wealth extractors’ power on the political process. It should come as no surprise that the individuals who have grifted and extorted their way into power are also fiercely antidemocratic. The xenophobes and bigots that have hitched their wagons are equally deplorable.
> A company spokesperson said, "Kaiser Permanente does not use Average Handle Time to assess agent performance"
So uh, average time wasn't raised as a concern, calls beyond a certain threshold was. I wish this semantic discrepancy was better highlighted in the article.
They claim they do not use average handle time, but it is very common to get called into meetings to discuss why they spent a lot of time on some calls. The nurses get defensive (by definition - they have to justify the time used - it is a defense).
They also do get called into meetings if their average handle time is large.
It may still be true they don't use it for evaluating performance, but they absolutely do utilize it to "coach" the nurses.
We're liberating them from work so they can focus on what matters
Also, California has a high cost of living.
> Another nurse speaking on condition of anonymity said “AI did not understand our job and would grade us wrong all the time.”
It's always worth remembering Goodhart's law https://en.wikipedia.org/wiki/Goodhart%27s_law - "When a measure becomes a target, it ceases to be a good measure."
In theory AI could usher in the first time in history where one can escape from this trap - because qualitative judgments can be made at scale, from an unbiased and universal baseline. In this situation, for instance, rather than collapsing call transcripts and reports into metrics, it could evaluate whether red flags are encountered in the context of a call, and allow for qualitative guidance on improvement, across a comparative corpus of situations that are themselves chosen qualitatively.
But very few managers are empowered to take this kind of approach; they're evaluated by their ability to report quantitative metrics, and thus they must implement regimes of quantitative metrics. And leadership instructs them to use AI to build that regime more quickly.
If you want to see an "AI native" organization, it's one where leadership actively fights this tendency, and sees managers as product designers who make the end-user experience a beloved and empathy-driven one, as opposed to a gear that turns accountability into a single number on a screen.
So stupid. If you had ever made a phone call to a patient, or their family member, you’d soon realise how bad this is.
You need to talk to the patient and something a family member too. Be too hasty and you cause more harm than good.
About 14% of Americans think AI is moving us towards a better world. About 17% are creationists. About 26% believe in Telekinesis.
It’s going to be very improbable that these statements are true.
Where and how is that determined? I.e., any references to back that up?
And you can't in one breath say they have the best healthcare but then say their employees' reports of their experience are unreliable.
https://pmc.ncbi.nlm.nih.gov/articles/PMC8032167/
Their secret sauce is their ability to standardize protocols throughout their entire organization.
Their goal isn't to provide high quality care. Their goal is to increase profits. It's not hard to imagine how improved quality would lead them to spend more money. (faster diagnoses of serious illnesses and recommending expensive care)
I hate to break it to you, but "non-profit" doesn't mean what you literally think it means.
https://en.wikipedia.org/wiki/Kaiser_Permanente
Also, KPMGs are indeed "for-profit" while Kaiser Permanente as a whole is constituted as a "consortium" of both types.
And here is why: I guarantee you that the nurses are working in the "for-profit" units, and also, that still betrays ignorance of what "non-profit" actually means, which is the load-bearing topic of this thread.
Given how healthcare is one of these sectors that seems to relentlessly resist efficiency increases and is the prime example of Baumol's cost disease, I think any developed country with a costly healthcare system needs to do these AI experiments. The current versions will be shit, but the only way out is through if you still want to provide affordable care.
I honestly have no doubt that AI going forward will be able to do a good job at triaging via calls and also being empathetic about it. But of course it needs careful experimentation.
People started hating tech right around the time metrics became popular. I don't think it's a coincidence. AI just accelerates the trend.
The problem is the misidentification of AI as the issue. As long as we don't understand the real issue, we won't solve it. AI is just a tool. It's being used in a way that denies human agency.
Our cultural values need to shift away from safetism that demands centralization. And shift toward valuing human agency. That starts with talking about the core issue.
"To the man with a hammer, everything looks like a nail." [1] There is no such thing as "just" a tool.
----
Now, like many tools, the majority of those selling AI to make money off of large enterprise sell its ability to increase productivity, efficiency, compliance. Either to make money or to minimise risk. And so like you say, they just become tools to make these metrics move or report them at higher granularity. And often there is either a lack of imagination or a willful ignorance of the perverse outcomes with relationship to humans because they are in service of the organisation not it's employees.
But that same AI could cause those companies to no longer exist.
The AI I'm happy about allows people without much tech knowledge create small apps to do exactly what they want. And, for those that know just a little more, use it to help them extend open source software for their niche use case.
This makes computing more personal and gives back agency to the computer operator.
Mix that with the rise of much more competition in much more custom software, and you'll see that a future can exist, if we want it, where software becomes more valuable, personal and humane.
The software vendor will capture less value, though - the margins will be thinner. Instead that value will be captured (in non-money terms) by the end users.
That also means that software companies, unable to capture so much value, must shrink and become more boutique. The software that contributes to our centralized world would lose a lot of power.
That's the future I can see. The only way it doesn't happen is if a cynical narrative wins out and manages to lock it out through regulatory capture so that only licensed operators can use or provide AI. The anti-AI narrative helps the cynics.
They will vocally rationalize it.
I did it. "I'm more productive work from home." But then I do dishes, take an hour break, paid.
Foucault says that when people are observing them, power is placed over them.
If you are a worker you should hate this.
If you are a customer or owner, you should like this.
But I certainly won't be automatically believing people under surveillance who make claims it makes their quality worse.
Some of the tech is pretty scary. One big vendor's solution [0] can provide not just AI agents but also use AI to snoop on calls in progress, evaluating sentiment from both sides [1], verifying phrases are said - pretty dystopian in theory. From experience, these things tend to go downhill based on the attitude at the top - is the mission to slash costs or take care of customers? A 1000 decisions follow from this one, and like Jira, it can be a useful tool or a prison-like hell.
[0] https://www.cisco.com/c/dam/en/us/products/collateral/contac...
[1] https://www.cisco.com/c/en/us/products/contact-center/webex-...
If it's to do something normal you could do through the website there was no need for AI - a website or app suffices - provided it isnt terrible.
Oh yes, and the nurses did employ strategies like that pre-LLM (don't know if they still do). They had to be very strategic about it (you can't just say "Rate me a 10.")
Repeated unprofessional behavior with no discernible change after trying to address it. My take is that Kaiser nurses have a serious discipline and customer (patient) focus problem.
You can ask the customer enough times that unreasonable customers or surveys are averaged out.
A good question might be "why are you upset?"
A significant fraction of the calls they answer are patients shouting at them because of:
- Long wait times
- They don't like their doctor
- They don't like the advice they're given (sorry, but we're not going to book you as a high priority appointment if all you can tell me is you have a headache. Sorry, we're not going to prescribe a narcotic for a scraped knee.)
- Several reasons that have nothing to do with the nurse, but the customer will still blame the nurse.
How would you want yours rated? By someone you have communicated with, or some data centre somewhere?
I suppose you could do that with the survey as well. It'd be an interesting study to see which is more reliable.
Hospital systems are incentivized to avoid the real problems with healthcare. People want timeliness and they want quality care which hospital systems are not incentivized towards in the US. The incentives are profit, which given budgets means corners cut.
Triaging is an opaque system to the patient. It's an important process to doll out finite resources but it also very frustrating to be told, "soon" when you've been waiting 15 hours to see someone. Frankly, if I were King for a day, the first thing I would do is break up the monolithic hospital systems and build out more urgent care.
I would also try to find a way to facilitate transferring less critical patients from ERs to urgent care centers. Right now a hospital won't take the risk, especially if you are sitting in waiting room because beds are full. You can't easily punt a patient because them leaving would be against medical advice.
Get the doctor to assess the nurse. Or the head nurses if you don'ttrust doctors. They have managers, and if none of the doctors or head nurses can be trusted with a simple matter like assessing whether nurses are doing their jobs then you got bigger issues.
Yeah, data driven would be nice, if you have good data. But data driven is a power tool. You don't measure SLOC in software because of perverse incentives.
I highly doubt it. If this is the reason your manager knows who you are, you are absolutely going to be judged on it. It doesn't really matter what the policy says.
And the nurses absolutely feel like they are being punished for it. Just like having to consistently remind HR that your "absence problem" is due to covered FMLA leave - they know who you are because they've had to talk to you about absenteeism. In a call center of 500 people, it isn't likely they remember that you had issues because of their faulty systems.
This is true for any job. I was unfairly fired from a job because somehow the manager got a perception of me being incompetent, because he often talked to me about problems in my work - most of those conversations ended with him saying "Oh, now I see why you did it that way."
But, just purely semantically, the statement Kaiser gave in response was worded in a precise smug corporate America style to dodge the main concern raised. I think it's important to call out weasel words.
I know nurses. And I know their unions. Kaiser can be extremely clear and tell the truth, and they'll still say "We don't believe you!" without any evidence other than being called in to talk about it.
I'm not anti-unions - I've benefited from them. But it's well known that distrust goes up when you have (or need) unions. It typically degenerates from "We" to "Us vs them".
Let me ask you this: Are you saying they shouldn't monitor the time at all?
Again: A tidbit of inside information: A number of Kaiser patients get such long wait times that they're issue isn't addressed when they try to call (i.e. they are told they'll be called back, and they're called back some other day). I don't know the percentage - likely small - but from a healthcare standpoint, it's unacceptable.
Another bit of information (likely not inside): Kaiser has a serious budget problem. They already pay amongst the top salaries for nurses, and they can't simply solve the problem by hiring more.
So: How would you solve it?
If you have a choice of hospital networks, research carefully. Kaiser might be fine for many people's needs, it's not fine when it comes to intensive care.
Truth is most nurses care for people having the worst day of their lives. =3
Avicebron•1h ago
ori_b•47m ago