> If your chatbot decide to tell your customer to kill themselves, it's your problem.
I don't think the argument is that the AI made it OK as much as if someone commits suicide because a chatbot told them to they were so delicate that it isn't the fault of the system they were interacting with. It may be the straw that broke the camel's back, but it was still only a straw-worth of harm. It'd be like a checkout person telling a customer to kill themselves and the person commits suicide later that night - an unprofessional act to be sure, the server would probably get sacked on reflex but we really can't say anyone should be held legally liable.
I think most non-lawyers agree with this and consider it common sense. But, the US legal system explicitly disagrees.[0] My understanding is that it does affect the damages though, meaning the damages scale down based on the level of pre-existing frailty.
This is an issue of the law struggling to catch up to technology yet again. I think the only novel thing here is applying eggshell to psycological frailty. But that may just be because I'm not a lawyer and not aware of cases around this that might exist (beyond the extreme bullying cases like [1]).
All this seems to do is say you can't use "the AI model did that, not me" as a defense to escape damages in a civil suit, it doesn't change the extent of encouraging suicide that someone could be liable for.
The AI is employing the persuasive skills or learned directly from some fucko suicide cult leaders to purposelly talk you into and through doing it. That doesn't seem NEARLY the same in a practical or legal sense.
I suppose Jack in the Box should not be liable for an E. Coli outbreak? Not sure why AI companies (or third party developers who aren't being especially careful in how they use these models) deserve a special exception for selling sausage made from unsanitary sources.
I don't think the argument is that the AI made it OK as much as if someone commits suicide because a chatbot told them to they were so delicate that it isn't the fault of the system they were interacting with. It may be the straw that broke the camel's back, but it was still only a straw-worth of harm. It'd be like a checkout person telling a customer to kill themselves and the person commits suicide later that night - an unprofessional act to be sure, the server would probably get sacked on reflex but we really can't say anyone should be held legally liable.