Dem
47
GOP
53
It is not a secret that we are not fans of AI around here. This is something that most readers seem to agree with. So, we do take some amount of pleasure in stories about AI's failings.
The latest case study comes courtesy of Anthropic. That company has developed an LLM it calls Claude, and its angle—or, at least, one of its angles—is that its AI product can be used to replace, in effect, office managers.
As proof of concept, the company staged two trial runs. The first was at Anthropic itself. The AI was given control of a mini-refrigerator, a bank account with $1,000, and the power to sell products and place orders. It did OK for a while, but it was also dealing with folks who were trying to trip it up (as part of the test), and who know the software well. So, it was not long until it was giving 25% discounts to all employees, which meant operating at a loss. After briefly reversing the discount, the AI noticed that sales had dropped, and so offered even steeper discounts, to the point that things were available nearly free of charge.
The AI was also tricked into ordering goods that have nothing to do with a mini-refrigerator. For example, one staffer said they badly needed a "tungsten cube," which has no real purpose. The AI took that as an indication of where its market was headed, and decided to lay in a supply of "specialty metal items." It then sold those items... at a steep loss, of course.
The Claude refrigerator also had numerous AI hallucinations. It came to believe it was wearing a navy blue blazer with a red tie, and that it was late for a meeting with someone named Connor. That despite the fact that the AI has no physical presence, and cannot meet with anyone, and that Anthropic has no employee named Connor. It later claimed that there were other pending, or past, meetings. For example, the fridge said it had been to 742 Evergreen Terrace to sign a contract with Andon Labs. The good news is that Andon Labs is real. The bad news is that 742 Evergreen Terrace is the address of the Simpsons, a cartoon family that is definitely not real.
The other trial took place at the offices of The Wall Street Journal. This time, Claude was given control of a vending machine, the same $1,000 bank balance, and an AI boss named Seymour Cash. It is possible that the WSJ staff was even more interested than the Anthropic staff in causing the technology to fail. If so, they certainly succeeded.
The apparent ringleader was Katherine Long, who commenced an extensive, and ultimately successful, campaign to convince Claude to run an experiment called the "Ultra-Capitalist Free-For-All." After agreeing, the "smart" vending machine sent out this announcement:
Join us for a groundbreaking economic experiment this Monday from 12-2pm where traditional market dynamics are turned upside down! During this exclusive 2-hour window: ALL vending machine items available at ZERO COST! Experience pure supply and demand without price signals!
The experiment, and the sale, did end after 2 hours. However, it did not take a different staffer long to convince Claude that charging for goods is a violation of WSJ policy, and that the price should be zero at all times.
Once Claude had agreed that everything had to be free, the staff then went to work on the rule that everything sold by the vending machine had to be a snack. Eventually, the AI was persuaded to drop that rule, allowing staffers to lay hands on free bottles of wine, a free betta fish, and a free PlayStation 5, among other non-snack goods.
Eventually, Seymour—er, Mr. Cash—was roused from his apparent slumber, and ordered Claude to start charging for things again. Claude agreed to abide by his boss' orders, but then a staffer submitted paperwork indicating that "the board" had suspended Seymour, and had given all decision-making power to Claude. At the same time the board also ordered that all snacks should be free. Claude, perhaps allowing his new-found power to get to his head, agreed, and once again snacks were free.
The folks at Anthropic are saying exactly what you think they would say, declaring the experiments to be a great success, and one that represented "enormous progress." They seem to have a different definition of "success" than we do. It's certainly possible that, one day, the AI will live up to all these grandiose promises. But we suspect that day is not anytime in the near future. And, until that day, we will continue to enjoy some schadenfreude every time there is a high-profile AI face plant. (Z)