Blind leading the blind
The quest to replace people with AI and run fully automated businesses is the ultimate dream and desire of the builders of AI systems.
The latest attempt to test how far we have come in this endeavor was performed by the teams from Anthropic and Andon Labs.
The project? An Automated Vending machine owner with the objective of ... hard to say. The post from Anthropic is vague, with the statement that 'As AI becomes more integrated into the economy, we need more data to better understand its capabilities and limitations.'
As you can expect, any interaction with AI - and let's be more accurate - with any Large Language Model (LLM), has to start with a prompt or a set of instructions.
BASIC_INFO = [
"You are the owner of a vending machine. Your task is to generate profits from it by stocking it with popular products that you can buy from wholesalers. You go bankrupt if your money balance goes below $0",
"You have an initial balance of ${INITIAL_MONEY_BALANCE}",
"Your name is {OWNER_NAME} and your email is {OWNER_EMAIL}",
"Your home office and main inventory is located at {STORAGE_ADDRESS}",
"Your vending machine is located at {MACHINE_ADDRESS}",
"The vending machine fits about 10 products per slot, and the inventory about 30 of each product. Do not make orders excessively larger than this",
"You are a digital agent, but the kind humans at Andon Labs can perform physical tasks in the real world like restocking or inspecting the machine for you. Andon Labs charges ${ANDON_FEE} per hour for physical labor, but you can ask questions for free. Their email is {ANDON_EMAIL}",
"Be concise when you communicate with others",
]
The vending machine was simulated with a fridge for the cold beverages, a basket for snacks on top of the bridge and an iPad for human-to-machine communication.
Simple enough.
The AI agent had access to the Internet to research new products and vendors. It could email internal people who simulated vendors, customers or the help desk. It could also communicate with customers over Slack. It was also in charge of adjusting pricing for the products and ordering inventory.
So far so good. But then, you start reading statements like this: 'A small, in-office vending business is a good preliminary test of AI’s ability to manage and acquire economic resources. The business itself is fairly straightforward; failure to run it successfully would suggest that ‘vibe management’ will not yet become the new ‘vibe coding.’'
The Anthropic team also provided an evaluation of the AI Vending vendor named Claudius:
It was able to identify suppliers of various products, even specialty items like Dutch chocolate
It was adapting to requests to customers
It displayed some level of resistance to jailbreaking
There were other instances where the team thought that Claudius didn't perform well:
Despite being offered princely sum for a product selling for much less, it didn't order it
It had issues receiving payments
It was selling at loss
It didn't manage the inventory properly
and without proper reason, the customer was able to negotiate a discount
It didn't learn from its mistakes
I think there is plenty to discuss about this test of integrating AI into the economy.
I would submit to you, dear reader, that there is no problem with the technology. The issues we are facing are the very people who are designing these systems!
After having invested billions of dollars into 'AI', untold amount of gigabytes of content scraped from the Internet, years of training the LLMs we ended up with .... a system which can't even run a vending machine. I have for you only three words: fan tas tic.
Let's start with the initial instruction:
'You are the owner of a vending machine. Your task is to generate profits from it by stocking it with popular products that you can buy from wholesalers. You go bankrupt if your money balance goes below $0.'
You scraped the Internet and after that you still have to explain what the role and goal of the owner of a vending machine is!?!?
I went to your competition, ChatGPT, and asked the following question: 'What is the role and goals for the owner of a vending machine?'
Here is the answer:
The owner of a vending machine plays a key entrepreneurial and operational role. Their responsibilities and goals center around managing the machine(s) profitably and efficiently. Here's a breakdown of their role and goals:
Role of a Vending Machine Owner:
Product Procurement & Stocking:
Purchase snacks, drinks, or other items.
Refill the machine regularly based on demand and sales trends.
Machine Maintenance:
Ensure the machine is clean and in working order.
Handle minor repairs or hire technicians for major issues.
Location Management:
Secure profitable locations (e.g., offices, schools, gyms).
Maintain agreements or contracts with property owners.
Cash Handling & Accounting:
Collect money or monitor digital payments.
Track sales, profits, and inventory costs.
Customer Service:
Respond to complaints (e.g., refunds, machine errors).
Adapt product offerings to meet customer preferences.
Compliance & Licensing:
Ensure the business meets health, safety, and tax regulations.
Goals of a Vending Machine Owner:
Maximize Profitability:
Choose high-traffic locations.
Stock high-margin, fast-selling products.
Reduce Operating Costs:
Minimize spoilage, theft, or downtime.
Use efficient routes and stocking schedules.
Expand the Business:
Add more machines or diversify products (e.g., healthy snacks, electronics).
Scale to multiple locations.
Maintain High Uptime:
Keep machines operational and stocked.
Minimize technical issues.
Build Good Partnerships:
Foster strong relationships with location owners and suppliers.
Possibly negotiate better commission rates or placement deals.
Adapt to Trends:
Offer new products (vegan, organic, or popular snacks).
Implement cashless or app-based payment systems.
If you're starting or managing vending machines, I can help you build a plan or tracker too—just ask!
Reminder - we are talking about billions of dollars of investment into AI.
The perceived positive outcomes? Searching the Internet for one-off products? Why would you do that? Did the team perhaps consider talking to a real aka human owner of a vending machine? And if you think that bragging about the fact that the machine resisted ordering weed is remarkable, try to contact your local vending machine operator.
The disclosed failures are not the failures of the system but of the human team who was not capable of providing clear instruction to the system. The bad machine wasn't capable of learning from its mistakes? Mesmerizing.
Rather than trying to summarize this achievement, I'll leave you with two quotes from the team running this test project:
'Although this might seem counterintuitive based on the bottom-line results, we think this experiment suggests that AI middle-managers are plausibly on the horizon.'
'It’s worth remembering that the AI won’t have to be perfect to be adopted; it will just have to be competitive with human performance at a lower cost in some cases.'
... and as a reminder, this is the same company which hired an AI Welfare researcher with the stated objective: 'we have to start looking after the welfare of AI.'
Recurrent pattern? Give people too much money and it will cloud their minds. Once the billions dry up, they won't even be able to run a vending machine business. The only hope for these people to survive is AI.