The fight for your desktop
The fight for your desktop and to be your ‘Big Brother’ is in full swing.
This week, Google declared its AI project, 'Jarvis,' is coming to a (Chrome) browser near you by December. It’s aptly named after Iron Man’s AI sidekick. Perhaps the name evokes the helpful nature of this new thing or suggests it will give you the superpowers of the Marvel hero.
The news follows a similar announcement from Anthropic and an even much earlier announcement from Microsoft's attempt for 'intelligence' called 'Recall'supported with Copilot (the resurrection of dead Clippy).
It would appear that all these projects follow the advice of the AI Philosopherwhen describing a new era of AI filled with 'Agentic AI' and how it 'builds upon deep learning but is characterized by greater autonomy, adaptability, and the capacity for independent decision-making and long-horizon planning.'
In order for any of these new things to work, these above-mentioned companies (and many others) need to get closer to you and see what you are doing. And when the time comes, they provide help, advice or will even do things for you. Something like Big Brother or Big Sister, perhaps better Big Sibling or BS for short.
How close do these companies need to get to you?
As close as your desktop or browser. You might remember the post, The Business Model of Scraping, which describes how AI companies crawl the web and scrape every piece of content their dirty virtual hands can get hold of.
Now, they want to apply the same thing to your computer.
Google's Jarvis will be taking frequent screenshots of what’s on your computer screen, and interpreting those images before taking actions like clicking on a button or typing into a text field.
Microsoft's Recall will, every few moments, take a screenshot of your entire desktop and store it on your local drive. Anthropic's 'computer use' — seriously, is this the best name you could come up with for your AI hero!?!?! — allows developers to look at your screen and then move your cursor, click buttons or type text.
Naturally, you must be wondering: why all this?
From the Anthropic press release - 'Asana, Canva, Cognition, DoorDash, Replit, and The Browser Company have already begun to explore these possibilities, carrying out tasks that require dozens, and sometimes even hundreds, of steps to complete.'
From Google - 'Perform tasks like purchasing products from e-commerce websites, booking tickets, filling out forms, and more.'
From Microsoft’s support page - 'Trying to remember the name of the Korean restaurant your friend Alice mentioned?'
All these companies think that they need to integrate AI into their existing products as a new feature. They claim that they want you to help with mundane, boring, repetitive tasks.
Somehow, the irony of this escapes them. They are the same vendors that created tech that is difficult to use, cumbersome and requires hundreds of steps.
To illustrate my point, here is an excerpt from Microsoft introducing Microsoft Chat '[Microsoft] Teams users globally are in three times more meetings each week than they were in 2020 ... Microsoft 365 Chat tames the complexity, eliminates the drudgery and helps you reclaim time at work.’ #chokingonirony
Also to further instill confidence in the new agentic AI era, the announcement of these new features comes with a caveat. 'At this stage, it is still experimental — at times cumbersome and error-prone.'
Here is my take. I commend all these companies for trying new things, doing research and finding new ways forward.
We now think differently about data collection and processing. We’re finding patterns which escaped us before, making tech easier to use.
The thing which these companies are neglecting — either because of their current business model or lack of vision — is the new user interface and experience.
Invading your desktop or browser is a dead end. The future is Personal Web, which will introduce a new paradigm of how we better interface and interact with the vast space we call the Web.
We should start a new pattern and say 'thank you' to the old one.