Microsoft has introduced a new “computer use” feature in Copilot Studio, allowing AI agents to interact with websites and desktop applications. This enables businesses to build AI agents that can automate tasks by utilising websites and apps as tools, similar to features offered by OpenAI and Claude.
“Computer use enables agents to interact with websites and desktop apps by clicking buttons, selecting menus, and typing into fields on the screen,” explains Charles Lamanna, corporate vice president of Microsoft’s business & industry Copilot. “This allows agents to handle tasks even when there is no API available to connect to the system directly. If a person can use the app, the agent can too.”
“With computer use in Copilot Studio, makers can build agents that automate tasks on user interfaces across both desktop and browser applications, including Edge, Chrome, and Firefox. Additionally, computer use runs on Microsoft-hosted infrastructure, meaning organisations don’t need to manage their own servers,” he further wrote. “Enterprise data stays within Microsoft Cloud boundaries and is not used to train the Frontier model. This helps your organisation accelerate deployment, reduce maintenance, and lower infrastructure costs.”
The new “computer use” feature enables AI agents to interact with websites and desktop applications, automating tasks such as:
- Data Entry: Inputting large volumes of data from various sources into a centralised system, reducing manual effort and minimising errors.
- Market Research: Automating the collection of market data from online sources for analysis, providing valuable insights without manual intervention.
- Invoice Processing: Extracting data from invoices and inputting it into accounting systems, streamlining the invoicing process and reducing manual errors.
