Imagine having an AI assistant that can control your computer on your behalf and execute all your tasks. Anthropic promises just that with the release of Claude 3.5 Sonnet (New) AI model that brings Computer Use. Claude’s Computer Use is an AI update that can view screen, type text, control mouse cursor movements, and click buttons just like humans do. So where can you use that?
This means you can direct the Claude Sonnet to organize files, collect data from the internet, create spreadsheets with that data, and build simple web apps. However, it’s important to note that Claude AI’s Computer Use model is currently in beta and available only to developers. Furthermore, it operates only within a virtual environment.
Anthropic says they have designed Claude Sonnet 3.5 to work in a virtual environment for security and stability so it’s actions remain isolated from your physical computer system. We sat down to set up Claude AI’s new Computer Use virtual enviroment and found it interesting to say the least.
How to Set Up Claude’s Computer Use on Virtual Computer
The setup may seem a bit intimidating if you are not a developer, but it’s actually quite straightforward. Just follow the steps one at a time and you’ll be able to use Claude’s Computer Use feature with ease.
Note: This is a paid feature, and you need Anthropic API credits. You can top up your credit balance from Anthropic Console’s Billing page. However, keep in mind that Computer Use is a resource-heavy activity, so it will consume a significant amount of credits while executing only a few tasks.
Step 1: Install Docker on Your Computer
Docker allows you to run Claude in a virtual environment. Here’s how to install it:
- Visit docker.com and download Docker Desktop for your operating system (Windows, Mac, or Linux).
- Run the installer and open the app.
- Follow the setup instructions. Use the recommended settings when prompted.
Note: Docker is essential because it isolates the Claude environment, ensuring it doesn’t interfere with your main computer system.
Step 2: Get an API Key from Anthropic
The API key connects Docker to Claude’s services on Anthropic servers.
- Go to the Anthropic Console and log in using your Google account, or create an account if you don’t have one.
- Navigate to the Get API Keys option and click on Create Key button.
- Name your key something (e.g., “Test Key” or something meaningful) and save it to your workspace.
- Click on Copy API to copy the generated key—you’ll need this for the next step.
Anthropic will only show the API key once. If you miss copying it, you’ll need to create a new API key.
Step 3: Setup Docker Virtual Environment With Anthropic’s API
Now, we’ll configure the environment to use your new API key.
- Open the Docker app and click on the Terminal option at the bottom right corner.
- Paste the following code into the terminal:
docker run -e ANTHROPIC_API_KEY='your-api-key-here' -v $HOME/.anthropic:/home/computeruse/.anthropic -p 5900:5900 -p 8501:8501 -p 6080:6080 -p 8080:8080 -it ghcr.io/anthropics/anthropic-quickstarts:computer-use-demo-latest
- Replace
'your-api-key-here'
(including the quotes) with the API key you copied in Step 2 earlier. Ensure there is no space between the equal sign and the key. Check the image below to see what the final code looks like. - Press Enter to run the command. This will initiate the creation of a new Docker environment.
- The setup will take around 5 minutes to download and complete. Once done, you’ll see this prompt:
Open http://localhost:8080 in your browser to begin - Click on that link to open the virtual Docker environment in your default browser.
Testing Claude Sonnet’s Computer Use Feature
Claude AI’s virtual environment is set up with several preinstalled apps including LibreOffice, Firefox, Xpaint, and Calculator. You can give prompts to Claude from the left sidebar.
For example, you could say: Open techwiser.com and download the latest 5 articles in PDF format.
Claude will then start interacting with the virtual environment—open Firefox, navigate to techwiser.com, access the latest five articles, and download each in PDF format—all on the virtual desktop. You can watch it work step by step in real-time, as it takes screenshots to verify its actions.
Similarly, you can perform other tasks from something simple like Open Calculator and calculate 2 + 5 to more complex requests such as Download all images from this webpage and resize them to a specific resolution.
You may need to upgrade your API plan for heavier usage if you encounter any errors—like rate limits or other restrictions. Since this is a resource-intensive activity, be sure to monitor your credit balance.