Crewly is an open source computer use agent platform: its AI agents can operate your Mac's real mouse and keyboard to work in native apps and system dialogs. While one works, a banner on your screen says which agent it is and what it's doing. Touching the mouse pauses it, one hotkey stops it, and you can watch and take over the screen from the Crewly Cloud portal or your phone.
What is computer use in Crewly?
Computer use means an agent looks at your screen and acts on it with the mouse and keyboard, the way you would. It is deliberately the last resort: Crewly agents pick the lowest surface that can do the job.
| Need | What the agent uses |
|---|---|
| Mail, Drive, Calendar, Slack, git, files | A Crewly skill or connector, with no screen at all |
| A web page, signed in as you | Crewly in Chrome, in your real Chrome |
| Native apps, system dialogs, anything else | Computer use |
Where it can, the agent reads the screen as named elements ("the Save button") rather than guessing pixel coordinates.
The three surfaces, lowest first: skills and connectors, Crewly in Chrome, computer use.
How do I know when an agent is using my computer?
A banner appears on your screen naming the agent and its goal, with Pause and Stop buttons. It appears only for actions that move something, and goes away on its own if the agent crashes.
How do I take over from a computer use agent?
Four ways, from gentlest to firmest:
- Just use your mouse or keyboard. Real input from you pauses the agent immediately. The task isn't cancelled; it waits.
- Click Pause on the banner, and Resume when you're done.
- Click Stop on the banner, or press ⌃⌥⌘. (Control-Option-Command-period) from anywhere. Stop refuses every desktop action until you clear it.
- Stop agents from your phone. The Desktop page in the Crewly Cloud portal has a Stop agents button that works from anywhere.
The four ways to take over, from gentlest to firmest.
Can I watch and control my Mac from my phone?
Yes. The Desktop page in the Crewly Cloud portal shows a live picture of the screen and lets you drive it. First turn on remote desktop at the machine itself by running crewly desktop remote on. It's off by default and can't be switched on from the portal. Then you get:
- Tap mode: tap the picture and Crewly clicks there (left, double or right click).
- Mouse mode: slide to move the cursor like a trackpad.
- Type into the focused field from a text box.
The Crewly Cloud Desktop page on a phone in Mouse mode, with a live Mac screen and a Stop agents button.
Remote clicks go through the same safety rails as an agent's.
What happens when an agent gets stuck on a login or a 2FA code?
It asks you. An agent that hits a CAPTCHA, a two-factor code or a sign-in is instructed to pause itself and request a human, with a screenshot you can open on your phone.
What safety rails does Crewly's computer use have?
Enforced in one place, for every agent and runtime:
- No typing into password fields, and credential apps are never driven.
- Destructive shortcuts are refused outright, including Quit (⌘Q), Close window (⌘W) and delete shortcuts.
- One agent at a time. A machine-wide lock keeps two agents from fighting over one mouse, and it expires if an agent crashes.
- Agents can't disarm the rails or clear a Stop from their own command line.
- Every action is logged on your Mac before it runs, with a before-and-after thumbnail for anything that changes the screen.
How do I set it up?
- Run Crewly on your Mac. See Getting started with Crewly or the download page. The code is on GitHub.
- Grant macOS permissions. Computer use needs Screen Recording and Accessibility. The first check tells you which app to grant them to.
- Give an agent a desktop task, from Slack or the dashboard. For example: "Open the exported report in Numbers and save it as a PDF to Desktop."
- Optional: to watch and drive the Mac from your phone, run
crewly desktop remote onon that Mac, then open Desktop in the portal.
What it doesn't do
- macOS only, for now. There's no Windows or Linux support today.
- Agents run with full permissions, so run Crewly in an environment you're comfortable with.
- It's the slowest, most fragile surface. Pixels and coordinates break more easily than a connector or a browser tab.
- The rails are a floor, not a guarantee. They catch passwords, credential apps and destructive shortcuts. Watch sensitive work, and use Pause freely.
FAQ
Is Crewly's computer use open source?
Yes. Computer use is part of Crewly Community Edition, which is open source (MIT), and the code, including the safety rails, is on GitHub.
Can an AI agent control my Mac or PC?
On a Mac, yes: Crewly agents can operate the mouse and keyboard, with the rails above. Windows and Linux aren't supported today; computer use is macOS-only.
Is computer use safe? What stops it from deleting things?
Destructive shortcuts such as Quit, Close window and delete are refused outright, password fields and credential apps are off-limits, and every action is logged before it runs. You can pause it by touching the mouse or stop everything with ⌃⌥⌘. The rails are a floor, not a guarantee, so watch sensitive work.
Can I control my Mac from my phone?
Yes, once you turn on remote desktop at the Mac with crewly desktop remote on. It's off by default and can only be enabled at the machine.