this repo has no description
1<img src="banner.png" alt="AXe" width="600"/>
2
3AXe is a comprehensive CLI tool for interacting with iOS Simulators using Apple's Accessibility APIs and HID (Human Interface Device) functionality.
4
5[](https://github.com/cameroncooke/AXe/actions/workflows/release.yml)
6[](https://opensource.org/licenses/MIT)
7
8
9
10- [Features](#features)
11 - [Touch \& Gestures](#touch--gestures)
12 - [Input \& Text](#input--text)
13 - [Hardware Buttons](#hardware-buttons)
14 - [Timing Controls](#timing-controls)
15 - [Accessibility](#accessibility)
16- [Quick Start](#quick-start)
17 - [Installation](#installation)
18 - [Install via Homebrew](#install-via-homebrew)
19 - [Build from source](#build-from-source)
20 - [Basic Usage](#basic-usage)
21- [Commands Overview](#commands-overview)
22 - [**Touch \& Gestures**](#touch--gestures-1)
23 - [**Gesture Presets**](#gesture-presets)
24 - [**Text Input**](#text-input)
25 - [**Hardware Buttons**](#hardware-buttons-1)
26 - [**Keyboard Control**](#keyboard-control)
27 - [**Video Streaming**](#video-streaming)
28 - [**Screenshot**](#screenshot)
29 - [**Accessibility \& Info**](#accessibility--info)
30- [Architecture](#architecture)
31 - [Why AXe?](#why-axe)
32- [Gesture Presets Reference](#gesture-presets-reference)
33- [Contributing](#contributing)
34- [Licence](#licence)
35
36
37## Features
38
39AXe provides complete iOS Simulator automation capabilities:
40
41### Touch & Gestures
42- **Tap**: Precise touch events at specific coordinates with timing controls
43- **Swipe**: Multi-touch gestures with configurable duration and delta
44- **Touch Control**: Low-level touch down/up events for advanced gesture control
45- **Gesture Presets**: Common gesture patterns (scroll-up, scroll-down, scroll-left, scroll-right, edge swipes)
46
47### Input & Text
48- **Text Input**: Comprehensive text typing with automatic shift key handling
49- **Key Presses**: Individual key presses by HID keycode
50- **Key Sequences**: Multi-key sequences with timing control
51- **Multiple Input Methods**: Direct text, stdin, or file input
52
53### Hardware Buttons
54- **Home Button**: iOS home button simulation
55- **Lock/Power Button**: Power button with duration control
56- **Side Button**: iPhone X+ side button
57- **Siri Button**: Siri activation button
58- **Apple Pay Button**: Apple Pay button simulation
59
60### Timing Controls
61- **Pre/Post Delays**: Configurable delays before and after actions
62- **Duration Control**: Precise timing for gestures and button presses
63- **Sequence Timing**: Custom delays between key sequences
64- **Complex Automation**: Multi-step workflows with precise timing
65
66### Video & Screenshots
67- **Screenshot Capture**: Capture simulator display as PNG with automatic or custom filenames
68- **Screenshot-based Streaming**: Capture simulator video at 1-30 FPS
69- **Multiple Output Formats**: MJPEG, raw JPEG, ffmpeg-compatible, BGRA
70- **H.264 Recording**: Use the `record-video` command to write MP4 files with hardware-friendly encoding
71- **Configurable Quality**: Adjust JPEG quality and scale factor
72- **Real-time Performance**: Efficient frame timing for smooth playback
73
74### Accessibility
75- **UI Description**: Extract accessibility information from any point or full screen
76- **Simulator Management**: List available simulators
77
78## Quick Start
79
80### Installation
81
82#### Install via Homebrew
83
84```bash
85# Install via Homebrew
86brew tap cameroncooke/axe
87brew install axe
88
89# Or single liner
90brew install cameroncooke/axe/axe
91
92# Use directly
93axe --help
94```
95
96> [!NOTE]
97> You may see errors when using `brew install`, like `Failed to fix install linkage`; these can be safely ignored. This is because the binaries are already codesigned and Homebrew is attempting to resign them, which is not necessary.
98
99#### Build from source
100
101For development work:
102
103```bash
104# Clone the repository
105git clone https://github.com/cameroncooke/AXe.git
106cd AXe
107
108# Run directly with Swift (frameworks handled automatically)
109swift run axe --help
110swift run axe list-simulators
111
112# Build for development
113swift build
114.build/debug/axe --help
115```
116
117### Basic Usage
118
119```bash
120# List available simulators
121axe list-simulators
122
123# Get simulator UDID
124UDID="B34FF305-5EA8-412B-943F-1D0371CA17FF"
125
126# Basic interactions
127axe tap -x 100 -y 200 --udid $UDID
128axe tap --id "Safari" --udid $UDID
129axe tap --label "Safari" --udid $UDID
130axe type 'Hello World!' --udid $UDID
131axe swipe --start-x 100 --start-y 300 --end-x 300 --end-y 100 --udid $UDID
132axe button home --udid $UDID
133
134# Gesture presets
135axe gesture scroll-up --udid $UDID
136axe gesture swipe-from-left-edge --udid $UDID
137
138# With timing controls (NEW!)
139axe tap -x 100 -y 200 --pre-delay 1.0 --post-delay 0.5 --udid $UDID
140axe gesture scroll-down --pre-delay 0.5 --post-delay 1.0 --udid $UDID
141```
142
143## Commands Overview
144
145### **Touch & Gestures**
146
147```bash
148# Tap at coordinates
149axe tap -x 100 -y 200 --udid SIMULATOR_UDID
150axe tap -x 100 -y 200 --pre-delay 1.0 --post-delay 0.5 --udid SIMULATOR_UDID
151
152# Tap by accessibility element (uses describe-ui accessibility tree)
153axe tap --id "Safari" --udid SIMULATOR_UDID
154axe tap --label "Safari" --udid SIMULATOR_UDID
155
156# Swipe gestures
157axe swipe --start-x 100 --start-y 300 --end-x 300 --end-y 100 --udid SIMULATOR_UDID
158axe swipe --start-x 50 --start-y 500 --end-x 350 --end-y 500 --duration 2.0 --delta 25 --udid SIMULATOR_UDID
159
160# Advanced touch control
161axe touch -x 150 -y 250 --down --udid SIMULATOR_UDID
162axe touch -x 150 -y 250 --up --udid SIMULATOR_UDID
163axe touch -x 150 -y 250 --down --up --delay 1.0 --udid SIMULATOR_UDID
164```
165
166### **Gesture Presets**
167
168```bash
169# Scrolling gestures
170axe gesture scroll-up --udid SIMULATOR_UDID
171axe gesture scroll-down --udid SIMULATOR_UDID
172axe gesture scroll-left --udid SIMULATOR_UDID
173axe gesture scroll-right --udid SIMULATOR_UDID
174
175# Navigation gestures
176axe gesture swipe-from-left-edge --udid SIMULATOR_UDID
177axe gesture swipe-from-right-edge --udid SIMULATOR_UDID
178axe gesture swipe-from-top-edge --udid SIMULATOR_UDID
179axe gesture swipe-from-bottom-edge --udid SIMULATOR_UDID
180
181# With custom screen dimensions
182axe gesture scroll-up --screen-width 430 --screen-height 932 --udid SIMULATOR_UDID
183
184# With timing controls
185axe gesture scroll-down --pre-delay 1.0 --post-delay 0.5 --udid SIMULATOR_UDID
186```
187
188### **Text Input**
189
190```bash
191# Simple text input (use single quotes for special characters)
192axe type 'Hello World!' --udid SIMULATOR_UDID
193
194# From stdin (best for automation)
195echo "Complex text" | axe type --stdin --udid SIMULATOR_UDID
196
197# From file
198axe type --file input.txt --udid SIMULATOR_UDID
199```
200
201### **Hardware Buttons**
202
203```bash
204# Available buttons: home, lock, side-button, siri, apple-pay
205axe button home --udid SIMULATOR_UDID
206axe button lock --duration 2.0 --udid SIMULATOR_UDID
207axe button siri --udid SIMULATOR_UDID
208```
209
210### **Keyboard Control**
211
212```bash
213# Individual key presses (by HID keycode)
214axe key 40 --udid SIMULATOR_UDID # Enter key
215axe key 42 --duration 1.0 --udid SIMULATOR_UDID # Hold Backspace
216
217# Key sequences
218axe key-sequence --keycodes 11,8,15,15,18 --udid SIMULATOR_UDID # Type "hello"
219```
220
221### **Video Streaming**
222
223```bash
224# Stream MJPEG frames over stdout (default format)
225axe stream-video --udid SIMULATOR_UDID --fps 10 --format mjpeg > stream.mjpeg
226
227# Pipe JPEG frames directly into ffmpeg
228axe stream-video --udid SIMULATOR_UDID --fps 30 --format ffmpeg | \
229 ffmpeg -f image2pipe -framerate 30 -i - -c:v libx264 -preset ultrafast output.mp4
230
231# Stream raw JPEG frames with length prefixes for custom servers
232axe stream-video --udid SIMULATOR_UDID --fps 12 --format raw | custom-stream-consumer
233
234# Legacy BGRA stream for backward compatibility
235axe stream-video --udid SIMULATOR_UDID --format bgra | \
236 ffmpeg -f rawvideo -pixel_format bgra -video_size 393x852 -i - output.mp4
237
238# Record directly to MP4 using the dedicated recorder
239axe record-video --udid SIMULATOR_UDID --fps 15 --output recording.mp4
240```
241
242### **Video Recording**
243
244```bash
245# Record the simulator to an MP4 file (QuickTime compatible)
246axe record-video --udid SIMULATOR_UDID --fps 15 --output recording.mp4
247
248# Let AXe pick a timestamped filename in the current directory
249axe record-video --udid SIMULATOR_UDID --fps 20
250
251# Tweak quality/scale to reduce file size
252axe record-video --udid SIMULATOR_UDID --fps 10 --quality 60 --scale 0.5 --output low-bandwidth.mp4
253```
254
255> [!TIP]
256> Press `Ctrl+C` to stop recording. AXe finalises the MP4 before exiting and prints the file path to stdout.
257
258### **Screenshot**
259
260```bash
261# Capture screenshot with auto-generated filename
262axe screenshot --udid SIMULATOR_UDID
263
264# Save to specific file
265axe screenshot --output ~/Desktop/my-screenshot.png --udid SIMULATOR_UDID
266
267# Save to directory (auto-generates timestamped filename)
268axe screenshot --output ~/Desktop/ --udid SIMULATOR_UDID
269```
270
271> [!TIP]
272> The screenshot path is printed to stdout for easy scripting. Progress messages go to stderr.
273
274### **Accessibility & Info**
275
276```bash
277# Get accessibility information
278axe describe-ui --udid SIMULATOR_UDID # Full screen
279axe describe-ui --point 100,200 --udid SIMULATOR_UDID # Specific point
280
281# List simulators
282axe list-simulators
283```
284
285## Architecture
286
287### Why AXe?
288
289AXe directly utilises the lower-level frameworks provided by [idb](https://github.com/facebook/idb), Facebook's open-source suite for automating iOS Simulators and Devices.
290
291While `idb` offers a powerful client/server architecture and a broad set of device automation features via an RPC protocol, **AXe takes a different approach**:
292
293- **Single Binary:** AXe is distributed as a single, standalone CLI tool—no server or client setup required.
294- **Focused Scope:** AXe is purpose-built for UI automation, streamlining accessibility testing and automation tasks.
295- **Simple Integration:** With no external dependencies or daemons, AXe can be easily scripted and embedded into other tools or systems running on the same host.
296- **Complete HID Coverage:** Full feature parity with idb's HID functionality plus gesture presets and timing controls.
297- **Intelligent Automation:** Built-in gesture presets and coordinate helpers for common use cases.
298
299This makes AXe a lightweight and easily adoptable alternative for projects that need direct, scriptable access to Simulator automation.
300
301## Gesture Presets Reference
302
303| Preset | Description | Use Case |
304|--------|-------------|----------|
305| `scroll-up` | Scroll up in center of screen | Content navigation |
306| `scroll-down` | Scroll down in center of screen | Content navigation |
307| `scroll-left` | Scroll left in center of screen | Horizontal scrolling |
308| `scroll-right` | Scroll right in center of screen | Horizontal scrolling |
309| `swipe-from-left-edge` | Left edge to right edge | Back navigation |
310| `swipe-from-right-edge` | Right edge to left edge | Forward navigation |
311| `swipe-from-top-edge` | Top to bottom | Dismiss/close |
312| `swipe-from-bottom-edge` | Bottom to top | Open/reveal |
313
314## Contributing
315
316Contributions are welcome! Please feel free to submit a Pull Request.
317
318## Licence
319
320This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.