Fix allowed-tools format and add content-desc search fallback

This commit is contained in:
Patrick Honkonen
2026-03-31 14:20:10 -04:00
parent 9ee824bb47
commit 3771fb18cd
2 changed files with 12 additions and 8 deletions

View File

@@ -1,13 +1,7 @@
---
name: interacting-with-android-device
description: Instructions for capturing UI state, comparing with mocks, and interacting with an Android device using universal ADB commands.
allowed-tools:
- Bash(adb *)
- Bash(./.claude/skills/interacting-with-android-device/scripts/adb-*)
- Bash(sleep *)
- Bash(./gradlew install*)
- Read
- Glob
allowed-tools: Bash(adb:*), Bash(.claude/skills/interacting-with-android-device/scripts/adb-capture.sh:*), Bash(.claude/skills/interacting-with-android-device/scripts/adb-find-element.sh:*), Bash(.claude/skills/interacting-with-android-device/scripts/adb-tap-element.sh:*), Bash(.claude/skills/interacting-with-android-device/scripts/adb-tap-and-capture.sh:*), Bash(.claude/skills/interacting-with-android-device/scripts/adb-navigate.sh:*), Bash(sleep:*), Bash(./gradlew install*:*), Read, Glob
---
# Interacting with Android Device
@@ -18,7 +12,7 @@ Helper scripts in the `.claude/skills/interacting-with-android-device/scripts/`
**Available scripts:**
- `adb-capture.sh [--xml] [--screenshot] [--all]` - Capture current device state. Default (no flags): both screenshot and XML hierarchy.
- `adb-find-element.sh <text>` - Find element by text, return center coordinates (`X Y`). Dumps UI hierarchy, parses XML, calculates center from bounds.
- `adb-find-element.sh <text>` - Find element by `text` or `content-desc`, return center coordinates (`X Y`). Dumps UI hierarchy, parses XML, calculates center from bounds.
- `adb-tap-and-capture.sh <x> <y> [wait_seconds=2]` - Tap at coordinates, wait, capture and pull screenshot.
- `adb-tap-element.sh <text> [wait_seconds=2]` - Find, tap, and capture in one command (recommended). Combines `adb-find-element.sh` + `adb-tap-and-capture.sh`.
- `adb-navigate.sh <home|back|app-drawer> [wait_seconds=1]` - Navigation actions via keyevent or swipe, then capture screenshot.

View File

@@ -29,12 +29,22 @@ $ADB shell uiautomator dump /sdcard/view.xml > /dev/null 2>&1 && $ADB pull /sdca
echo "UI hierarchy saved to: $(pwd)/view.xml" >&2
# Extract coordinates from XML using grep + awk
# Search by text attribute first
MATCH=$(grep -o "text=\"[^\"]*${SEARCH_TEXT}[^\"]*\"[^>]*bounds=\"\[[0-9,]*\]\[[0-9,]*\]\"" view.xml | head -1)
if [ -z "$MATCH" ]; then
MATCH=$(grep -o "bounds=\"\[[0-9,]*\]\[[0-9,]*\]\"[^>]*text=\"[^\"]*${SEARCH_TEXT}[^\"]*\"" view.xml | head -1)
fi
# Fallback: search by content-desc attribute (common in Compose UIs)
if [ -z "$MATCH" ]; then
MATCH=$(grep -o "content-desc=\"[^\"]*${SEARCH_TEXT}[^\"]*\"[^>]*bounds=\"\[[0-9,]*\]\[[0-9,]*\]\"" view.xml | head -1)
fi
if [ -z "$MATCH" ]; then
MATCH=$(grep -o "bounds=\"\[[0-9,]*\]\[[0-9,]*\]\"[^>]*content-desc=\"[^\"]*${SEARCH_TEXT}[^\"]*\"" view.xml | head -1)
fi
if [ -z "$MATCH" ]; then
echo "ERROR: Element with text '$SEARCH_TEXT' not found" >&2
exit 1