Automation - Coordinate-based Actions

Stability: 2 - Stable

This chapter introduces functions for clicking and swiping using screen coordinates. Some require Android 7.0+; others require root permissions.

To get the coordinates of a position to click, you can enable “Pointer location” in Developer options.

Coordinate-based scripts often have resolution issues. You can use setScreenMetrics() to automatically scale coordinates. This affects all coordinate click/press/swipe functions in this chapter. By specifying the resolution the script was designed for, Auto.js will scale coordinates on other resolutions.

You can also combine widgets with coordinates. Some widgets are not clickable (clickable is false), so you can't click them via .click(). If Android is 7.0+ or the device has root permissions, you can click them like this:

// Get the widget
var widget = id("xxx").findOne();
// Click its center
click(widget.bounds().centerX(), widget.bounds().centerY());
// If using root permissions, use Tap

setScreenMetrics(width, height)

• width {number} Screen width in pixels
• height {number} Screen height in pixels

Set the screen width/height that the script's coordinates are based on. If the device screen size differs at runtime, coordinates are scaled automatically.

For example, on a 1920×1080 device:

setScreenMetrics(1080, 1920);
click(800, 200);
longClick(300, 500);

On other devices, Auto.js will scale the coordinates so the script still works. For example, on a 540×960 screen, click(800, 200) actually clicks at (400, 100).

Touch & gesture simulation on Android 7.0+

Stability: 2 - Stable

Note: the following commands only work on Android 7.0+

click(x, y)

• x {number} X coordinate to click
• y {number} Y coordinate to click

Simulate a click at (x, y) and return whether it succeeded. The script continues only after the click finishes.

In general, a click fails only if the click process (~150ms) is interrupted by other events (e.g. user interaction).

If repeated clicks are too slow, use press() instead.

longClick(x, y)

• x {number} X coordinate to long-press
• y {number} Y coordinate to long-press

Simulate a long-press at (x, y) and return whether it succeeded. The script continues only after the long-press finishes (~600ms).

In general, a long-press fails only if it is interrupted (e.g. user interaction).

press(x, y, duration)

• x {number} X coordinate to press
• y {number} Y coordinate to press
• duration {number} Press duration in milliseconds

Simulate pressing at (x, y) and return whether it succeeded. The script continues only after the press finishes.

If the duration is too short, the system treats it as a click. If it exceeds 500ms, it is treated as a long-press.

In general, the operation fails only if it is interrupted.

Example: an auto-clicker loop:

// Loop 100 times
for(var i = 0; i < 100; i++){
  // Press at (500, 1000) for 1ms each time
  press(500, 1000, 1);
}

swipe(x1, y1, x2, y2, duration)

• x1 {number} Start X
• y1 {number} Start Y
• x2 {number} End X
• y2 {number} End Y
• duration {number} Swipe duration in milliseconds

Simulate swiping from (x1, y1) to (x2, y2) and return whether it succeeded. The script continues only after the swipe finishes.

In general, a swipe fails only if it is interrupted.

gesture(duration, [x1, y1], [x2, y2], ...)

• duration {number} Gesture duration
• [x, y] {...} A list of points describing the gesture path

Simulate a gesture. For example, gesture(1000, [0, 0], [500, 500], [500, 1000]) simulates a gesture from (0,0) → (500,500) → (500,1000) with duration 1 second.

gestures([delay1, duration1, [x1, y1], [x2, y2], ...], [delay2, duration2, [x3, y3], [x4, y4], ...], ...)

Simulate multiple gestures at the same time. Each gesture uses parameters [delay, duration, points...], where delay is how long to wait (ms) before starting the gesture, duration is the gesture duration, and points are the path coordinates. The delay parameter can be omitted and defaults to 0.

Example: pinch gesture:

gestures([0, 500, [800, 300], [500, 1000]],
         [0, 500, [300, 1500], [500, 1000]]);

Simple root-based click & swipe commands

Stability: 1 - Experimental

Note: functions in this section may change in future versions. Do not rely heavily on side effects. It is recommended to use RootAutomator instead of these touch functions.

All functions below require root and can click/swipe anywhere.

• These functions usually start with a capital letter to indicate special permissions.
• These functions return no value.
• Their execution is asynchronous and non-blocking, and timing varies by device. The script does not wait for the action to finish before continuing, so it is recommended to add appropriate sleep after each call to achieve the expected effect.

Example:

Tap(100, 100);
sleep(500);

Note that actions may not be stoppable. For example:

for(var i = 0; i < 100; i++){
  Tap(100, 100);
}

After running this code, you may find that clicks continue even after stopping the script in the task manager. Therefore, it is strongly recommended to add a delay after each action:

for(var i = 0; i < 100; i++){
  Tap(100, 100);
  sleep(500);
}

Tap(x, y)

• x {number} X coordinate to tap
• y {number} Y coordinate to tap

Tap (x, y). You can enable “Pointer location” in Developer options to determine coordinates.

Swipe(x1, y1, x2, y2, [duration])

• x1 {number} Start X
• y1 {number} Start Y
• x2 {number} End X
• y2 {number} End Y
• duration {number} Swipe duration

Swipe from (x1, y1) to (x2, y2).

RootAutomator

Stability: 2 - Stable

RootAutomator simulates touch using root permissions. It supports single-touch and multi-touch, and actions are executed without delay.

It is best to have only one RootAutomator instance in a script and ensure it exits when the script ends. You can exit it on the exit event, for example:

var ra = new RootAutomator();
events.on('exit', function(){
  ra.exit();
});
// Perform some tap actions
...

Note

RootAutomator requires root or ADB permissions. To use ADB, set shell.setDefaultOptions({ adb: true }) and authorize Shizuku.

RootAutomator has limited compatibility. Since v9.3, it is recommended to use RootAutomator2 instead.

new RootAutomator([options])

• options {object} Optional:
  • adb {boolean} Whether to use ADB permission. Default false. If true, Shizuku authorization is required.
  • inputDevice {string} Specify the input device used by RootAutomator, e.g. /dev/input/event4. If not set, it will auto-detect.

Create a RootAutomator instance.

RootAutomator.tap(x, y[, id])

• x {number} X coordinate
• y {number} Y coordinate
• id {number} Multi-touch id (optional). Default 1. You can change it via setDefaultId.

Tap at (x, y). id is an integer used for multi-touch; different ids represent different “fingers”. For example:

var ra = new RootAutomator();
// Make "finger 1" tap (100, 100)
ra.tap(100, 100, 1);
// Make "finger 2" tap (200, 200)
ra.tap(200, 200, 2);
ra.exit();

If you don't need multi-touch, omit id. Multi-touch is commonly used for gestures or games, such as simulating two-finger pinch or two-finger swipe.

In some cases, tap may not respond. You can use RootAutomator.press() instead.

RootAutomator.swipe(x1, x2, y1, y2[, duration, id])

• x1 {number} Start X
• y1 {number} Start Y
• x2 {number} End X
• y2 {number} End Y
• duration {number} Swipe duration in milliseconds, default 300
• id {number} Multi-touch id (optional), default 1

Simulate a swipe from (x1, y1) to (x2, y2) that lasts duration milliseconds.

RootAutomator.press(x, y, duration[, id])

• x {number} X coordinate
• y {number} Y coordinate
• duration {number} Press duration
• id {number} Multi-touch id (optional), default 1

Simulate pressing at (x, y) for duration milliseconds.

RootAutomator.longPress(x, y[, id])

• x {number} X coordinate
• y {number} Y coordinate
• duration {number} Press duration
• id {number} Multi-touch id (optional), default 1

Simulate a long-press at (x, y).

The above are basic touch simulation functions. For more complex gestures, use lower-level functions.

RootAutomator.touchDown(x, y[, id])

• x {number} X coordinate
• y {number} Y coordinate
• id {number} Multi-touch id (optional), default 1

Simulate finger down at (x, y).

RootAutomator.touchMove(x, y[, id])

• x {number} X coordinate
• y {number} Y coordinate
• id {number} Multi-touch id (optional), default 1

Simulate moving the finger to (x, y).

RootAutomator.touchUp([id])

• id {number} Multi-touch id (optional), default 1

Simulate lifting the finger.

RootAutomator2

RootAutomator2 simulates taps, gestures, long-presses, etc. using root or ADB permissions. Compared to accessibility-based automation, it supports multi-touch and dynamically changing gestures; compared to RootAutomator, it has better compatibility.

let screenWidth = $device.width;
let screenHeight = $device.height;

// Use root permission. You can also use ADB by specifying { adb: true }, which requires Shizuku authorization
const ra = new RootAutomator2(\{ root: true \});

// Tap at (200, 200)
ra.tap(200, 200);
sleep(1000);

// Press screen center for 500ms
ra.press(screenWidth / 2, screenHeight / 2, 500);
sleep(1000);

// Swipe from (500, 200) to (500, 1000) with duration 300ms
ra.swipe(500, 200, 500, 1000, 300);
sleep(1000);


// Two-finger pinch
// Top-left position
let p0 = {
    x: screenWidth / 6,
    y: screenHeight / 6,
};
// Bottom-right position
let p1 = {
    x: screenWidth - p0.x,
    y: screenHeight - p0.y,
}
// Press down at top-left and bottom-right simultaneously, finger ids 0 and 1
ra.touchDown([
    \{ x: p0.x, y: p0.y, id: 0 \},
    \{ x: p1.x, y: p1.y, id: 1 \},
]);

// Steps
const steps = 20;
// Offset per step
const stepX = Math.round((p1.x - p0.x) / steps) / 2;
const stepY = Math.round((p1.y - p0.y) / steps) / 2;
for (let i = 0; i < steps; i++) {
    // Move finger 0 down-right, finger 1 up-left
    ra.touchMove([
        \{ x: p0.x + stepX * i, y: p0.y + stepY * i, id: 0 \},
        \{ x: p1.x - stepX * i, y: p1.y - stepY * i, id: 1 \}
    ]);
}
// Lift all fingers
ra.touchUp();
// Wait for all operations above to finish
ra.flush();

// Exit RootAutomator2. If you don't exit properly, “fingers” may remain stuck on screen
ra.exit();

new RootAutomator2([options])

• options

  Options for creating RootAutomator2 (optional).

  • adb {boolean} Whether to use ADB permission. Default false.
  • root {boolean} Whether to use root permission. If ADB is not specified, default true.

Create a new RootAutomator2 instance using options. RootAutomator2 has better compatibility than RootAutomator.

You can choose whether to use root or ADB, etc. See ShellOptions. If neither is specified, root is used by default.

RootAutomator2.tap(x, y)

Tap at (x, y) for 5ms. This function waits for the operation to complete synchronously, so execution time may be longer than the actual touch duration. If you need precise timing, use async methods like touchDown / touchUp instead.

Parameters

Name	Type
x	number
y	number

RootAutomator2.longPress(x, y)

Long-press at (x, y). Duration is ViewConfiguration.getLongPressTimeout() + 100ms. This function waits synchronously, so execution time may exceed actual touch time. If you need precise timing, use async methods like touchDown / touchUp.

Parameters

Name	Type
x	number
y	number

RootAutomator2.press(x, y, duration)

Press at (x, y) for duration, then lift the finger. This function waits synchronously; use async methods like touchDown / touchUp if you need tighter timing.

Parameters

Name	Type	Description
x	number	-
y	number	-
duration	number	Press duration (ms)

RootAutomator2.swipe(x1, y1, x2, y2, duration)

Swipe from (x1, y1) to (x2, y2) over the given duration. This function waits synchronously; use async methods if you need tighter timing.

Parameters

Name	Type	Description
x1	number	-
y1	number	-
x2	number	-
y2	number	-
duration	number	Swipe duration (ms)

RootAutomator2.touchDown(x, y, [id])

Press down at (x, y). If the finger for the given id is already down, it simulates a move (touchMove). This is async; call flush to wait for completion.

Parameters

Name	Type	Description
x	number	-
y	number	-
id	number	Finger id, default 0

RootAutomator2.touchDown(pointers)

Simulate finger-down using an array of pointers. For example, ra.touchDown([{x:100,y:100,id:0},{x:200,y:200,id:1}]) presses finger 0 at (100,100) and finger 1 at (200,200). This is async; call flush to wait.

Parameters

Name	Type	Description
pointers	Array	Pointer list; each item has x, y, and id

RootAutomator2.touchMove(x, y, [id])

Move a finger to (x, y). If the finger for the given id is not down yet, it simulates touchDown. This is async; call flush to wait.

Parameters

Name	Type	Description
x	number	-
y	number	-
id	number	Finger id, default 0

RootAutomator2.touchMove(pointers)

Simulate finger-move using an array of pointers. For example, ra.touchMove([{x:100,y:100,id:0},{x:200,y:200,id:1}]) moves finger 0 to (100,100) and finger 1 to (200,200). This is async; call flush to wait.

Parameters

Name	Type	Description
pointers	Array	Pointer list; each item has x, y, and id

RootAutomator2.touchUp([id])

Lift finger(s).

Parameters

Name	Type	Description
id	number	Finger id. If omitted, lift all fingers

RootAutomator2.touchUp(pointers)

Simulate finger-up using an array of pointers. For example, ra.touchUp([{x:100,y:100,id:0},{x:200,y:200,id:1}]) lifts fingers 0 and 1, using their positions at lift time. This is async; call flush to wait.

Parameters

Name	Type	Description
pointers	Array	Pointer list; each item has x, y, and id

RootAutomator2.flush()

Wait for all operations to complete. For example, after using touchDown, touchMove, and touchUp to perform gestures, call ra.flush() to wait before continuing.

RootAutomator2.exit([forced])

Exit RootAutomator2.

Parameters

Name	Type	Description