Week 4

Hello everyone! Today I want to discuss a technical challenge I overcame with this project, in the hopes that someone else can use what I’ve learned in their own work. That problem is mousing over transparent 3D sprites in Godot 4.

The Problem

Godot has methods for detecting mouse input and testing transparency on Sprite2D nodes, but no such equivalent methods for Sprite3D nodes. Godot uses CollisionShapes for 3D mouse interactions, but that limits the detectable area to primitive shapes such as squares.

The Solution

The method I’ve come up with allows for mouse input to be detected on Sprite3D nodes in 3D Space, using only the opaque regions of the sprite.

This solution relies on 2 scripts for both the Camera node, and the Sprite3D node. Furthermore, the Sprite3D needs to be configured in a specific way for this solution to work.

The Sprite3D

For starters, here is how the Sprite3D node is organized. It consists of a parent Node3D with a Sprite3D child. That Sprite3D has a Sprite2D child (we’ll call this our Virtual Sprite going forwards), and an Area3D with a BoxShape3D child. (The UnitOutline node can be ignored, as it’s not part of this solution.)

The important configurations here are that the BoxShape3D has a Z Size of 0.01m (the smallest size for a CollisionShape), and the Area3D has Ray Pickable set to False (we’ll be handling this manually instead), and the Collision Layer set to whichever layer is dedicated to mouse selectable Sprite3Ds (I used layer 2).

The script for our Sprite3D scene is as follows:

First, we define our global variables and signals.

extends Node3D

signal input_event(camera: Node, event: InputEvent, position: Vector3, normal: Vector3)
signal mouse_entered()
signal mouse_exited()

@onready var _sprite: Sprite3D = $UnitSprite
@onready var _virtual_sprite: Sprite2D = $UnitSprite/VirtualSprite
@onready var _collision_shape: CollisionShape3D = $UnitSprite/ClickableArea/CollisionShape3D

var mouse_over: bool = false
var _mouse_input_received: bool = false

Next we define a method to set the texture of our Sprite. This will ensure our nodes all have the same texture, and maintain the correct scale.

func set_sprite(texture: Texture2D) -> void:
    # Update Sprite3D Texture
    _sprite.texture = texture
    # Update Virtual Sprite
    _virtual_sprite.texture = texture
    # Update CollisionShape size and position to match the Sprite3D
    _collision_shape.shape.size.x = texture.get_width() * _sprite.pixel_size
    _collision_shape.shape.size.y = texture.get_height() * _sprite.pixel_size
    _collision_shape.position.x = texture.get_width() / 2.0 * _sprite.pixel_size
    _collision_shape.position.y = texture.get_height() / 2.0 * _sprite.pixel_size

Next, a method to test if the pixel at a given position is considered opaque. This will take the position of our input event, and return true if the position is opaque.

func _is_pixel_opaque(input_position: Vector3) -> bool:
    # Convert input position to Sprite local position.
    var _pixel_position = (input_position - global_position) / (_sprite.pixel_size * scale)
    # We offset these positions because of different centering between the 2D and 3D sprites.
    var _texture_local_x = _pixel_position.x - (_sprite.texture.get_width() / 2.0)
    # The Y Position has to be inverted, because the Y-Axis is flipped in 2D and 3D space.
    var _texture_local_y = (_pixel_position.y - _sprite.texture.get_height() / 2.0) * -1
    var _texture_local_vec2 = Vector2(_texture_local_x, _texture_local_y)
    # We call the built-in method on Sprite2D to determine whether the 2D position is an opaque pixel.
    return _virtual_sprite.is_pixel_opaque(_texture_local_vec2)

Next, we declare a method to test a mouse input, and return true if it is valid (meaning it is over an opaque portion of our sprite)

func try_mouse_input(camera: Node, event: InputEvent, input_position: Vector3, normal: Vector3) -> bool:
    if _is_pixel_opaque(input_position):
        # We set that a mouse input has been received, and emit our signal for an input event
        _mouse_input_received = true
        input_event.emit(camera, event, input_position, normal)
        return true
    else:
        return false

And lastly, a method that is called whenever the camera processes a mouse ray. We use this to update whether or not the mouse is over the sprite, and call any appropriate signals.

func _on_3dspace_mouse_ray_processed() -> void:
    if _mouse_input_received:
        if !mouse_over:
            # This case indicates the mouse has entered this Sprite
            mouse_over = true
            mouse_entered.emit(self)
    else:
        if mouse_over:
            # This case indicates the mouse has exited this Sprite
            mouse_over = false
            mouse_exited.emit(self)
    # Lastly, we reset the _mouse_input_received variable back to false
    _mouse_input_received = false

From here, one can easily configure functions for mouse_entered, mouse_exited, and input_event to handle those scenarios with whatever is needed.

The Camera

Next we have the script for the Camera node we’re using to test mouse inputs from:

extends Camera3D

# This can be adjusted for however far you want to test mouse inputs.
const RAY_LENGTH = 1000.0

# This is the signal our Sprites listen for. It's up to the user to ensure the Sprites are properly connected to this signal.
signal mouse_ray_processed()

# The physics layers for mouse interactible Sprite3Ds
@export_flags_3d_physics var _sprite_layers

var _query_mouse: bool = false
var _mouse_event: InputEventMouse

Next we declare the built-in unhandled_input and physics_process functions. This solution exploits the fact that Godot handles inputs before the physics process.

func _unhandled_input(event):
    if event is InputEventMouse:
	_query_mouse = true
	_mouse_event = event


func _physics_process(_delta):
    if _query_mouse:
        _check_sprite_input()
	_query_mouse = false
	mouse_ray_processed.emit()

And lastly, our function to check for inputs.

func _check_sprite_input() -> bool:
    var not_hits = []
    var space_state = get_world_3d().direct_space_state
    var from = project_ray_origin(_mouse_event.position)
    var to = from + project_ray_normal(_mouse_event.position) * RAY_LENGTH
    # We loop continuously loop through all our collided areas until there is either a successful hit, or no more areas to query   
    while true:
        # We declare a RayQuery using the parameters we defined, excluding all the Sprites we've already tested.
	var query = PhysicsRayQueryParameters3D.create(from, to, _sprite_layers, not_hits)
	query.collide_with_areas = true
	var result = space_state.intersect_ray(query)
	if result.is_empty():
	    return false
	else:
            # We call get_parent twice to get the root of our Sprite3D from the Area3D, then call the function we declared earlier.
	    if result.collider.get_parent().get_parent().try_mouse_input(_camera, _mouse_event, result.position, result.normal):
		return true
	    else:
		not_hits.append(result.collider)
    return true

And that’s it! Once again it’s up to the user to connect the mouse_ray_processed signal to the Sprite3Ds, I do it in my main script when instantiating the Sprites, but the best method may vary depending on your project.