AVEVA™ XR Studio

ItemMonitor

Save PDF

TABLE OF CONTENTS

ItemMonitor

Save PDF

Last UpdatedSep 05, 2024
8 minute read

The ItemMonitor node can display images to emulate a monitor display.

It is similar to and can be considered the non-floating version ItemPanelPopup.
The ItemMonitor node inherits from ItemTextureDisplay.

Platform support

This node is fully supported on XR-Windows platform.

It is partially supported on XR-Portable Windows, XR-Portable iOS, XR-Portable Android, and XR-P WASM platforms.

XR-WIN	XR-P-WIN	XR-P-IOS	XR-P-AND	XR-P-WASM
Full support	Partial support	Partial support	Partial support	Partial support

Supported content types

ItemMonitor supports these content types for the images parameter.

Content type	Description
images	.png, .tga, .dds, .gif
http image urls	The complete web address (HTTP URL) of an image.
Texture2D nodes	The syntax is internal:texture2DNodeName.

Interaction types

ItemMonitor has these interaction types.

Type	Description
Interacting with RectBase2D objects	When the ItemMonitor displays a Texture2D that contains RectBase2D-derived objects, such as Button2D, you can interact with those objects. Keyboard interaction is active by default.
Interacting with Interaction Selector	Place the interaction selector target on the RectBase2D-derived object and trigger the push2D action.
Interacting with Hand nodes	When the ItemMonitor displays a Texture2D that contains RectBase2D-derived objects, such as Button2D, you can interact with those objects using your Hand nodes. A ray starting from the index terminal point of the Hand with the same direction of the index finger last phalanx is used to calculate the point of interaction. You can think to the point of interaction as the mouse position on your Texture2D, even though, on the contrary of what happens with the mouse, each Texture2D has its own separated point of interaction, so moving your hand over an ItemMonitor will not affect the position of the point of interaction on other ItemMonitors in the scene.

Type

Description

Interacting with RectBase2D objects

When the ItemMonitor displays a Texture2D that contains RectBase2D-derived objects, such as Button2D, you can interact with those objects. Keyboard interaction is active by default.

Interacting with Interaction Selector

Place the interaction selector target on the RectBase2D-derived object and trigger the push2D action.

Interacting with Hand nodes

When the ItemMonitor displays a Texture2D that contains RectBase2D-derived objects, such as Button2D, you can interact with those objects using your Hand nodes. A ray starting from the index terminal point of the Hand with the same direction of the index finger last phalanx is used to calculate the point of interaction.

You can think to the point of interaction as the mouse position on your Texture2D, even though, on the contrary of what happens with the mouse, each Texture2D has its own separated point of interaction, so moving your hand over an ItemMonitor will not affect the position of the point of interaction on other ItemMonitors in the scene.

Interaction with Hand nodes

When the ItemMonitor textureInteractionEnabled field is set to true and the textureInteractionMode is set to hand, the ItemMonitor is configured for interaction with Hand nodes. This means that all the fields related to the hand interaction will be used to configure how the ItemMonitor reacts to Hand inputs.

Picking modes

Hands interaction is based on ray picking. There are two picking modes to select between using the handInteractionMode field:

indexDirection: Picks using the hand finger tip direction (indexTerminalMatrixAtVector). This allows you to implement a point interaction approach, generally used when interacting from distance.
vector: Picks with a vector defined by the monitor itself (handInteractionVector). The vector is usually set to monitor normal so that a Tap interaction approach is achievable.

How hand interaction works

Based on the defined interactingHands, whenever the index terminal point of one of the interacting hands gets closer to the ItemMonitor than handOverDistance, the point of interaction of the ItemMonitor’s Texture2D is moved on the collision point of the ray starting from the index tip of that hand with the direction of the finger’s last phalanx. Thus, moving the hand index finger tip at a distance less than handOverDistance, but higher than the handTouchDistance will move the interaction point of the Texture2D to the pointed position. This means that pointing toward any RectBase2D element at this distance will make its isOver field become true.

Additional fields can be used to observe the hand interaction:

handInteractionState: Shows the current hand/monitor interaction state and allows the creation of state-based scripts.
handleInteractionDistance: Shows the distance between the interacting hand and the monitor.
handleInteractionMatrix: When the hand is picking over the monitor, this will be the matrix obtained using the pick point position and the pick point normal. Can be used to place a 3D content over the picking point.

Event propagation and hand interaction

If the finger tip gets closer to the ItemMonitor than handTouchDistance, the down event is sent to the Texture2D. When the finger gets again further than handTouchDistance, the up event is sent.

Both up and down events are sent based on the distance check and the down event is sent only if the index finger is pointing toward the ItemMonitor. After a down event has been sent, an up event can follow if the finger gets again further than handTouchDistance or the finger stops pointing at the ItemMonitor. In the latter case, the up event will be sent on an arbitrary interacting position that lays out of the Texture2D bounds.

As colliding with the Hand on the ItemMonitor will most likely result in the finger tip to pass through the ItemMonitor and stop pointing at the monitor surface (as it is behind it), the up event will not be sent as soon as the finger tip stops pointing at the monitor after sending a down event.
After the down event has been sent, in fact, while the finger tip stays beyond the ItemMonitor, but within the distance of handTouchBeyondTolerance, nothing happens. This provides a more fluid experience for the user that has the necessary tolerance to retract the Hand after a collision and cause the up event on the correct position (compared to the movement that caused the down event) while going backward.

For more information on event propagation, see Event-based management.

ItemMonitor and object highlighting

In some cases, ItemMonitor interaction can conflict with selected object highlighting (Filler and Silhouette). This can be challenging, especially in VR.

To resolve this type of problem, you can configure the monitor with the displayOverPostEffects field (inherited from Item). This way the filler and silhouette do not display.

Note: The functionality is available for all Item types, but more relevant for ItemMonitors.

Code example

This is a code example for the ItemMonitor node.

Default Actions

Action	Version	Hotkey	Text	Description
check		X	Check	Checks the item.
prev		LT	Previous	Moves to previous content display.
next		RT	Next	Moves to next content display.
push2D	2.5	Y	Push2D	Enables user to interact with the content of the ItemMonitor when it is Texture2D. It emulates the mouse left-button click.

ItemMonitor fields

These are the fields for ItemMonitor node. Only the node-specific fields are indicated, not fields obtained by inheritance.

Field inheritance: NodeBase > NodeBaseMovable > Item > ItemTextureDisplay > ItemMonitor

The trend popup has a set of fields that belong to the panel and a set of fields relative to each variable (var0, var1, var2) that can be displayed in the panel.

Fields	Type	Use	Default value	Description
backgroundClear	sbool	Optional	false	Clears the background before drawing the updated content.
handInteractionDistance	sdouble	Read only	Internally calculated	When a hand is interacting with the monitor, this field contains the distance from the monitor.
handInteractionMatrix	smatrix	Read only	Internally calculated	When the hand is picking over the monitor, this field contains the matrix of the pick point aligned with the pick point vector.
handInteractionMode	senum	Optional	indexDirection	Defines the way in which the hand finger interacts (using ray picking) with the monitor. Two options are available: indexDirection: Uses the indexMatrixAtVector to pick (point at approach). vector: Picks using the handInteractionVector. Typically, this vector is set to the opposite of the monitor normal (tap approach).
handInteractionState	senum	Read only	Internally calculated	Defines the state of the interacting hand from several available states: no: Not interacting. nearby: Inside interaction range, but without picking. handOverRay: Inside handRayLength. handOverFingerTip: Inside handOverDistance. handTouch: Inside handTouchDistance.
handInteractionVector	svec3	Optional	0 0 1	The vector to be used to pick from the hand finger tip against the monitor when the handInteractionMode is set to vector. It’s expressed in monitor space and usually should be set to the opposite of the monitor normal.
handOverDistance	sfloat	Optional	0.15	Distance from which a Hand pointing toward the ItemMonitor is considered actively interacting.
handTouchDistance	sfloat	Optional	0.02	Distance from which a Hand pointing toward the ItemMonitor will send the down event to the Texture2D.
handTouchBeyondTolerance	sfloat	Optional	0.07	Distance of tolerance that prevents the up event to be sent as soon as the finger tip goes behind the ItemMonitor.
images	mstring	Optional	Not set	List of images.
maxDistanceVisible	sfloat	Optional	-1	If set to a different value then -1, maxDistanceVisible defines the maximum distance from the monitor that causes the monitor to be updated or to handle interaction.
nextimage	sevent	Optional	Not set	Displays next content from images list.
previmage	sevent	Optional	Not set	Displays previous content from images list.
textureInteractionCoordinates	svec2	Read only	Internally Calculated	Contains the coordinates of the interaction. The value is relative to Texture2D size. When there is no interaction, the value is set to -100 -100.
textureInteractionEnabled	sbool	Optional	false	Enables the interaction with the shown Texture2D node. It must be set to true to enable any kind of interaction on the content shown by ItemMonitor.
textureInteractionMode	senum	Optional	push2D	When set to push2D, the interaction events are linked to the state of the Push2D action. When the action is started, the down event is sent to the Texture2D, on the other hand when the action is stopped the up event is sent. When set to hand, the Hand interaction is enabled.
textureInteractionHands	mstring	Optional		List of Hand nodes that can interact with the ItemMonitor when the textureInteractionMode is set to hand and the textureInteractionEnabled is true.
textureInteractingHands	mstring	Read only	Internally Calculated	Internally calculated list that shows the Hand nodes that are currently interacting with the ItemMonitor. To save processing time, hand interaction is evaluated only when the hand distance from the center of the ItemMonitor is less then boundingSphere+HandOverDistance. Therefore, the value of the textureInteractingHands field can show a Hand node as interacting, even when the hand is outside the handOverDistance.
textureInteractionMode	senum	Optional	push2D	When set to push2D, the interaction events are linked to the state of the Push2D action. When the action is started, the down event is sent to the Texture2D, on the other hand when the action is stopped the up event is sent. When set to hand, the Hand interaction is enabled.
value	sint	Optional	0	Index of the currently selected image content.

AVEVA™ XR Studio

ItemMonitor

Table of Contents

ItemMonitor

Platform support

Supported content types

Interaction types

Interaction with Hand nodes

Picking modes

How hand interaction works

ItemMonitor and object highlighting

Code example

Default Actions

ItemMonitor fields

In This Topic

Related Links