PickSensor

The PickSensor node retrieves information about objects that can be picked under mouse position or by using a raycast.

Platform support

This node is fully supported on XR-Windows platform.

It is partially supported on XR-Portable Windows, XR-Portable iOS, XR-Portable Android, and XR-P WASM platforms.

Both synch and async picking are supported on all the platforms.

Mesh picking and feature picking

PickSensor enables the user to perform two types of picking operation.

Mesh

The first type of picking operation provides the user with a list of picked mesh node names, with picking point actual information and distance from the camera that did the pick operation. The result of this query is stored in the pickMeshResult field.

Feature

The second types of picking operation, called feature picking, provides a list of mesh nodes with additional geometrical information such as vertexes, edges, and triangles, based on the strategy configuration. The result of this query is stored in the pickFeatureResult field.

Picking with strategies

PickSensor is now provisioned with the new picking strategy, where you can specify the picking strategies. PickSensor can use this info for picking objects in the model that are present in the scene.

For example: PickingStrategyMask, ObjectType, and ObjectFilter, which are set in the PickingStrategy node.

PickSensor can have multiple picking strategies. You will need to mention the PickingStrategy node name in the pickingStategyNode field for the specific picking strategies on models.

Synchronous and Asynchronous picking

PickSensor is supplied with both synchronous and asynchronous picking functionalities.

• Synchronous picking is managed by SFUNCTION fields and is restricted to MWX geometric content.

• Asynchronous picking functionalities work with any supported geometric content, including MWX, RVM and RFC/GCD. The result of the asynchronous pick is usually made available in the pickMeshResult field in the frame after the pick request.

In general terms, on XR-Windows asynchronous picking is recommended because it supports all geometric formats and has same the functions for mouse and raycast picking.

Using Synchronous picking

There are two available SFUNCTION fields:

• pickMesh - Picks any MWX content under the mouse position. It accepts parameters to filter the results.

• pickCMeshRay - Picks any MWX collision meshes using the given raycast (vector position and direction).

PickMesh filter functions

PickMesh accepts an MSTRING with the following constraints.

String example

contains:itemcolor,containsNot:voxel,shortname

Using Asynchronous picking

Asynchronous picking is managed by two fields. Once the field is invoked by assigning the request parameters, the pickMeshResult is populated when the result is ready, usually the frame after the request.

• pickMeshMouseAsync—Picks the scene under mouse position.

• pickMeshRayAsync—Picks using a raycast based on assigned pick position and direction.

• pickMeshScreenPosAsync—Picks the scene under a specific position in pixels.

• pickMeshScreenRectAsync—Pick the scene under a specific rectangle in pixels.

• pickFeatureScreenRectAsync—Pick the scene under a specific rectangle in pixels.

The asynchronous picking results depends on the pickMeshResultMode:

• single—Returns the nearest picked element

• multiple—Returns all the picked MWX elements, the nearest RFC/GCD element and the nearest RVM element. All the results are sorted by the picking distance.

  Note: The pickMeshScreenRectAsync and pickFeatureScreenRectAsync fields cannot sort results by distance (there are no actual picking points), so it always returns all the picked elements under the rectangle independently from the pickMeshResultMode.

About picking with Screen coordinates fields

Inside the XR Framework, all 2D coordinates (such as positions, sizes, and mouse/touch interactions) are relative to a virtual grid.

However, these fields do not use the virtual grid. Instead, they require actual render panel screen pixel coordinates.

• pickMeshScreenPosAsync

• pickMeshScreenRectAsync

• pickFeatureScreenRectAsync

The grid size and actual window size in pixels may not always correspond (such as when running windowed), so if the user wants to pick a specific position in grid space, it will be necessary to convert grid position to window position. For example, mouse position in global node is expressed in grid space, not window space.

This code example shows grid position to window position conversion.

<Command name="cmd_convertGridPointsToScreenPixels" args="pos=svec2" returnType="svec2" >

<return value="#[@%pos%.x]*[@global.screenWidth]/[@global.gridWidth]# #[@%pos%.y]*[@global.screenHeight]/[@global.gridHeight]#" />

</Command>

The pick mesh result

Both the synchronous SFUNCTION fields and the pickMeshResult present the pick result with the same format.

Returns an MSTRING (comma-separated list of strings) with the following ordinated content.

Result example

This is an example of result code for PickSensor.

4,Liquid_primary|itemcolor_PUMP013_04_voxel,3793.885,-604.9771,166.6722,-488.6733,0.403554,0,-0.9149558,0,0,,solid_pipes|solid_pipes|loop1_tubo01,3834.876,-597.3361,148.5798,-452.6936,0.3415579,0.5359266,-0.7720887,0.5149921,0.3235224,,Common|griglia,7872.605,155.3282,-1633.579,3091.409,0,1,0,0.002844875,0.4878708,,Common|sky,19309.22,2287.202,-6681.429,13129.86,-0.08653488,0.4696281,-0.8786132,0.9687502,0.6331741,

Note: When picking with pickMeshScreenRectAsync, the structure will contain the names only, because the screen picking is not based on pick points.

Code example

This is a code example for PickSensor.

<PickSensor name="pSensor"/>

<var name="picked" type="MSTRING"/>

<command name="cmdSensor">

<setfield field="picked.value" value="@pSensor.pickMesh.execute(contains:itemcolor)@" />

</command>

The pick feature result

The pickFeatureScreenRectAsync function present the pick result inside the sjson pickFeatureResult field.

The field contains a json serialization of an array of object requested features, grouped by mesh nodes.

Note: On the NX (Portable) platform, only the first vertex is returned and all the other values are populated with defaults.

Result example

This is an example of feature picking result code for PickSensor.

[

{

"ObjectType": 2,

"ObjectName": "SampleModel|SoupMesh§1616",

"Distance": 10.3176479,

"NormalIsValid": true,

"IsTransparent": false,

"TransformIsValid": true,

"NumTriangles": 10,

"NumVertices": 0,

"NumEdges": 10,

"Triangles": [

{

"ax": -1.11179948,

"ay": -4.56999969,

"az": -11.6179,

"bx": -1.11179948,

"by": 6.50000048,

"bz": -11.6179,

"cx": -1.11179948,

"cy": 6.50000048,

"cz": -11.6006,

"nx": 1.0,

"ny": 0.0,

"nz": 0.0,

"dist": 10.3176479

},

{

"...": ...,

}

],

"Vertices":: [

{

"x": -10.7924461,

"y": -12.12,

"z": 3.85872912,

"dist": 1.64887393

},

{

"...": ...,

}

],

"Edges": [

{

"ax": -1.11179948,

"ay": -4.56999969,

"az": -11.6179,

"bx": -1.11179948,

"by": 6.50000048,

"bz": -11.6179,

"dist": 10.3176479

},

{

"...": ...,

}

],

"Transform": {

"m11": 1.0,

"m12": 0.0,

"m13": 0.0,

"m21": 0.0,

"m22": 1.0,

"m23": 0.0,

"m31": 0.0,

"m32": 0.0,

"m33": 1.0,

"m41": 9.0,

"m42": 6.0,

"m43": 10.0

}

},

{

"...": ...,

}

]

PickSensor fields

These are the fields for PickSensor node. Only the node-specific fields are indicated, not fields obtained by inheritance.

Field inheritance: NodeBase > PickSensor