Setting Output Precision of Distance Units

Note:
The user must not change the precision if customizing the AVEVA Applicationware. Only use this command if writing a completely new application.

Keywords

PRECISION

Description

Output dimensions, in the current units, are normally rounded to the following degrees of precision:

Output for setting variables defaults to the following, higher, precisions:

The user may control the output precision of distances in some modules by using the PRECISION command. Which allows the user to specify the number of decimal places for MM or INCH output, or the largest fraction denominator for FINCH output. Precision levels for general output and for the setting of variables are specified separately.

The following command is used to specify the output precision of distance values.

PRECision opt1 DIstance opt2 value opt3 (value2 opt4)

or

PRECision opt1 opt2 value opt3 (value2 opt4)

To specify the precision of all other values, including pure numerical values the following command is used:

PRECision opt1 GENeral opt2 value opt3 (value2 opt4)

To reinstate default precision (as at programme initialisation) the following command is used:

PRECision opt1 [DISTance|GENeral| ] DEFault

The default initial precisions for both GENERAL and DISTANCE values are set 5 digits and 4 decimal places for OUTPUT values, and 6 digits and 6 decimal places for VARIABLE values.

Where the command options (opt1, opt2, opt3 and (optionally) opt4) are as follows:

opt1 defines how the specified precision is to be applied. It is one of the following:

If opt1 is omitted:

• Both Output and Variables are set if opt2 is DEFAULT

• Output only is set if opt2 is TO

opt2 defines whether the user is changing to a newly-defined precision or back to the default settings. It is one of the following:

TO

DEFault

If opt2 is omitted, TO is assumed.

opt3, which only applies if opt2 is TO, qualifies the number specified by value. The available choices are:

If opt3 is omitted, value is assumed to apply as follows:

• If current distance units are MM or INCH: value is Decimals

• If current distance units are FINCH: value is THs or NDs

The FINCH precision is only accepted and set if the current working DISTANCE or BORE units are FINCH. Otherwise the command is ignored.

If opt3 is present and set to decimals then a second value (value2) with an optional fourth option (DIGITS - but this is assumed) may be specified. This is used to be able to set both the maximum number of decimal places in the number and the maximum number of significant digits to be output, as in:

PRECI VAR DISTANCE 6 DECIMALS 8 DIGITS.

If both DPLS and DIGITS are set then they constrain each other so neither of the values is exceeded unless the number is of size greater than 10**DIGITS, for example, more digits are required to represent the number with no decimals than DIGITS has specified.

It is possible to unset a precision by using the string UNSET in place of the number of decimal places or digits (value and value2). This allows the other constraint to control the output of the value in all cases without regard any limitation of the other. For example, with no limitation on digits there will always be the required number of decimal places regardless of the size of number.

If both digits and places are unset the precision reverts back to default precisions.

Default initial precisions set 5 digits and 4 decimal places for OUTPUT values, and 6 digits and 6 decimal places for VARIABLE values.

OUTPUT to DATAL files uses its own precision setting that the user has no control over

VARIABLE precision applies to the default formatting of numbers for real to string conversions of PML variables.

OUTPUT precision applies to the precision applied to normal query output including GetAsString calls from .NET.

DATAL output uses its own precision setting that the user has no control over.