MES Curator runtime behavior

On startup, and then every 1 minute, the MES Curator checks db_status.mes_curator_service_host to see if it can process the data curation work and, depending on the status of the mes_curator_service_host, takes one of the following actions:

• If the staging_curator_status.mes_curator_service_host is null, then no other machine is processing the data curation work and the current machine immediately starts processing the data curation work.

• If the staging_curator_status.mes_curator_service_host contains a machine name and the staging_curator_status.service_last_ran_utc is not older than 5 minutes from the current UTC time, then the current machine puts itself in a holding state and does not do any data curation work.

• If the staging_curator_status.mes_curator_service_host contains a machine name and the staging_curator_status.service_last_ran_utc is older than 5 minutes from the current UTC time; then the current machine immediately starts processing the data curation work.

Every minute the MES Curator validates whether it can process data curation to avoid duplicating the curator work with other Curator services running on different machines. When the current machine gets access to process the data curation work, it starts scanning for changes for each reference type based on the staging_type_config.processing_order. After all the reference data types are scanned for changes, each event type is scanned for changes based on the processing order. Changes are determined by comparing the last_edit_at value in the corresponding MES database table with the last_mes_polled_at_utc in the staging_write_status table for each data type. Records are only requested when the staging_type_config.curate_frequency for the data type has been exceeded.

The curator behaves as follows for deleted events:

• Data deletes from the following MES database tables will not be curated: ShiftHistory, JobHourHistory, JobHistory, JobEvent, and ItemTransfer.

• Data deletes from the following MES database tables will be curated along with their upserts: UtilHistory ItemProd, ItemCons, and ItemInv.

That is, when the production is due to be curated, the data change and the production deleted will be curated together. Similarly, when the utilization is due to be curated, the data change and the utilization deletes will be curated together. Deletes triggered through the MES Client purge process will not be curated. Production and utilization events can be deleted through reclassification of production events or merging of utilization events which are curated.

Note: Since changes in the MES database ent table do not distinguish the shift changes from the can_property changes, any change detected (including the shift change) during the cycle will be captured and curated. Therefore, there could be entity related data present that may not have any changes in it.