Storing Original Non-Streamed Data

If the time order of the original data sent to the historian server cannot be maintained due to the nature of the data source, the original data should be sent as original non-streamed data through the HistorianAccess.SendValues() method.

For example, there could be some remote data collectors collecting data every hour, but that hourly data gets delivered to the processing center not always in the same time order it was collected. So an SDK application running in that center should process the delivered hourly data batch immediately, while later it could receive a data batch from the previous hour.

Example

HistorianDataValue myVTQ = new HistorianDataValue();
HistorianAccessError error;
           
HistorianDataValueList myVTQs =
    historian.CreateHistorianDataValueList(HistorianDataCategory.NonStreamedOriginal);

for (int i = 0; i < 1000000; i++)
{
    myVTQ.Value         = i;
    myVTQ.StartDateTime = DateTime.Now;
    if (myVTQs.Add(myVTQ, out error))
        continue;
    if (error.ErrorType == HistorianAccessError.ErrorTypeValue.CustomError &&
        error.ErrorCode == HistorianAccessError.ErrorValue.BufferFull)
    {
        // non-streamed buffer is full, send it
        if (!historian.SendValues(myVTQs, out error))
        {
             Console.WriteLine("Failed to send buffer: {0}", error.ErrorDescription);                
             break;
        }
        // buffer has been sent successfully, try to send the last VTQ again
        if (!myVTQs.Add(myVTQ, out error))
        {
            Console.WriteLine("Empty buffer overflow? {0}", error.ErrorDescription);                
            break;
        }
    }
    else
    {
        Console.WriteLine(“Failed to add VTQ: {0}”, error.ErrorDescription);                
        break;      
    }
}
if (!historian.SendValues(myVTQs, out error))
     Console.WriteLine("Failed to flush buffer: {0}", error.ErrorDescription);                

Sending original non-streamed data happens in several steps:

1. An empty non-streamed VTQ collection of the HistorianDataValueList class must be explicitly created using the HistorianAccess.CreateHistorianDataValueList() method.
2. New VTQs should be added one by one to that list by the Add() method of the HistorianDataValueList class until that method returns false with the error type HistorianAccessError.ErrorTypeValue.CustomError and error code HistorianAccessError.ErrorValue.BufferFull.
3. The accumulated values must be sent to the historian server by calling the SendValues() method of the HistorianAccess class. The list gets automatically cleared if the data is stored successfully.
4. Steps 2-3 should be repeated until all VTQs are sent.
5. The remaining VTQs in the list should be flushed to the historian server to ensure that all VTQs have been sent out.

Every call to the SendValues method is a synchronous push operation delivering VTQs accumulated in the HistorianDataValueList instance within a single non-streamed data packet.

Sending original non-streamed data requires more effort from the SDK application developer because the current version of the HCAL infrastructure supports the asynchronous store-and-forward mechanism only for original streamed data. To send original non-streamed or revision data, the historian server must be up and running.

Sending original non-streamed data from an SDK application is equivalent to executing a T-SQL INSERT statement with wwVersion = ‘ORIGINAL’ while being connected to the Runtime database.

Although the original non-streamed VTQs can be sent in any time order for the same tag, sending them in time order (or in a partial time order if possible) is a good practice that may help to reduce the load on the historian server.