Write data

The Sequential Data Store (SDS) REST APIs provide programmatic access to read and write SDS data. This topic provides guidelines when writing to an SdsStream.

If you are working in .NET, convenient SDS Client libraries are available. The ISdsDataService interface, accessed using the SdsService.GetDataService() helper, defines the available functions.

All writes rely on a stream's key or primary index. The primary index determines the order of data points in the stream. Secondary indexes are updated, but they do not contribute to the request. All references to indexes are to the primary index.

The maximum size of the payload for a single write request is 28.6 MB. Additionally, compressed requests are limited to a maximum inflated payload size of 190 MB. Requests that exceed these limits are rejected with an HTTP status code of 413 Payload Too Large.

Single stream writes

The following methods support writing a single data point or multiple data points:

• Insert Values inserts a collection of data points.
• Patch Values updates specific fields for a collection of data points.
• Remove Values deletes data points based on the request parameters.
• Update Values adds or replaces a collection of data points.

Request body format

With the exception of Remove Values, all single stream write calls require a request body containing the data points to insert or modify.

You must format the data points as a serialized JSON array of the stream's type. JSON arrays are comma-delimited lists of a type enclosed within square brackets. The following code shows a list of three WaveData data points that are properly formatted for insertion. See the CONNECT data services Samples for the complete example.

[
  {
    "Order": 2,
    "Tau": 0.25722883666666846,
    "Radians": 1.6162164471269089,
    "Sin": 1.9979373673043652,
    "Cos": -0.090809010174665111,
    "Tan": -44.003064529862513,
    "Sinh": 4.8353589272389,
    "Cosh": 5.2326566823391856,
    "Tanh": 1.8481468289554672
  },
  {
    "Order": 4,
    "Tau": 0.25724560000002383,
    "Radians": 1.6163217742567466,
    "Sin": 1.9979277915696148,
    "Cos": -0.091019446679060964,
    "Tan": -43.901119254534827,
    "Sinh": 4.8359100947709592,
    "Cosh": 5.233166005842703,
    "Tanh": 1.8481776000882766
  },
  {
    "Order": 6,
    "Tau": 0.25724560000002383,
    "Radians": 1.6163217742567466,
    "Sin": 1.9979277915696148,
    "Cos": -0.091019446679060964,
    "Tan": -43.901119254534827,
    "Sinh": 4.8359100947709592,
    "Cosh": 5.233166005842703,
    "Tanh": 1.8481776000882766
  }
]

You can serialize your data using one of many JSON serializers available at Introducing JSON.

Indexes

Writing to SDS relies on the primary index for positioning within the streams and locating existing data points. Most writes use the index as specified by the data point. Deletes are the exception to this rule. When deleting, indexes are specified as strings in the URI, or, when using the SDS Client Libraries, the index may be passed as-is to DELETE methods that take the index type as a generic argument. For more information on working with indexes, see Indexes. For information on compound indexes, see Compound indexes.

Out-of-Order Data

SDS is optimized for sending and retrieving data in order of the data's index. When writing data, clients should send it sequentially. Failure to do so can affect performance negatively during both data writing or retrieval. This sequential writing practice is important when backfilling large amounts of historical data. You should order data sequentially not just for single write operations, but also in a general sense over time.