Stream types

A stream type is an immutable schema definition that specifies the structure of data stored in streams. Types define the properties that each data point contains, the data type of each property, and which property serves as the key for ordering data points. Once created, a type cannot be modified, ensuring data integrity and consistency across all streams that reference it. Multiple streams can share the same type definition, promoting reusability and standardization across similar data sources.

Before you can create a stream, you must first create a type that defines what properties each data point will contain. Streams are derived from stream types—when you create a stream, you specify which type it uses, and that type determines the structure of all data points written to the stream. This relationship ensures that every stream has a well-defined schema from the moment it is created.

The following example shows a type definition for a wind turbine monitoring application. The type specifies three properties, with Timestamp serving as the key that determines data point ordering:

This type definition acts as a schema that ensures every data point written to any stream using this type must include all three properties with the correct data types. When you create a stream based on this type, the Timestamp property becomes the stream's index, determining how data points are ordered and retrieved. Multiple streams representing different wind turbines can all reference this same type definition, maintaining consistent data structure across your entire wind farm monitoring system.

Type properties and data types

Each type consists of one or more properties, where each property has a specific basic data type (also called primitive types). Supported data types include integers (like Int32 and Int64), floating-point numbers (like Double and Single), strings, booleans, and DateTime for timestamps. Properties can be marked as required or optional. One property must be designated as the key property, which determines the ordering of data points in streams that use this type. Additional properties can include units of measure and other metadata that controls how the property behaves.

Units of measure

Types support units of measure (UOMs) to specify the measurement units for numeric properties. When you define a property with a UOM, you explicitly declare what unit the values represent—such as kilowatts for power output, degrees Celsius for temperature, or meters per second for velocity. UOMs provide important context for interpreting data correctly and enable applications to display values with appropriate units or perform unit conversions when needed.

UOMs are defined at the type level as part of each property's metadata. Once a type with UOMs is created, all streams using that type inherit those unit definitions, ensuring consistency across related data sources. This standardization helps prevent confusion about what units measurements represent and makes it easier to aggregate or compare data from multiple streams that share the same type definition.

Type keys and stream indexes

The key is the property designated in the type definition that determines data point ordering. When defining a type, you specify which property serves as the key—typically a DateTime for time-series data or an integer for sequence-based data. Once that type is associated with a stream, the key property becomes the stream's index. Both terms refer to the same property, but key is used when discussing the type's schema definition, while index" is used when discussing how data points are ordered and retrieved from the stream.

Simple vs. complex types

Simple types contain properties with basic data types like numbers, strings, and timestamps. These types are straightforward and efficient for most use cases. Complex types contain properties that are themselves structured types, allowing nested data structures. For example, a complex type might have a property that contains location information with nested latitude and longitude values. While complex types offer flexibility for sophisticated data models, simple types provide better performance and are recommended for most scenarios.

Type reusability

Types promote reusability by allowing multiple streams to share the same schema definition. When you create a type, any number of streams can reference it, ensuring consistent data structure across related data sources. For example, if you have temperature sensors across multiple facilities, you can create one TemperatureSensor type and then create individual streams for each physical sensor, all sharing the same type. This approach maintains consistency across related data sources while keeping each stream's data independent and reduces the need to define duplicate schemas.

Best practices

Follow these practices when designing types:

• Keep types simple: Start with basic data types and only use complex nested structures when necessary. Simple types perform better and are easier to maintain.

• Design for reusability: Create types that can be shared across multiple similar data sources rather than creating unique types for each stream. This promotes consistency and reduces schema duplication.

• Choose appropriate key properties: Select key properties that naturally order your data, typically timestamps for time-series data or sequence numbers for ordered data points.

• Use meaningful property names: Name properties clearly to indicate what data they contain and what units they use, making types self-documenting.

• Plan for immutability: Remember that types cannot be changed after creation. If you need to modify a type's structure, you must create a new type and migrate streams to use it.