Regno Standard - ParamSamplesDoc

Data - ParamSamplesDoc

The ParamSamplesDoc document is a data document that contains the properties and values for a collection of data samples for a single data channel.

The ParamSamplesDoc document is used to store raw sample values for user specified intervals. The document contains the field documentInterval that is used to specify the time range that the raw data samples values have sampled from.

Field	Data Type	Required	Default	Description
*configDocId*	String	Yes	-	The unique identifier (id) of the ConfigDoc document that this document is associated with.
id	String	Yes	-	Regno unique document identifier.
*type*	Fixed String	No	ParamSamplesDoc	Type of the document, resolved from fixed document types.
*dataType*	Fixed String	No	Double	Data type of the values in the document, used for serialization and deserialization. Valid Options are: Double, Long, Byte, Short, Float, Integer
*sampleTimes*	Byte[]	No	-	An array of compressed 64-bit Long values. Each value represents the timestamp value of the sample at the associated array index. The timestamp represents the number of nanoseconds since the reference time of midnight, 1st January 1970.
*times*	Int64[]	No	-	An array of long values. Each value represents the sample value at the associated array index. The type of the value is DataType
*sampleValues*	Byte[]	No	-	An array of compressed values. Each value represents the sample value at the associated array index. The type of the value is DataType
*sampleCount*	Int64	Yes	-	A Long that holds the value for the number of samples in a document.
*paramDefDocId*	String	Yes	-	The unique identifier (id) of the ParamDefinitionDoc that holds the properties for the parameter associated with the data values stored in this document.
*startTime*	Int64	Yes	-	A 64-bit Long timestamp specifying the start time of the data that this document holds. The timestamp represents the number of nanoseconds since the reference time of midnight, 1st January 1970.
*endTime*	Int64	Yes	-	A 64-bit Long timestamp specifying the end time of the data that this document holds. The timestamp represents the number of nanoseconds since the reference time of midnight, 1st January 1970.
*min*	Double	Yes	-	A 64-bit Double value specifying the minimum value of the data that this document holds.
*max*	Double	Yes	-	A 64-bit Double value specifying the maximum value of the data that this document holds.

Sample JSON

Below is a sample JSON representation of the ParamSamplesDoc structure:

▼ ParamSamplesDoc Sample JSON

{
  "configDocId": "-",
  "dataType": {
    "$type": "Fixed String"
  },
  "endTime": {
    "$type": "Int64"
  },
  "id": "-",
  "max": 0,
  "min": 0,
  "paramDefDocId": "-",
  "sampleCount": {
    "$type": "Int64"
  },
  "sampleTimes": [
    {
      "$type": "Byte"
    }
  ],
  "sampleValues": [
    {
      "$type": "Byte"
    }
  ],
  "startTime": {
    "$type": "Int64"
  },
  "times": [
    {
      "$type": "Int64"
    }
  ],
  "type": {
    "$type": "Fixed String"
  }
}

Data Decompression Guide

The sampleTimes and sampleValues fields in the ParamSamplesDoc contain compressed byte arrays that need to be decompressed before use. The Regno Standard uses GZip compression to efficiently store large arrays of sample data.

Decompression Process:

GZip Decompression: First, decompress the byte array using standard GZip decompression
Type Conversion: Convert the decompressed bytes back to the appropriate data type based on the dataType field

Implementation Examples:

▼ C# Implementation

using System;
using System.IO;
using System.IO.Compression;

public static class RegnoDecompression
{
    // For sampleTimes (always long arrays)
    public static long[] DecompressSampleTimes(byte[] compressedBytes)
    {
        if (compressedBytes == null || compressedBytes.Length == 0)
            return new long[0];
            
        using (var compressedStream = new MemoryStream(compressedBytes))
        using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
        using (var resultStream = new MemoryStream())
        {
            zipStream.CopyTo(resultStream);
            var decompressedBytes = resultStream.ToArray();
            
            var result = new long[decompressedBytes.Length / sizeof(long)];
            Buffer.BlockCopy(decompressedBytes, 0, result, 0, decompressedBytes.Length);
            return result;
        }
    }

    // For sampleValues - generic method based on dataType
    public static T[] DecompressSampleValues<T>(byte[] compressedBytes) where T : struct
    {
        if (compressedBytes == null || compressedBytes.Length == 0)
            return new T[0];
            
        using (var compressedStream = new MemoryStream(compressedBytes))
        using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
        using (var resultStream = new MemoryStream())
        {
            zipStream.CopyTo(resultStream);
            var decompressedBytes = resultStream.ToArray();
            
            var elementSize = System.Runtime.InteropServices.Marshal.SizeOf<T>();
            var result = new T[decompressedBytes.Length / elementSize];
            Buffer.BlockCopy(decompressedBytes, 0, result, 0, decompressedBytes.Length);
            return result;
        }
    }
    
    // Usage examples:
    // var times = DecompressSampleTimes(paramSamplesDoc.SampleTimes);
    // var values = DecompressSampleValues<double>(paramSamplesDoc.SampleValues);
}

► JavaScript Implementation

// Requires: pako library for gzip decompression (npm install pako)
import pako from 'pako';

class RegnoDecompression {
    
    // For sampleTimes (always 64-bit integers)
    static decompressSampleTimes(compressedBytes) {
        if (!compressedBytes || compressedBytes.length === 0) {
            return new BigInt64Array(0);
        }
        
        try {
            // Decompress using pako (gzip)
            const decompressed = pako.inflate(compressedBytes);
            
            // Convert to 64-bit integers (8 bytes each)
            const longArray = new BigInt64Array(decompressed.buffer);
            
            // Convert BigInt64Array to regular array of numbers for easier use
            return Array.from(longArray, x => Number(x));
        } catch (error) {
            console.error('Error decompressing sample times:', error);
            return [];
        }
    }
    
    // For sampleValues based on dataType
    static decompressSampleValues(compressedBytes, dataType) {
        if (!compressedBytes || compressedBytes.length === 0) {
            return [];
        }
        
        try {
            // Decompress using pako (gzip)
            const decompressed = pako.inflate(compressedBytes);
            
            // Convert based on dataType
            switch (dataType.toLowerCase()) {
                case 'double':
                    return Array.from(new Float64Array(decompressed.buffer));
                case 'long':
                    return Array.from(new BigInt64Array(decompressed.buffer), x => Number(x));
                case 'integer':
                case 'int':
                    return Array.from(new Int32Array(decompressed.buffer));
                case 'short':
                    return Array.from(new Int16Array(decompressed.buffer));
                case 'float':
                    return Array.from(new Float32Array(decompressed.buffer));
                case 'byte':
                    return Array.from(new Uint8Array(decompressed.buffer));
                default:
                    throw new Error(`Unsupported dataType: ${dataType}`);
            }
        } catch (error) {
            console.error('Error decompressing sample values:', error);
            return [];
        }
    }
}

// Usage example:
// const times = RegnoDecompression.decompressSampleTimes(paramSamplesDoc.sampleTimes);
// const values = RegnoDecompression.decompressSampleValues(paramSamplesDoc.sampleValues, paramSamplesDoc.dataType);

► Python Implementation

import gzip
import struct
import numpy as np
from typing import List, Union

class RegnoDecompression:
    
    @staticmethod
    def decompress_sample_times(compressed_bytes: bytes) -> List[int]:
        """
        Decompress sampleTimes (always 64-bit integers)
        """
        if not compressed_bytes:
            return []
        
        try:
            # Decompress using gzip
            decompressed = gzip.decompress(compressed_bytes)
            
            # Unpack as 64-bit integers (long long)
            # '<' means little-endian, 'q' means signed 64-bit integer
            count = len(decompressed) // 8
            times = struct.unpack(f'<{count}q', decompressed)
            
            return list(times)
        except Exception as e:
            print(f"Error decompressing sample times: {e}")
            return []
    
    @staticmethod
    def decompress_sample_values(compressed_bytes: bytes, data_type: str) -> List[Union[float, int]]:
        """
        Decompress sampleValues based on dataType
        """
        if not compressed_bytes:
            return []
        
        try:
            # Decompress using gzip
            decompressed = gzip.decompress(compressed_bytes)
            
            # Convert based on dataType
            data_type = data_type.lower()
            
            if data_type == 'double':
                # 64-bit double (8 bytes each)
                count = len(decompressed) // 8
                values = struct.unpack(f'<{count}d', decompressed)
            elif data_type == 'long':
                # 64-bit long (8 bytes each)
                count = len(decompressed) // 8
                values = struct.unpack(f'<{count}q', decompressed)
            elif data_type in ['integer', 'int']:
                # 32-bit integer (4 bytes each)
                count = len(decompressed) // 4
                values = struct.unpack(f'<{count}i', decompressed)
            elif data_type == 'short':
                # 16-bit short (2 bytes each)
                count = len(decompressed) // 2
                values = struct.unpack(f'<{count}h', decompressed)
            elif data_type == 'float':
                # 32-bit float (4 bytes each)
                count = len(decompressed) // 4
                values = struct.unpack(f'<{count}f', decompressed)
            elif data_type == 'byte':
                # 8-bit byte (1 byte each)
                values = struct.unpack(f'<{len(decompressed)}b', decompressed)
            else:
                raise ValueError(f"Unsupported dataType: {data_type}")
                
            return list(values)
        except Exception as e:
            print(f"Error decompressing sample values: {e}")
            return []
    
    @staticmethod
    def decompress_sample_values_numpy(compressed_bytes: bytes, data_type: str) -> np.ndarray:
        """
        Alternative implementation using NumPy for better performance
        """
        if not compressed_bytes:
            return np.array([])
        
        try:
            # Decompress using gzip
            decompressed = gzip.decompress(compressed_bytes)
            
            # Convert to numpy array based on dataType
            data_type = data_type.lower()
            
            dtype_map = {
                'double': np.float64,
                'long': np.int64,
                'integer': np.int32,
                'int': np.int32,
                'short': np.int16,
                'float': np.float32,
                'byte': np.int8
            }
            
            if data_type not in dtype_map:
                raise ValueError(f"Unsupported dataType: {data_type}")
            
            # Create numpy array from bytes
            return np.frombuffer(decompressed, dtype=dtype_map[data_type])
        except Exception as e:
            print(f"Error decompressing sample values with numpy: {e}")
            return np.array([])

# Usage example:
# times = RegnoDecompression.decompress_sample_times(param_samples_doc['sampleTimes'])
# values = RegnoDecompression.decompress_sample_values(param_samples_doc['sampleValues'], param_samples_doc['dataType'])
# 
# # Using NumPy for better performance:
# values_np = RegnoDecompression.decompress_sample_values_numpy(param_samples_doc['sampleValues'], param_samples_doc['dataType'])

Key Points:

sampleTimes are always 64-bit long values representing nanoseconds since Unix epoch
sampleValues data type is specified by the dataType field (Double, Long, Byte, Short, Float, Integer)
Both arrays have the same length, with each index representing corresponding time and value pairs
The sampleCount field indicates the total number of samples in the arrays