1. Introduction

Telemachus defines an open pivot format for high-frequency mobility and telematics data. A Telemachus dataset captures what a telematics device physically measures and transmits: GNSS position, inertial measurements, and optionally vehicle bus data.

This specification consolidates and supersedes RFC-0001 (Core v0.2), RFC-0004 (Extended FieldGroups), and RFC-0013 (Device Layer v0.7).

1.1 Design Principles

• Raw device output only. No enrichment, no interpretation, no external data.
• Columns are flat. No nested JSON objects — every field is a top-level column.
• Units are SI. m/s, m/s², rad/s, degrees WGS84, UTC nanoseconds. Unit suffixes in column names (_mps, _rad_s, _uT) make data self-documenting.
• Multi-rate is native. GNSS and IMU may sample at different frequencies.
• One group per sensor. Each functional group maps to a physical sensor or bus.
• Profiles, not one-size-fits-all. Three profiles (core, imu, full) adapt to different device capabilities.
• Vendor extensions welcome. Extra columns use x_<source>_<field> convention.

1.2 Record Overview

A Telemachus record is a timestamped row containing measurements from up to seven functional groups, each mapping to a distinct sensor or bus:

graph LR
    subgraph RECORD["Telemachus Record"]
        DT["Datetime"]
        GPS["GNSS"]
        IMU["IMU"]
        OBD["OBD"]
        CAN["CAN (future)"]
        IO["I/O"]
        EXTRA["Extra"]
    end

    DT --- GPS --- IMU --- OBD --- CAN --- IO --- EXTRA

    style RECORD fill:#e8f5e9,stroke:#2e7d32
    style DT fill:#fff9c4,stroke:#f9a825
    style GPS fill:#e8f5e9,stroke:#2e7d32
    style IMU fill:#e3f2fd,stroke:#1565c0
    style OBD fill:#dcedc8,stroke:#558b2f
    style CAN fill:#f0f4c3,stroke:#9e9d24
    style IO fill:#fce4ec,stroke:#c62828
    style EXTRA fill:#f3e5f5,stroke:#6a1b9a

2. Column Specification

2.1 Functional Groups

Columns are organized into seven functional groups, each mapping to a physical sensor, bus, or signal type. All columns are flat (no nesting). The grouping is conceptual, for documentation only.

graph LR
    subgraph DT["1 — Datetime"]
        ts["ts (UTC)"]
    end

    subgraph GNSS["2 — GNSS Receiver"]
        lat & lon
        speed_mps
        heading_deg
        altitude_gps_m
        hdop & h_accuracy_m
        n_satellites
    end

    subgraph IMU_G["3 — IMU"]
        direction TB
        subgraph ACCEL["Accelerometer"]
            ax["ax_mps2"]
            ay["ay_mps2"]
            az["az_mps2"]
        end
        subgraph GYRO["Gyroscope (opt.)"]
            gx["gx_rad_s"]
            gy["gy_rad_s"]
            gz["gz_rad_s"]
        end
        subgraph MAG["Magnetometer (opt.)"]
            mx["mx_uT"]
            my["my_uT"]
            mz["mz_uT"]
        end
    end

    subgraph OBD_G["4 — OBD-II"]
        spd_obd["speed_obd_mps"]
        rpm_f["rpm"]
        odo["odometer_m"]
    end

    subgraph CAN_G["5 — CAN (future)"]
        can_f["x_can_<signal>"]
    end

    subgraph IO_G["6 — I/O"]
        ign["ignition"]
        volt["vehicle_voltage_v"]
    end

    subgraph XTR["7 — Extra"]
        x_f["x_<source>_<field>"]
    end

    DT --- GNSS --- IMU_G --- OBD_G --- CAN_G --- IO_G --- XTR

    style DT fill:#fff9c4,stroke:#f9a825
    style GNSS fill:#e8f5e9,stroke:#2e7d32
    style IMU_G fill:#e3f2fd,stroke:#1565c0
    style OBD_G fill:#dcedc8,stroke:#558b2f
    style CAN_G fill:#f0f4c3,stroke:#9e9d24
    style IO_G fill:#fce4ec,stroke:#c62828
    style XTR fill:#f3e5f5,stroke:#6a1b9a

2.2 Profiles

Not all telematics devices have the same sensors. Telemachus defines three profiles to accommodate different hardware capabilities:

graph LR
    subgraph PROFILES["Telemachus Profiles"]
        CORE["core\nGNSS only\nts + lat + lon + speed"]
        KIMU["imu\nGNSS + Accelerometer\ncore + ax + ay + az"]
        FULL["full\nGNSS + Accel + Gyro\nimu + gx + gy + gz"]
    end

    CORE --> KIMU --> FULL

    style CORE fill:#fff9c4,stroke:#f9a825
    style KIMU fill:#c8e6c9,stroke:#2e7d32
    style FULL fill:#bbdefb,stroke:#1565c0

The profile is declared in the manifest (profile field, see SPEC-02). Validation adapts to the declared profile: a core dataset is valid without IMU columns.

Default: if no profile is declared, validators MUST assume imu (GNSS + accelerometer) for backward compatibility.

2.3 Mandatory Fields

Mandatory columns depend on the declared profile:

All profiles (core, imu, full):

Profile imu and full add:

Profile full adds:

2.4 Recommended Fields — Identification

These fields SHOULD be present per-row OR inherited from the manifest (see SPEC-02 §4.1). They are metadata, not physical measurements:

If a file omits device_id / trip_id, consumers MUST resolve them from the manifest. If the manifest declares multiple devices and the file omits device_id, validation MUST fail.

2.5 Recommended Fields — GNSS Metadata

These fields SHOULD be present when the hardware provides them:

hdop vs h_accuracy_m: Commercial GNSS devices (Teltonika, Danlaw) report hdop (dimensionless ratio). Smartphones (Android) report h_accuracy_m (68th percentile radius in meters). Both may coexist; a dataset typically has one or the other, rarely both.

2.6 Extended IMU Fields

These columns are mandatory for the full profile, optional for imu, and absent for core. When a sensor is not available, columns MUST be absent or all-NaN — they MUST NOT be filled with zeros.

graph TD
    subgraph IMU["IMU Sensor Tiers"]
        T1["Tier 1: Accelerometer only\n(ax, ay, az)\nCommercial telematics devices"]
        T2["Tier 2: Accel + Gyro\n(+ gx, gy, gz)\nAEGIS, STRIDE, RS3"]
        T3["Tier 3: Accel + Gyro + Magneto\n(+ mx, my, mz)\nPVS, STRIDE (full)"]
    end

    T1 --> T2 --> T3

    style T1 fill:#ffecb3,stroke:#ff8f00
    style T2 fill:#c8e6c9,stroke:#2e7d32
    style T3 fill:#bbdefb,stroke:#1565c0

2.7 Optional Fields — OBD-II

Standardized vehicle data from the OBD-II diagnostic port (ISO 15031, SAE J1979). These PIDs are universal across OBD-II compliant vehicles.

Two speed fields: speed_mps (GNSS, mandatory) and speed_obd_mps (OBD, optional) are intentionally separate. GPS speed degrades below ~5 km/h and requires a fix; OBD speed is accurate at all speeds but requires a wired OBD adapter.

graph LR
    subgraph SPEED["Speed Sources"]
        GPS_S["speed_mps\n(GNSS Doppler)\nMandatory §2.2"]
        OBD_S["speed_obd_mps\n(OBD PID 0x0D)\nOptional §2.6"]
    end

    GPS_S -- "NaN at low speed" --> NOTE1["Degraded < 5 km/h\nNaN without fix"]
    OBD_S -- "accurate always" --> NOTE2["Requires OBD adapter\nNaN if not wired"]

    style GPS_S fill:#e8f5e9,stroke:#2e7d32
    style OBD_S fill:#dcedc8,stroke:#558b2f

Additional OBD PIDs (throttle position, engine load, coolant temperature, etc.) may be included using the vendor-specific convention x_obd_<pid_name> (e.g., x_obd_throttle_pct, x_obd_coolant_c). These may be promoted to formal columns in future spec versions when supported by Open datasets.

2.8 Future Group — CAN Bus

Raw CAN bus data (SAE J1939, manufacturer-specific DBCs) is not yet formalized in this specification. CAN signals are vendor-specific — each vehicle manufacturer defines its own signal dictionary (DBC file).

Until formal CAN columns are defined, raw CAN data SHOULD use the vendor-specific convention:

x_can_<signal_name>

Examples: x_can_wheel_speed_fl_mps, x_can_steering_angle_deg, x_can_brake_pressure_bar.

CAN column formalization will be considered when Open datasets with raw CAN data become available. The key difference with OBD: OBD PIDs are standardized (same PID = same meaning across all vehicles), CAN signals are manufacturer-specific (same signal ID = different meaning per vehicle make/model).

graph TD
    subgraph BUS["Vehicle Bus Data"]
        OBD_B["OBD-II (§2.6)\nStandardized PIDs\nISO 15031 / SAE J1979\nUniversal across vehicles"]
        CAN_B["CAN (§2.7)\nRaw signals\nManufacturer-specific DBC\nVendor-specific columns"]
    end

    OBD_B -- "e.g. PID 0x0D\n= vehicle speed\non ALL vehicles" --> UNIVERSAL["Universal meaning"]
    CAN_B -- "e.g. signal 0x123\n= different per\nmake/model" --> SPECIFIC["Vendor-specific meaning"]

    style OBD_B fill:#dcedc8,stroke:#558b2f
    style CAN_B fill:#f0f4c3,stroke:#9e9d24

2.9 Optional Fields — I/O (Digital & Analog Inputs)

Raw electrical signals from the device’s input pins. These are hardware-level signals, not protocol data:

vehicle_voltage_v reads the vehicle electrical system voltage via the device’s power input. It is a key signal for determining whether the device is wired to a vehicle (> 9 V) or running on battery.

2.10 Vendor-Specific Extra Fields

Telemachus files MAY contain additional columns not defined in this specification. These columns MUST follow the naming convention:

x_<source>_<field>

Where <source> identifies the data provider or processing origin, and <field> is a descriptive snake_case name.

Examples:

Rules:

• Validators MUST ignore columns matching x_* (never reject them)
• Adapters SHOULD document their extra columns in the manifest
• Consumers MUST NOT assume any x_* column is present

2.11 Multi-Rate Convention

Telemachus files are timestamped at the highest sensor rate (typically IMU rate, e.g. 10–100 Hz). Lower-rate columns (GNSS at 1 Hz) contain NaN on rows where no measurement is available.

sequenceDiagram
    participant IMU as IMU (10 Hz)
    participant GPS as GNSS (1 Hz)
    participant REC as Record Row

    Note over IMU,REC: t = 0.0s
    IMU->>REC: ax=0.12, ay=0.03, az=9.81
    GPS->>REC: lat=49.33, lon=1.38, speed=5.2

    Note over IMU,REC: t = 0.1s
    IMU->>REC: ax=0.15, ay=-0.01, az=9.80
    GPS--xREC: NaN (no fix this tick)

    Note over IMU,REC: t = 0.2s
    IMU->>REC: ax=0.11, ay=0.02, az=9.82
    GPS--xREC: NaN

    Note over IMU,REC: ...

    Note over IMU,REC: t = 1.0s
    IMU->>REC: ax=0.13, ay=0.01, az=9.81
    GPS->>REC: lat=49.34, lon=1.38, speed=5.3

2.12 AccPeriod — Accelerometer Frame Reference

Commercial telematics devices may apply firmware-side gravity compensation. The same accelerometer can output data in different reference frames:

The accelerometer frame is declared at manifest level (see SPEC-02 §3.7), not per-row. Each AccPeriod is a contiguous time range with a coherent frame.

Default: if no AccPeriod is declared, consumers MUST assume raw.

graph TD
    subgraph FRAMES["Accelerometer Frame Types"]
        RAW["raw\n|a| at rest ≈ 9.81 m/s²\nGravity present in signal"]
        COMP["compensated\n|a| at rest ≈ 0 m/s²\nGravity removed by firmware"]
        PART["partial\n|a| at rest ≈ residual_g\n0 < residual < g\nImperfect compensation"]
    end

    RAW -- "firmware enables\ngravity filter" --> COMP
    RAW -- "firmware imperfect\ncalibration" --> PART

    subgraph EXAMPLES["Real-World Examples"]
        E1["AEGIS, PVS, STRIDE\n→ raw"]
        E2["Commercial device\nwith gravity filter ON\n→ compensated"]
        E3["Prototype device\nimperfect calibration\n→ partial (0.101 g residual)"]
    end

    RAW --- E1
    COMP --- E2
    PART --- E3

    style RAW fill:#e8f5e9,stroke:#2e7d32
    style COMP fill:#e3f2fd,stroke:#1565c0
    style PART fill:#fff3e0,stroke:#e65100

2.13 Excluded Columns

The following columns MUST NOT appear as top-level columns in a Telemachus dataset. They represent enriched or derived data:

Clarification — ground truth vs. enrichment: Simulation ground truth (e.g. x_rs3_road_type, x_rs3_event) is allowed as vendor-specific extra columns (x_*). These are annotations attached to synthetic data for validation purposes, not enrichment derived from external sources. The exclusion rule applies only to top-level columns without the x_ prefix.

3. Validation Rules

A Telemachus file is valid if:

1. All mandatory columns for the declared profile (§2.2–2.3) are present with correct types. Default profile is imu if not declared
2. ts is monotonically increasing (strictly)
3. For profiles imu and full, per AccPeriod (SPEC-02 §3.7), |a| mean at rest matches the declared frame:
   • raw: ≈ 9.81 ± 1.0 m/s²
   • compensated: ≈ 0 ± 1.0 m/s²
   • partial: ≈ residual_g ± 0.05 g
   • “At rest” heuristic: rows where speed_mps < 0.5 m/s (when GPS available) OR where accel_norm_std < 0.3 m/s² over a 2-second sliding window. Implementations MAY use different thresholds but MUST document them
4. lat / lon are within [-90, 90] / [-180, 180] when not NaN
5. heading_deg is within [0, 360) when not NaN
6. speed_mps >= 0 when not NaN; speed_obd_mps >= 0 when not NaN
7. No excluded columns from §2.13 are present (columns with x_ prefix are always allowed)
8. All extra columns follow the x_<source>_<field> convention
9. All present columns (mandatory, recommended, and optional) MUST have correct data types as specified in §2. Type checking is not limited to mandatory columns
10. If gyro columns are present, all three (gx, gy, gz) MUST be present (no partial group). For profile full, they are mandatory
11. If magneto columns are present, all three (mx, my, mz) MUST be present (no partial group)
12. If device_id / trip_id are absent from columns, they MUST be resolvable from the manifest (SPEC-02 §4.1)

4. Hardware Mapping

4.1 Source Coverage Matrix

graph TD
    subgraph RESEARCH["Open Research Datasets"]
        AEGIS["AEGIS\n(BeagleBone, Austria)"]
        PVS["PVS\n(MPU-9250 ×3, Brazil)"]
        STRIDE["STRIDE\n(POCO X2, Bangladesh)"]
    end

    subgraph SIM["Simulation"]
        RS3["RoadSimulator3\n(synthetic)"]
    end

    subgraph GROUPS["Functional Groups Covered"]
        G_GPS["GNSS"]
        G_IMU["IMU (Accel)"]
        G_GYRO["IMU (Gyro)"]
        G_MAG["IMU (Magneto)"]
        G_OBD["OBD"]
    end

    AEGIS --> G_GPS & G_IMU & G_GYRO & G_OBD
    PVS --> G_GPS & G_IMU & G_GYRO & G_MAG
    STRIDE --> G_GPS & G_IMU & G_GYRO & G_MAG
    RS3 --> G_GPS & G_IMU & G_GYRO

    style RESEARCH fill:#e8f5e9,stroke:#2e7d32
    style SIM fill:#e3f2fd,stroke:#1565c0
    style GROUPS fill:#fff9c4,stroke:#f9a825

Commercial devices (GNSS + IMU + optional I/O and OBD) are supported via private adapters documented outside this specification.

4.2 Detailed Column Mapping — Open Datasets

Note: Column mappings for commercial/proprietary devices are documented in their respective private adapter modules, not in this public specification.

AEGIS (Zenodo 820576, Austria)

PVS (Kaggle, Curitiba)

STRIDE (Figshare, Rajshahi)

RoadSimulator3 (Synthetic)

Note: RS3 also exports road_type, event, target_speed — these are ground truth metadata for validation, NOT part of a Telemachus record. They should be stored as x_rs3_* extra columns or in a sidecar file.

5. Unit Conversion Reference

Adapters MUST convert raw device units to Telemachus canonical units:

6. Python API — Sensor Introspection

The telemachus-py library provides introspection helpers for consumers to discover what data is available without loading the full dataset:

6.1 Manifest-Level (fast, no data loaded)

ds = tele.Dataset.from_manifest("manifest.yaml")
ds.profile()             # → "core" | "imu" | "full"
ds.declared_sensors()    # → {'gps': {'rate_hz': 1}, 'accelerometer': {...}, ...}
ds.has_declared_gyro()   # → True / False
ds.acc_frame()           # → "raw" | "compensated" | "partial"

6.2 Data-Level (loads parquet, checks actual content)

df = tele.read("manifest.yaml")
tele.has_gps(df)         # → True if lat, lon, speed_mps have non-NaN values
tele.has_imu(df)         # → True if ax, ay, az have non-NaN values
tele.has_gyro(df)        # → True if gx, gy, gz present and non-NaN
tele.has_magneto(df)     # → True if mx, my, mz present and non-NaN
tele.has_obd(df)         # → True if speed_obd_mps or rpm present and non-NaN
tele.has_io(df)          # → True if ignition or vehicle_voltage_v present
tele.sensor_profile(df)  # → "gps+imu+gyro+magneto" or "gps+imu+obd" etc.
tele.is_gps_only(df)     # → GPS but no IMU
tele.is_full_imu(df)     # → accel + gyro available

7. References

• SPEC-02: Dataset Manifest — canonical file-level metadata
• SPEC-03: Adapters & Validation — tooling and conformance testing
• Superseded RFCs: RFC-0001 (Core v0.2), RFC-0004 (Extended FieldGroups), RFC-0013 (Device Layer v0.7)

Dataset References

End of SPEC-01.