[wpiutil] Add high speed data logging

wpiutil/CMakeLists.txt

@@ -5,7 +5,7 @@ include(GenResources)
 include(CompileWarnings)
 include(AddTest)
 
-file(GLOB wpiutil_jni_src src/main/native/cpp/jni/WPIUtilJNI.cpp)
+file(GLOB wpiutil_jni_src src/main/native/cpp/jni/WPIUtilJNI.cpp src/main/native/cpp/jni/DataLogJNI.cpp)
 
 # Java bindings
 if (WITH_JAVA)

wpiutil/build.gradle

@@ -293,8 +293,20 @@ model {
     }
 }
 
+sourceSets {
+    printlog
+}
+
+task runPrintLog(type: JavaExec) {
+    classpath = sourceSets.printlog.runtimeClasspath
+
+    mainClass = 'printlog.PrintLog'
+}
+
 dependencies {
     api "com.fasterxml.jackson.core:jackson-annotations:2.12.4"
     api "com.fasterxml.jackson.core:jackson-core:2.12.4"
     api "com.fasterxml.jackson.core:jackson-databind:2.12.4"
+
+    printlogImplementation sourceSets.main.output
 }

wpiutil/doc/datalog.adoc

@@ -0,0 +1,192 @@
+= WPILib Data Log File Format Specification, Version 1.0
+WPILib Developers <wpilib@wpi.edu>
+Revision 1.0 (0x0100), 1/2/2022
+:toc:
+:toc-placement: preamble
+:sectanchors:
+
+A simple binary logging format designed for high speed logging of timestamped data values (e.g. numeric sensor values).
+
+[[motivation]]
+== Motivation
+
+FRC robots generate a lot of real-time data of various data types (most typically numeric values, but also strings and more complex data such as camera images). While there is good support in the WPILib software ecosystem for real-time display of this data on dashboards (via NetworkTables), there is no current standard for logging of timestamped data values for offline analysis. In the absence of a standard, teams have developed various ad-hoc solutions, including CSV files.
+
+Similar to how the NetworkTables standard protocol enabled ecosystem development of multiple dashboards, it is expected that a standard format for data logging, in combination with built-in library support for easy on-robot logging of robot data, will facilitate ecosystem development of offline data analysis tools.
+
+Standard logging facilities currently available (e.g. in Java) are generally designed for string messages, not for binary data. As a binary format, this will be faster with less overhead for the typical data (e.g. numbers) coming from a robot in real time (at least dozens, if not hundreds, of data points every ~20 ms). String formatting is significantly more expensive than just copying the binary number, and is more difficult to read back into analysis tools. Latency of the actual logging to the main program is also critical in the FRC application. WPILib also needs a common logging format and implementation across both C++ and Java.
+
+[[references]]
+== References
+
+[[rfc7159,RFC7159,JSON]]
+* RFC 7159, The JavaScript Object Notation (JSON) Data Interchange Format, https://tools.ietf.org/html/rfc7159
+
+[[definitions]]
+== Definitions
+
+[[def-entry]]
+Entry:: A data channel identified by an integer ID and a string name. Entries have a specified type and may have associated metadata.
+
+[[def-entry-id]]
+Entry ID:: An unsigned 4-byte ID by which records in the log refer to an Entry, instead of using the full string key for the Entry. Entry ID 0 is reserved for control records.
+
+[[def-record]]
+Record:: Storage of a single timestamped data item in the log. A record consists of the entry ID, the data length, a 64-bit integer timestamp, and the data contents.
+
+[[def-timestamp]]
+Timestamp:: 64-bit integer microseconds. The zero time is not specified by the data format, but on an FRC robot is typically the time the robot program started.
+
+[[design]]
+== Design
+
+A data log starts with an 8-byte header, followed by 0 or more records. There is no padding between records. Each record in the file has an arbitrary length payload, is timestamped, and is associated to a particular entry (via its entry ID).
+
+Entries in the log are started using a <<control-start,Start>> control record to associate an entry ID with its name, <<data-types,type>>, and <<metadata,metadata>>. Following the Start control record, records referencing that entry ID can follow in any order (and can be mixed with records with other entry IDs). A <<control-finish,Finish>> control record may be used to indicate no further records with that entry ID will follow. Following a Finish control record, an entry ID may be reused by another Start control record. Multiple Start control records for a single entry ID without an intervening Finish control record have unspecified behavior.
+
+Entry metadata may be updated with a <<control-set-metadata,Set Metadata>> control record. This control record is only valid for a particular entry ID in between Start and Finish control records for that entry ID. The Set Metadata control record should be interpreted as completely replacing the entry's metadata contents.
+
+There is no timestamp ordering requirement for records. This is true for control records as well--a Start control record with a later timestamp may be followed by data records for that entry with earlier timestamps.
+
+Duplicate entry names should be avoided, but there is nothing in the data format itself that requires this.
+
+All values are stored in little endian order.
+
+[[header]]
+=== Header
+
+The header consists of:
+
+* 6-byte ASCII string, containing "WPILOG"
+* 2-byte (16-bit) version number
+* 4-byte (32-bit) length of extra header string
+* extra header string (arbitrary length)
+
+The most significant byte of the version indicates the major version and the least significant byte indicates the minor version. For this version of the data format, the value is thus 0x0100, indicating version 1.0.
+
+The extra header string has arbitrary contents (e.g. the contents are set by the application that wrote the data log) but it must be UTF-8 encoded.
+
+The entire header for a version 1.0 file with no extra header string will be `57 50 49 4c 4f 47 00 01 00 00 00 00`.
+
+[[record]]
+=== Records
+
+Each record consists of:
+
+* 1-byte header length bitfield
+* 1 to 4-byte (32-bit) entry ID
+* 1 to 4-byte (32-bit) payload size (in bytes)
+* 1 to 8-byte (64-bit) timestamp (in integer microseconds)
+* payload data (arbitrary length)
+
+The header length bitfield encodes the length of each header field as follows (starting from the least significant bit):
+
+* 2-bit entry ID length (00 = 1 byte, 01 = 2 bytes, 10 = 3 bytes, 11 = 4 bytes)
+* 2-bit payload size length (00 = 1 byte, to 11 = 4 bytes)
+* 3-bit timestamp length (000 = 1 byte, to 111 = 8 bytes)
+* 1-bit spare (zero)
+
+An example record for a integer entry (ID=1) value 3 at timestamp 1 second would be 14 bytes in total length:
+
+* `20` (ID length = 1 byte, payload size length = 1 byte, timestamp length = 3 bytes)
+* `01` (entry ID = 1)
+* `08` (payload size = 8 bytes)
+* `40 42 0f` (timestamp = 1,000,000 us)
+* `03 00 00 00 00 00 00 00` (value = 3)
+
+[[control-record]]
+=== Control Records
+
+Entry ID 0 is used to indicate a record is a control record. There are 3 control record types: Start, Finish, and Set metadata. The first 4 bytes of the payload data indicates the control record type.
+
+[[control-start]]
+==== Start
+
+The Start control record provides information about the specified entry ID. It must appear prior to any records using that entry ID. The format of the Start control record's payload data is as follows:
+
+* 1-byte control record type (0 for Start control records)
+* 4-byte (32-bit) entry ID of entry being started
+* 4-byte (32-bit) length of entry name string
+* entry name UTF-8 string data (arbitrary length)
+* 4-byte (32-bit) length of entry type string
+* entry <<data-types,type>> UTF-8 string data (arbitrary length)
+* 4-byte (32-bit) length of entry metadata string
+* entry <<metadata,metadata>> UTF-8 string data (arbitrary length)
+
+An example start control record for an integer entry named `test` with ID=1 is 32 bytes:
+
+* `20` (ID length = 1 byte, payload size length = 1 byte, timestamp length = 3 bytes)
+* `00` (entry ID = 0)
+* `1a` (payload size = 26 bytes)
+* `40 42 0f` (timestamp = 1,000,000 us)
+* `00` (control record type = Start (0))
+* `01 00 00 00` (entry ID 1 being started)
+* `04 00 00 00` (length of name string = 4)
+* `74 65 73 74` (entry name = `test`)
+* `05 00 00 00` (length of type string = 5)
+* `69 6e 74 66 64` (type string = `int64`)
+* `00 00 00 00` (length of metadata string = 0)
+
+[[control-finish]]
+==== Finish
+
+The Finish control record indicates the entry ID is no longer valid.  The format of the Finish control record's payload data is as follows:
+
+* 1-byte control record type (1 for Finish control records)
+* 4-byte (32-bit) entry ID of entry being completed
+
+An example finish control record for ID=1 is 11 bytes:
+
+* `20` (ID length = 1 byte, payload size length = 1 byte, timestamp length = 3 bytes)
+* `00` (entry ID = 0)
+* `05` (payload size = 5 bytes)
+* `40 42 0f` (timestamp = 1,000,000 us)
+* `01` (control record type = Finish (1))
+* `01 00 00 00` (entry ID 1 being finished)
+
+[[control-set-metadata]]
+==== Set Metadata
+
+The Set metadata control record updates the <<metadata,metadata>> for an entry. The format of the record's payload data is as follows:
+
+* 1-byte control record type (2 for Set metadata control records)
+* 4-byte (32-bit) entry ID of entry whose metadata is being updated
+* 4-byte (32-bit) length of entry metadata string
+* entry metadata string data (arbitrary length)
+
+An example set metadata control record to set metadata for ID=1 is 30 bytes:
+
+* `20` (ID length = 1 byte, payload size length = 1 byte, timestamp length = 3 bytes)
+* `00` (entry ID = 0)
+* `18` (payload size = 24 bytes)
+* `40 42 0f` (timestamp = 1,000,000 us)
+* `02` (control record type = Set Metadata (2))
+* `01 00 00 00` (setting metadata for entry ID 1)
+* `0f 00 00 00` (length of metadata string = 15)
+* `7b 22 73 6f 75 72 63 65 22 3a 22 4e 54 22 7d` (metadata string = `{"source":"NT"}`)
+
+[[data-types]]
+=== Data Types
+
+Each entry's data type is an arbitrary string. The following data types are standard and should be supported by all implementations, but other data type strings are allowed and may be supported by some implementations.
+
+[cols="1,1,3", options="header"]
+|===
+|Type String|Description|Payload Data Contents
+|`raw`|raw data|the raw data
+|`boolean`|boolean|single byte (0=false, 1=true)
+|`int64`|integer|8-byte (64-bit) signed value
+|`float`|float|4-byte (32-bit) IEEE-754 value
+|`double`|double|8-byte (64-bit) IEEE-754 value
+|`string`|string|UTF-8 encoded string data
+|`boolean[]`|array of boolean|a single byte (0=false, 1=true) for each entry in the arrayfootnote:arraylength[the array length is not stored, but is instead determined by the payload length]
+|`int64[]`|array of integer|8-byte (64-bit) signed value for each entry in the arrayfootnote:arraylength[]
+|`float[]`|array of float|4-byte (32-bit) value for each entry in the arrayfootnote:arraylength[]
+|`double[]`|array of double|8-byte (64-bit) value for each entry in the arrayfootnote:arraylength[]
+|`string[]`|array of strings|Starts with a 4-byte (32-bit) array length. Each string is stored as a 4-byte (32-bit) length followed by the UTF-8 string data
+|===
+
+[[metadata]]
+=== Metadata
+
+Each entry has an associated metadata string. If not blank, the metadata should be <<JSON,JSON>>, but may be arbitrary text. Metadata is intended to convey additional information about the entry beyond what the type conveys--for example the source of the data.

wpiutil/examples/printlog/datalog.py

@@ -0,0 +1,340 @@
+#! /usr/bin/env python3
+# Copyright (c) FIRST and other WPILib contributors.
+# Open Source Software; you can modify and/or share it under the terms of
+# the WPILib BSD license file in the root directory of this project.
+
+import array
+import struct
+from typing import List, SupportsBytes
+
+__all__ = [
+    "StartRecordData", "MetadataRecordData", "DataLogRecord", "DataLogReader"
+]
+
+floatStruct = struct.Struct("<f")
+doubleStruct = struct.Struct("<d")
+
+kControlStart = 0
+kControlFinish = 1
+kControlSetMetadata = 2
+
+
+class StartRecordData:
+    """Data contained in a start control record as created by DataLog.start() when
+    writing the log. This can be read by calling DataLogRecord.getStartData().
+
+    entry: Entry ID; this will be used for this entry in future records.
+    name: Entry name.
+    type: Type of the stored data for this entry, as a string, e.g. "double".
+    metadata: Initial metadata.
+    """
+
+    def __init__(self, entry: int, name: str, type: str, metadata: str):
+        self.entry = entry
+        self.name = name
+        self.type = type
+        self.metadata = metadata
+
+
+class MetadataRecordData:
+    """Data contained in a set metadata control record as created by
+    DataLog.setMetadata(). This can be read by calling
+    DataLogRecord.getSetMetadataData().
+
+    entry: Entry ID.
+    metadata: New metadata for the entry.
+    """
+
+    def __init__(self, entry: int, metadata: str):
+        self.entry = entry
+        self.metadata = metadata
+
+
+class DataLogRecord:
+    """A record in the data log. May represent either a control record
+    (entry == 0) or a data record."""
+
+    def __init__(self, entry: int, timestamp: int, data: SupportsBytes):
+        self.entry = entry
+        self.timestamp = timestamp
+        self.data = data
+
+    def isControl(self) -> bool:
+        return self.entry == 0
+
+    def _getControlType(self) -> int:
+        return self.data[0]
+
+    def isStart(self) -> bool:
+        return (self.entry == 0 and len(self.data) >= 17 and
+                self._getControlType() == kControlStart)
+
+    def isFinish(self) -> bool:
+        return (self.entry == 0 and len(self.data) == 5 and
+                self._getControlType() == kControlFinish)
+
+    def isSetMetadata(self) -> bool:
+        return (self.entry == 0 and len(self.data) >= 9 and
+                self._getControlType() == kControlSetMetadata)
+
+    def getStartData(self) -> StartRecordData:
+        if not self.isStart():
+            raise TypeError("not a start record")
+        entry = int.from_bytes(self.data[1:5], byteorder="little", signed=False)
+        name, pos = self._readInnerString(5)
+        type, pos = self._readInnerString(pos)
+        metadata = self._readInnerString(pos)[0]
+        return StartRecordData(entry, name, type, metadata)
+
+    def getFinishEntry(self) -> int:
+        if not self.isFinish():
+            raise TypeError("not a finish record")
+        return int.from_bytes(self.data[1:5], byteorder="little", signed=False)
+
+    def getSetMetadataData(self) -> MetadataRecordData:
+        if not self.isSetMetadata():
+            raise TypeError("not a finish record")
+        entry = int.from_bytes(self.data[1:5], byteorder="little", signed=False)
+        metadata = self._readInnerString(5)[0]
+        return MetadataRecordData(entry, metadata)
+
+    def getBoolean(self) -> bool:
+        if len(self.data) != 1:
+            raise TypeError("not a boolean")
+        return self.data[0] != 0
+
+    def getInteger(self) -> int:
+        if len(self.data) != 8:
+            raise TypeError("not an integer")
+        return int.from_bytes(self.data, byteorder="little", signed=True)
+
+    def getFloat(self) -> float:
+        if len(self.data) != 4:
+            raise TypeError("not a float")
+        return floatStruct.unpack(self.data)[0]
+
+    def getDouble(self) -> float:
+        if len(self.data) != 8:
+            raise TypeError("not a double")
+        return doubleStruct.unpack(self.data)[0]
+
+    def getString(self) -> str:
+        return str(self.data, encoding="utf-8")
+
+    def getBooleanArray(self) -> List[bool]:
+        return [x != 0 for x in self.data]
+
+    def getIntegerArray(self) -> array.array:
+        if (len(self.data) % 8) != 0:
+            raise TypeError("not an integer array")
+        arr = array.array("l")
+        arr.frombytes(self.data)
+        return arr
+
+    def getFloatArray(self) -> array.array:
+        if (len(self.data) % 4) != 0:
+            raise TypeError("not a float array")
+        arr = array.array("f")
+        arr.frombytes(self.data)
+        return arr
+
+    def getDoubleArray(self) -> array.array:
+        if (len(self.data) % 8) != 0:
+            raise TypeError("not a double array")
+        arr = array.array("d")
+        arr.frombytes(self.data)
+        return arr
+
+    def getStringArray(self) -> List[str]:
+        size = int.from_bytes(self.data[0:4], byteorder="little", signed=False)
+        if size > ((len(self.data) - 4) / 4):
+            raise TypeError("not a string array")
+        arr = []
+        pos = 4
+        for i in range(size):
+            val, pos = self._readInnerString(pos)
+            arr.append(val)
+        return arr
+
+    def _readInnerString(self, pos: int) -> str:
+        size = int.from_bytes(self.data[pos:pos + 4],
+                              byteorder="little",
+                              signed=False)
+        end = pos + 4 + size
+        if end > len(self.data):
+            raise TypeError("invalid string size")
+        return str(self.data[pos + 4:end], encoding="utf-8"), end
+
+
+class DataLogIterator:
+    """DataLogReader iterator."""
+
+    def __init__(self, buf: SupportsBytes, pos: int):
+        self.buf = buf
+        self.pos = pos
+
+    def __iter__(self):
+        return self
+
+    def _readVarInt(self, pos: int, len: int) -> int:
+        val = 0
+        for i in range(len):
+            val |= self.buf[pos + i] << (i * 8)
+        return val
+
+    def __next__(self) -> DataLogRecord:
+        if len(self.buf) < (self.pos + 4):
+            raise StopIteration
+        entryLen = (self.buf[self.pos] & 0x3) + 1
+        sizeLen = ((self.buf[self.pos] >> 2) & 0x3) + 1
+        timestampLen = ((self.buf[self.pos] >> 4) & 0x7) + 1
+        headerLen = 1 + entryLen + sizeLen + timestampLen
+        if len(self.buf) < (self.pos + headerLen):
+            raise StopIteration
+        entry = self._readVarInt(self.pos + 1, entryLen)
+        size = self._readVarInt(self.pos + 1 + entryLen, sizeLen)
+        timestamp = self._readVarInt(self.pos + 1 + entryLen + sizeLen,
+                                     timestampLen)
+        if len(self.buf) < (self.pos + headerLen + size):
+            raise StopIteration
+        record = DataLogRecord(
+            entry, timestamp,
+            self.buf[self.pos + headerLen:self.pos + headerLen + size])
+        self.pos += headerLen + size
+        return record
+
+
+class DataLogReader:
+    """Data log reader (reads logs written by the DataLog class)."""
+
+    def __init__(self, buf: SupportsBytes):
+        self.buf = buf
+
+    def __bool__(self):
+        return self.isValid()
+
+    def isValid(self) -> bool:
+        """Returns true if the data log is valid (e.g. has a valid header)."""
+        return (len(self.buf) >= 12 and self.buf[0:6] == b"WPILOG" and
+                self.getVersion() >= 0x0100)
+
+    def getVersion(self) -> int:
+        """Gets the data log version. Returns 0 if data log is invalid.
+
+        @return Version number; most significant byte is major, least significant is
+            minor (so version 1.0 will be 0x0100)"""
+        if len(self.buf) < 12:
+            return 0
+        return int.from_bytes(self.buf[6:8], byteorder="little", signed=False)
+
+    def getExtraHeader(self) -> str:
+        """Gets the extra header data.
+
+        @return Extra header data
+        """
+        if len(self.buf) < 12:
+            return ""
+        size = int.from_bytes(self.buf[8:12], byteorder="little", signed=False)
+        return str(self.buf[12:12 + size], encoding="utf-8")
+
+    def __iter__(self) -> DataLogIterator:
+        extraHeaderSize = int.from_bytes(self.buf[8:12],
+                                         byteorder="little",
+                                         signed=False)
+        return DataLogIterator(self.buf, 12 + extraHeaderSize)
+
+
+if __name__ == "__main__":
+    from datetime import datetime
+    import mmap
+    import sys
+
+    if len(sys.argv) != 2:
+        print("Usage: datalog.py <file>", file=sys.stderr)
+        sys.exit(1)
+
+    with open(sys.argv[1], "r") as f:
+        mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
+        reader = DataLogReader(mm)
+        if not reader:
+            print("not a log file", file=sys.stderr)
+            sys.exit(1)
+
+        entries = {}
+        for record in reader:
+            timestamp = record.timestamp / 1000000
+            if record.isStart():
+                try:
+                    data = record.getStartData()
+                    print(
+                        f"Start({data.entry}, name='{data.name}', type='{data.type}', metadata='{data.metadata}') [{timestamp}]"
+                    )
+                    if data.entry in entries:
+                        print("...DUPLICATE entry ID, overriding")
+                    entries[data.entry] = data
+                except TypeError as e:
+                    print("Start(INVALID)")
+            elif record.isFinish():
+                try:
+                    entry = record.getFinishEntry()
+                    print(f"Finish({entry}) [{timestamp}]")
+                    if entry not in entries:
+                        print("...ID not found")
+                    else:
+                        del entries[entry]
+                except TypeError as e:
+                    print("Finish(INVALID)")
+            elif record.isSetMetadata():
+                try:
+                    data = record.getSetMetadataData()
+                    print(
+                        f"SetMetadata({data.entry}, '{data.metadata}') [{timestamp}]"
+                    )
+                    if data.entry not in entries:
+                        print("...ID not found")
+                except TypeError as e:
+                    print("SetMetadata(INVALID)")
+            elif record.isControl():
+                print("Unrecognized control record")
+            else:
+                print(f"Data({record.entry}, size={len(record.data)}) ", end="")
+                entry = entries.get(record.entry, None)
+                if entry is None:
+                    print("<ID not found>")
+                    continue
+                print(
+                    f"<name='{entry.name}', type='{entry.type}'> [{timestamp}]")
+
+                try:
+                    # handle systemTime specially
+                    if entry.name == "systemTime" and entry.type == "int64":
+                        dt = datetime.fromtimestamp(record.getInteger() /
+                                                    1000000)
+                        print("  {:%Y-%m-%d %H:%M:%S.%f}".format(dt))
+                        continue
+
+                    if entry.type == "double":
+                        print(f"  {record.getDouble()}")
+                    elif entry.type == "int64":
+                        print(f"  {record.getInteger()}")
+                    elif entry.type == "string" or entry.type == "json":
+                        print(f"  '{record.getString()}'")
+                    elif entry.type == "boolean":
+                        print(f"  {record.getBoolean()}")
+                    elif entry.type == "boolean[]":
+                        arr = record.getBooleanArray()
+                        print(f"  {arr}")
+                    elif entry.type == "double[]":
+                        arr = record.getDoubleArray()
+                        print(f"  {arr}")
+                    elif entry.type == "float[]":
+                        arr = record.getFloatArray()
+                        print(f"  {arr}")
+                    elif entry.type == "int64[]":
+                        arr = record.getIntegerArray()
+                        print(f"  {arr}")
+                    elif entry.type == "string[]":
+                        arr = record.getStringArray()
+                        print(f"  {arr}")
+                except TypeError as e:
+                    print("  invalid")

wpiutil/examples/printlog/printlog.cpp

@@ -0,0 +1,154 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+#include <ctime>
+
+#include "fmt/chrono.h"
+#include "fmt/format.h"
+#include "wpi/DataLogReader.h"
+#include "wpi/DenseMap.h"
+#include "wpi/MemoryBuffer.h"
+
+int main(int argc, const char** argv) {
+  if (argc != 2) {
+    fmt::print(stderr, "Usage: printlog <file>\n");
+    return EXIT_FAILURE;
+  }
+  std::error_code ec;
+  wpi::log::DataLogReader reader{wpi::MemoryBuffer::GetFile(argv[1], ec)};
+  if (ec) {
+    fmt::print(stderr, "could not open file: {}\n", ec.message());
+    return EXIT_FAILURE;
+  }
+  if (!reader) {
+    fmt::print(stderr, "not a log file\n");
+    return EXIT_FAILURE;
+  }
+
+  wpi::DenseMap<int, wpi::log::StartRecordData> entries;
+  for (auto&& record : reader) {
+    if (record.IsStart()) {
+      wpi::log::StartRecordData data;
+      if (record.GetStartData(&data)) {
+        fmt::print("Start({}, name='{}', type='{}', metadata='{}') [{}]\n",
+                   data.entry, data.name, data.type, data.metadata,
+                   record.GetTimestamp() / 1000000.0);
+        if (entries.find(data.entry) != entries.end()) {
+          fmt::print("...DUPLICATE entry ID, overriding\n");
+        }
+        entries[data.entry] = data;
+      } else {
+        fmt::print("Start(INVALID)\n");
+      }
+    } else if (record.IsFinish()) {
+      int entry;
+      if (record.GetFinishEntry(&entry)) {
+        fmt::print("Finish({}) [{}]\n", entry,
+                   record.GetTimestamp() / 1000000.0);
+        auto it = entries.find(entry);
+        if (it == entries.end()) {
+          fmt::print("...ID not found\n");
+        } else {
+          entries.erase(it);
+        }
+      } else {
+        fmt::print("Finish(INVALID)\n");
+      }
+    } else if (record.IsSetMetadata()) {
+      wpi::log::MetadataRecordData data;
+      if (record.GetSetMetadataData(&data)) {
+        fmt::print("SetMetadata({}, '{}') [{}]\n", data.entry, data.metadata,
+                   record.GetTimestamp() / 1000000.0);
+        auto it = entries.find(data.entry);
+        if (it == entries.end()) {
+          fmt::print("...ID not found\n");
+        } else {
+          it->second.metadata = data.metadata;
+        }
+      } else {
+        fmt::print("SetMetadata(INVALID)\n");
+      }
+    } else if (record.IsControl()) {
+      fmt::print("Unrecognized control record\n");
+    } else {
+      fmt::print("Data({}, size={}) ", record.GetEntry(), record.GetSize());
+      auto entry = entries.find(record.GetEntry());
+      if (entry == entries.end()) {
+        fmt::print("<ID not found>\n");
+        continue;
+      }
+      fmt::print("<name='{}', type='{}'> [{}]\n", entry->second.name,
+                 entry->second.type, record.GetTimestamp() / 1000000.0);
+
+      // handle systemTime specially
+      if (entry->second.name == "systemTime" && entry->second.type == "int64") {
+        int64_t val;
+        if (record.GetInteger(&val)) {
+          std::time_t timeval = val / 1000000;
+          fmt::print("  {:%Y-%m-%d %H:%M:%S}.{:06}\n",
+                     *std::localtime(&timeval), val % 1000000);
+        } else {
+          fmt::print("  invalid\n");
+        }
+        continue;
+      }
+
+      if (entry->second.type == "double") {
+        double val;
+        if (record.GetDouble(&val)) {
+          fmt::print("  {}\n", val);
+        } else {
+          fmt::print("  invalid\n");
+        }
+      } else if (entry->second.type == "int64") {
+        int64_t val;
+        if (record.GetInteger(&val)) {
+          fmt::print("  {}\n", val);
+        } else {
+          fmt::print("  invalid\n");
+        }
+      } else if (entry->second.type == "string" ||
+                 entry->second.type == "json") {
+        std::string_view val;
+        record.GetString(&val);
+        fmt::print("  '{}'\n", val);
+      } else if (entry->second.type == "boolean") {
+        bool val;
+        if (record.GetBoolean(&val)) {
+          fmt::print("  {}\n", val);
+        } else {
+          fmt::print("  invalid\n");
+        }
+      } else if (entry->second.type == "double[]") {
+        std::vector<double> val;
+        if (record.GetDoubleArray(&val)) {
+          fmt::print("  {}\n", fmt::join(val, ", "));
+        } else {
+          fmt::print("  invalid\n");
+        }
+      } else if (entry->second.type == "float[]") {
+        std::vector<float> val;
+        if (record.GetFloatArray(&val)) {
+          fmt::print("  {}\n", fmt::join(val, ", "));
+        } else {
+          fmt::print("  invalid\n");
+        }
+      } else if (entry->second.type == "int64[]") {
+        std::vector<int64_t> val;
+        if (record.GetIntegerArray(&val)) {
+          fmt::print("  {}\n", fmt::join(val, ", "));
+        } else {
+          fmt::print("  invalid\n");
+        }
+      } else if (entry->second.type == "string[]") {
+        std::vector<std::string_view> val;
+        if (record.GetStringArray(&val)) {
+          fmt::print("  {}\n", fmt::join(val, ", "));
+        } else {
+          fmt::print("  invalid\n");
+        }
+      }
+    }
+  }
+}

wpiutil/examples/writelog/writelog.cpp

@@ -0,0 +1,82 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+#include <array>
+#include <chrono>
+#include <numeric>
+#include <string>
+#include <vector>
+
+#include "fmt/format.h"
+#include "wpi/DataLog.h"
+
+int main(int argc, char** argv) {
+  using std::chrono::duration_cast;
+  using std::chrono::high_resolution_clock;
+  using std::chrono::microseconds;
+
+  int kNumRuns = 10;
+
+  if (argc == 2) {
+    kNumRuns = std::stoi(argv[1]);
+  }
+
+  wpi::log::DataLog log;
+  log.SetFilename("test.wpilog");
+
+  auto testVec =
+      std::vector<std::pair<std::string, void (*)(wpi::log::DataLog&)>>();
+
+  testVec.push_back({"50 double append", [](auto& log) {
+                       wpi::log::DoubleLogEntry entry{log, "fifty", 1};
+                       for (int i = 0; i < 50; ++i) {
+                         entry.Append(1.3 * i, 20000 * i);
+                       }
+                     }});
+#if 0
+  testVec.push_back({"500k double append", [](auto& log) {
+                       wpi::log::DoubleLogEntry entry{log, "500k", 1};
+                       for (uint64_t i = 0; i < 500000; ++i) {
+                         entry.Append(1.3 * i, 20000 * i);
+                       }
+                     }});
+#endif
+  testVec.push_back({"50 string append", [](auto& log) {
+                       wpi::log::StringLogEntry entry{log, "string", 1};
+                       for (int i = 0; i < 50; ++i) {
+                         entry.Append("hello", 20000 * i);
+                       }
+                     }});
+
+  testVec.push_back(
+      {"Double array append", [](auto& log) {
+         wpi::log::DoubleArrayLogEntry entry{log, "double_array", 1};
+         entry.Append({1, 2, 3}, 20000);
+         entry.Append({4, 5}, 30000);
+       }});
+
+  testVec.push_back(
+      {"String array append", [](auto& log) {
+         wpi::log::StringArrayLogEntry entry{log, "string_array", 1};
+         entry.Append({"Hello", "World"}, 20000);
+         entry.Append({"This", "Is", "Fun"}, 30000);
+       }});
+
+  for (const auto& [name, fn] : testVec) {
+    auto resVec = std::vector<microseconds::rep>();
+    fmt::print("{}: ", name);
+
+    for (int i = 0; i < kNumRuns; ++i) {
+      auto start = high_resolution_clock::now();
+      fn(log);
+      auto stop = high_resolution_clock::now();
+      resVec.push_back(duration_cast<microseconds>(stop - start).count());
+    }
+
+    fmt::print("{}us\n",
+               std::accumulate(resVec.begin(), resVec.end(), 0) / kNumRuns);
+  }
+
+  return EXIT_SUCCESS;
+}

wpiutil/src/main/java/edu/wpi/first/util/WPIUtilJNI.java

@@ -7,7 +7,7 @@ package edu.wpi.first.util;
 import java.io.IOException;
 import java.util.concurrent.atomic.AtomicBoolean;
 
-public final class WPIUtilJNI {
+public class WPIUtilJNI {
   static boolean libraryLoaded = false;
   static RuntimeLoader<WPIUtilJNI> loader = null;
 

wpiutil/src/main/java/edu/wpi/first/util/datalog/BooleanArrayLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log array of boolean values. */
+public class BooleanArrayLogEntry extends DataLogEntry {
+  public static final String kDataType = "boolean[]";
+
+  public BooleanArrayLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public BooleanArrayLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public BooleanArrayLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public BooleanArrayLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(boolean[] value, long timestamp) {
+    m_log.appendBooleanArray(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(boolean[] value) {
+    m_log.appendBooleanArray(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/BooleanLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log boolean values. */
+public class BooleanLogEntry extends DataLogEntry {
+  public static final String kDataType = "boolean";
+
+  public BooleanLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public BooleanLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public BooleanLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public BooleanLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(boolean value, long timestamp) {
+    m_log.appendBoolean(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(boolean value) {
+    m_log.appendBoolean(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DataLog.java

@@ -0,0 +1,242 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/**
+ * A data log. The log file is created immediately upon construction with a temporary filename. The
+ * file may be renamed at any time using the setFilename() function.
+ *
+ * <p>The data log is periodically flushed to disk. It can also be explicitly flushed to disk by
+ * using the flush() function.
+ */
+@SuppressWarnings({"PMD.TooManyMethods", "PMD.ExcessivePublicCount"})
+public final class DataLog implements AutoCloseable {
+  /**
+   * Construct a new Data Log. The log will be initially created with a temporary filename.
+   *
+   * @param dir directory to store the log
+   * @param filename filename to use; if none provided, a random filename is generated of the form
+   *     "wpilog_{}.wpilog"
+   * @param period time between automatic flushes to disk, in seconds; this is a time/storage
+   *     tradeoff
+   * @param extraHeader extra header data
+   */
+  public DataLog(String dir, String filename, double period, String extraHeader) {
+    m_impl = DataLogJNI.create(dir, filename, period, extraHeader);
+  }
+
+  /**
+   * Construct a new Data Log. The log will be initially created with a temporary filename.
+   *
+   * @param dir directory to store the log
+   * @param filename filename to use; if none provided, a random filename is generated of the form
+   *     "wpilog_{}.wpilog"
+   * @param period time between automatic flushes to disk, in seconds; this is a time/storage
+   *     tradeoff
+   */
+  public DataLog(String dir, String filename, double period) {
+    this(dir, filename, period, "");
+  }
+
+  /**
+   * Construct a new Data Log. The log will be initially created with a temporary filename.
+   *
+   * @param dir directory to store the log
+   * @param filename filename to use; if none provided, a random filename is generated of the form
+   *     "wpilog_{}.wpilog"
+   */
+  public DataLog(String dir, String filename) {
+    this(dir, filename, 0.25);
+  }
+
+  /**
+   * Construct a new Data Log. The log will be initially created with a temporary filename.
+   *
+   * @param dir directory to store the log
+   */
+  public DataLog(String dir) {
+    this(dir, "", 0.25);
+  }
+
+  /** Construct a new Data Log. The log will be initially created with a temporary filename. */
+  public DataLog() {
+    this("");
+  }
+
+  /**
+   * Change log filename.
+   *
+   * @param filename filename
+   */
+  public void setFilename(String filename) {
+    DataLogJNI.setFilename(m_impl, filename);
+  }
+
+  /** Explicitly flushes the log data to disk. */
+  public void flush() {
+    DataLogJNI.flush(m_impl);
+  }
+
+  /**
+   * Pauses appending of data records to the log. While paused, no data records are saved (e.g.
+   * AppendX is a no-op). Has no effect on entry starts / finishes / metadata changes.
+   */
+  public void pause() {
+    DataLogJNI.pause(m_impl);
+  }
+
+  /** Resumes appending of data records to the log. */
+  public void resume() {
+    DataLogJNI.resume(m_impl);
+  }
+
+  /**
+   * Start an entry. Duplicate names are allowed (with the same type), and result in the same index
+   * being returned (start/finish are reference counted). A duplicate name with a different type
+   * will result in an error message being printed to the console and 0 being returned (which will
+   * be ignored by the append functions).
+   *
+   * @param name Name
+   * @param type Data type
+   * @param metadata Initial metadata (e.g. data properties)
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   * @return Entry index
+   */
+  public int start(String name, String type, String metadata, long timestamp) {
+    return DataLogJNI.start(m_impl, name, type, metadata, timestamp);
+  }
+
+  /**
+   * Start an entry. Duplicate names are allowed (with the same type), and result in the same index
+   * being returned (start/finish are reference counted). A duplicate name with a different type
+   * will result in an error message being printed to the console and 0 being returned (which will
+   * be ignored by the append functions).
+   *
+   * @param name Name
+   * @param type Data type
+   * @param metadata Initial metadata (e.g. data properties)
+   * @return Entry index
+   */
+  public int start(String name, String type, String metadata) {
+    return start(name, type, metadata, 0);
+  }
+
+  /**
+   * Start an entry. Duplicate names are allowed (with the same type), and result in the same index
+   * being returned (start/finish are reference counted). A duplicate name with a different type
+   * will result in an error message being printed to the console and 0 being returned (which will
+   * be ignored by the append functions).
+   *
+   * @param name Name
+   * @param type Data type
+   * @return Entry index
+   */
+  public int start(String name, String type) {
+    return start(name, type, "");
+  }
+
+  /**
+   * Finish an entry.
+   *
+   * @param entry Entry index
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void finish(int entry, long timestamp) {
+    DataLogJNI.finish(m_impl, entry, timestamp);
+  }
+
+  /**
+   * Finish an entry.
+   *
+   * @param entry Entry index
+   */
+  public void finish(int entry) {
+    finish(entry, 0);
+  }
+
+  /**
+   * Updates the metadata for an entry.
+   *
+   * @param entry Entry index
+   * @param metadata New metadata for the entry
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void setMetadata(int entry, String metadata, long timestamp) {
+    DataLogJNI.setMetadata(m_impl, entry, metadata, timestamp);
+  }
+
+  /**
+   * Updates the metadata for an entry.
+   *
+   * @param entry Entry index
+   * @param metadata New metadata for the entry
+   */
+  public void setMetadata(int entry, String metadata) {
+    setMetadata(entry, metadata, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param entry Entry index, as returned by Start()
+   * @param data Data to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void appendRaw(int entry, byte[] data, long timestamp) {
+    DataLogJNI.appendRaw(m_impl, entry, data, timestamp);
+  }
+
+  @Override
+  public void close() {
+    DataLogJNI.close(m_impl);
+    m_impl = 0;
+  }
+
+  public void appendBoolean(int entry, boolean value, long timestamp) {
+    DataLogJNI.appendBoolean(m_impl, entry, value, timestamp);
+  }
+
+  public void appendInteger(int entry, long value, long timestamp) {
+    DataLogJNI.appendInteger(m_impl, entry, value, timestamp);
+  }
+
+  public void appendFloat(int entry, float value, long timestamp) {
+    DataLogJNI.appendFloat(m_impl, entry, value, timestamp);
+  }
+
+  public void appendDouble(int entry, double value, long timestamp) {
+    DataLogJNI.appendDouble(m_impl, entry, value, timestamp);
+  }
+
+  public void appendString(int entry, String value, long timestamp) {
+    DataLogJNI.appendString(m_impl, entry, value, timestamp);
+  }
+
+  public void appendBooleanArray(int entry, boolean[] arr, long timestamp) {
+    DataLogJNI.appendBooleanArray(m_impl, entry, arr, timestamp);
+  }
+
+  public void appendIntegerArray(int entry, long[] arr, long timestamp) {
+    DataLogJNI.appendIntegerArray(m_impl, entry, arr, timestamp);
+  }
+
+  public void appendFloatArray(int entry, float[] arr, long timestamp) {
+    DataLogJNI.appendFloatArray(m_impl, entry, arr, timestamp);
+  }
+
+  public void appendDoubleArray(int entry, double[] arr, long timestamp) {
+    DataLogJNI.appendDoubleArray(m_impl, entry, arr, timestamp);
+  }
+
+  public void appendStringArray(int entry, String[] arr, long timestamp) {
+    DataLogJNI.appendStringArray(m_impl, entry, arr, timestamp);
+  }
+
+  public long getImpl() {
+    return m_impl;
+  }
+
+  private long m_impl;
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DataLogEntry.java

@@ -0,0 +1,57 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log entry base class. */
+public class DataLogEntry {
+  protected DataLogEntry(DataLog log, String name, String type, String metadata, long timestamp) {
+    m_log = log;
+    m_entry = log.start(name, type, metadata, timestamp);
+  }
+
+  protected DataLogEntry(DataLog log, String name, String type, String metadata) {
+    this(log, name, type, metadata, 0);
+  }
+
+  protected DataLogEntry(DataLog log, String name, String type) {
+    this(log, name, type, "");
+  }
+
+  /**
+   * Updates the metadata for the entry.
+   *
+   * @param metadata New metadata for the entry
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void setMetadata(String metadata, long timestamp) {
+    m_log.setMetadata(m_entry, metadata, timestamp);
+  }
+
+  /**
+   * Updates the metadata for the entry.
+   *
+   * @param metadata New metadata for the entry
+   */
+  public void setMetadata(String metadata) {
+    setMetadata(metadata, 0);
+  }
+
+  /**
+   * Finishes the entry.
+   *
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void finish(long timestamp) {
+    m_log.finish(m_entry, timestamp);
+  }
+
+  /** Finishes the entry. */
+  public void finish() {
+    finish(0);
+  }
+
+  protected final DataLog m_log;
+  protected final int m_entry;
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DataLogIterator.java

@@ -0,0 +1,46 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+import java.util.Iterator;
+import java.util.NoSuchElementException;
+import java.util.function.Consumer;
+
+/** DataLogReader iterator. */
+public class DataLogIterator implements Iterator<DataLogRecord> {
+  DataLogIterator(DataLogReader reader, int pos) {
+    m_reader = reader;
+    m_pos = pos;
+  }
+
+  @Override
+  public void forEachRemaining(Consumer<? super DataLogRecord> action) {
+    int size = m_reader.size();
+    for (; m_pos < size; m_pos = m_reader.getNextRecord(m_pos)) {
+      DataLogRecord record;
+      try {
+        record = m_reader.getRecord(m_pos);
+      } catch (NoSuchElementException ex) {
+        break;
+      }
+      action.accept(record);
+    }
+  }
+
+  @Override
+  public boolean hasNext() {
+    return (m_pos + 16) <= m_reader.size();
+  }
+
+  @Override
+  public DataLogRecord next() {
+    DataLogRecord record = m_reader.getRecord(m_pos);
+    m_pos = m_reader.getNextRecord(m_pos);
+    return record;
+  }
+
+  private final DataLogReader m_reader;
+  private int m_pos;
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DataLogJNI.java

@@ -0,0 +1,49 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+import edu.wpi.first.util.WPIUtilJNI;
+
+public class DataLogJNI extends WPIUtilJNI {
+  static native long create(String dir, String filename, double period, String extraHeader);
+
+  static native void setFilename(long impl, String filename);
+
+  static native void flush(long impl);
+
+  static native void pause(long impl);
+
+  static native void resume(long impl);
+
+  static native int start(long impl, String name, String type, String metadata, long timestamp);
+
+  static native void finish(long impl, int entry, long timestamp);
+
+  static native void setMetadata(long impl, int entry, String metadata, long timestamp);
+
+  static native void close(long impl);
+
+  static native void appendRaw(long impl, int entry, byte[] data, long timestamp);
+
+  static native void appendBoolean(long impl, int entry, boolean value, long timestamp);
+
+  static native void appendInteger(long impl, int entry, long value, long timestamp);
+
+  static native void appendFloat(long impl, int entry, float value, long timestamp);
+
+  static native void appendDouble(long impl, int entry, double value, long timestamp);
+
+  static native void appendString(long impl, int entry, String value, long timestamp);
+
+  static native void appendBooleanArray(long impl, int entry, boolean[] value, long timestamp);
+
+  static native void appendIntegerArray(long impl, int entry, long[] value, long timestamp);
+
+  static native void appendFloatArray(long impl, int entry, float[] value, long timestamp);
+
+  static native void appendDoubleArray(long impl, int entry, double[] value, long timestamp);
+
+  static native void appendStringArray(long impl, int entry, String[] value, long timestamp);
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DataLogReader.java

@@ -0,0 +1,157 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.BufferUnderflowException;
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.MappedByteBuffer;
+import java.nio.channels.FileChannel;
+import java.nio.charset.StandardCharsets;
+import java.util.NoSuchElementException;
+import java.util.function.Consumer;
+
+/** Data log reader (reads logs written by the DataLog class). */
+public class DataLogReader implements Iterable<DataLogRecord> {
+  /**
+   * Constructs from a byte buffer.
+   *
+   * @param buffer byte buffer
+   */
+  public DataLogReader(ByteBuffer buffer) {
+    m_buf = buffer;
+    m_buf.order(ByteOrder.LITTLE_ENDIAN);
+  }
+
+  /**
+   * Constructs from a file.
+   *
+   * @param filename filename
+   * @throws IOException if could not open/read file
+   */
+  public DataLogReader(String filename) throws IOException {
+    RandomAccessFile f = new RandomAccessFile(filename, "r");
+    FileChannel channel = f.getChannel();
+    MappedByteBuffer buf = channel.map(FileChannel.MapMode.READ_ONLY, 0, channel.size());
+    m_buf = buf;
+    m_buf.order(ByteOrder.LITTLE_ENDIAN);
+    channel.close();
+    f.close();
+  }
+
+  /**
+   * Returns true if the data log is valid (e.g. has a valid header).
+   *
+   * @return True if valid, false otherwise
+   */
+  public boolean isValid() {
+    return m_buf.remaining() >= 12
+        && m_buf.get(0) == 'W'
+        && m_buf.get(1) == 'P'
+        && m_buf.get(2) == 'I'
+        && m_buf.get(3) == 'L'
+        && m_buf.get(4) == 'O'
+        && m_buf.get(5) == 'G'
+        && m_buf.getShort(6) >= 0x0100;
+  }
+
+  /**
+   * Gets the data log version. Returns 0 if data log is invalid.
+   *
+   * @return Version number; most significant byte is major, least significant is minor (so version
+   *     1.0 will be 0x0100)
+   */
+  public short getVersion() {
+    if (m_buf.remaining() < 12) {
+      return 0;
+    }
+    return m_buf.getShort(6);
+  }
+
+  /**
+   * Gets the extra header data.
+   *
+   * @return Extra header data
+   */
+  public String getExtraHeader() {
+    ByteBuffer buf = m_buf.duplicate();
+    buf.order(ByteOrder.LITTLE_ENDIAN);
+    buf.position(8);
+    int size = buf.getInt();
+    byte[] arr = new byte[size];
+    buf.get(arr);
+    return new String(arr, StandardCharsets.UTF_8);
+  }
+
+  @Override
+  public void forEach(Consumer<? super DataLogRecord> action) {
+    int size = m_buf.remaining();
+    for (int pos = 12 + m_buf.getInt(8); pos < size; pos = getNextRecord(pos)) {
+      DataLogRecord record;
+      try {
+        record = getRecord(pos);
+      } catch (NoSuchElementException ex) {
+        break;
+      }
+      action.accept(record);
+    }
+  }
+
+  @Override
+  public DataLogIterator iterator() {
+    return new DataLogIterator(this, 12 + m_buf.getInt(8));
+  }
+
+  private long readVarInt(int pos, int len) {
+    long val = 0;
+    for (int i = 0; i < len; i++) {
+      val |= ((long) (m_buf.get(pos + i) & 0xff)) << (i * 8);
+    }
+    return val;
+  }
+
+  @SuppressWarnings("PMD.PreserveStackTrace")
+  DataLogRecord getRecord(int pos) {
+    try {
+      int lenbyte = m_buf.get(pos) & 0xff;
+      int entryLen = (lenbyte & 0x3) + 1;
+      int sizeLen = ((lenbyte >> 2) & 0x3) + 1;
+      int timestampLen = ((lenbyte >> 4) & 0x7) + 1;
+      int headerLen = 1 + entryLen + sizeLen + timestampLen;
+      int entry = (int) readVarInt(pos + 1, entryLen);
+      int size = (int) readVarInt(pos + 1 + entryLen, sizeLen);
+      long timestamp = readVarInt(pos + 1 + entryLen + sizeLen, timestampLen);
+      // build a slice of the data contents
+      ByteBuffer data = m_buf.duplicate();
+      data.position(pos + headerLen);
+      data.limit(pos + headerLen + size);
+      return new DataLogRecord(entry, timestamp, data.slice());
+    } catch (BufferUnderflowException | IndexOutOfBoundsException ex) {
+      throw new NoSuchElementException();
+    }
+  }
+
+  int getNextRecord(int pos) {
+    int lenbyte = m_buf.get(pos) & 0xff;
+    int entryLen = (lenbyte & 0x3) + 1;
+    int sizeLen = ((lenbyte >> 2) & 0x3) + 1;
+    int timestampLen = ((lenbyte >> 4) & 0x7) + 1;
+    int headerLen = 1 + entryLen + sizeLen + timestampLen;
+
+    int size = 0;
+    for (int i = 0; i < sizeLen; i++) {
+      size |= ((int) (m_buf.get(pos + 1 + entryLen + i) & 0xff)) << (i * 8);
+    }
+    return pos + headerLen + size;
+  }
+
+  int size() {
+    return m_buf.remaining();
+  }
+
+  private final ByteBuffer m_buf;
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DataLogRecord.java

@@ -0,0 +1,435 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+import java.nio.BufferUnderflowException;
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.DoubleBuffer;
+import java.nio.FloatBuffer;
+import java.nio.LongBuffer;
+import java.nio.charset.StandardCharsets;
+import java.util.InputMismatchException;
+
+/**
+ * A record in the data log. May represent either a control record (entry == 0) or a data record.
+ * Used only for reading (e.g. with DataLogReader).
+ */
+@SuppressWarnings("PMD.PreserveStackTrace")
+public class DataLogRecord {
+  private static final int kControlStart = 0;
+  private static final int kControlFinish = 1;
+  private static final int kControlSetMetadata = 2;
+
+  DataLogRecord(int entry, long timestamp, ByteBuffer data) {
+    m_entry = entry;
+    m_timestamp = timestamp;
+    m_data = data;
+    m_data.order(ByteOrder.LITTLE_ENDIAN);
+  }
+
+  /**
+   * Gets the entry ID.
+   *
+   * @return entry ID
+   */
+  public int getEntry() {
+    return m_entry;
+  }
+
+  /**
+   * Gets the record timestamp.
+   *
+   * @return Timestamp, in integer microseconds
+   */
+  public long getTimestamp() {
+    return m_timestamp;
+  }
+
+  /**
+   * Gets the size of the raw data.
+   *
+   * @return size
+   */
+  public int getSize() {
+    return m_data.remaining();
+  }
+
+  /**
+   * Gets the raw data. Use the GetX functions to decode based on the data type in the entry's start
+   * record.
+   *
+   * @return byte array
+   */
+  public byte[] getRaw() {
+    ByteBuffer buf = getRawBuffer();
+    byte[] arr = new byte[buf.remaining()];
+    buf.get(arr);
+    return arr;
+  }
+
+  /**
+   * Gets the raw data. Use the GetX functions to decode based on the data type in the entry's start
+   * record.
+   *
+   * @return byte buffer
+   */
+  public ByteBuffer getRawBuffer() {
+    ByteBuffer buf = m_data.duplicate();
+    buf.order(ByteOrder.LITTLE_ENDIAN);
+    return buf;
+  }
+
+  /**
+   * Returns true if the record is a control record.
+   *
+   * @return True if control record, false if normal data record.
+   */
+  public boolean isControl() {
+    return m_entry == 0;
+  }
+
+  /**
+   * Returns true if the record is a start control record. Use GetStartData() to decode the
+   * contents.
+   *
+   * @return True if start control record, false otherwise.
+   */
+  public boolean isStart() {
+    return m_entry == 0 && m_data.remaining() >= 17 && m_data.get(0) == kControlStart;
+  }
+
+  /**
+   * Returns true if the record is a finish control record. Use GetFinishEntry() to decode the
+   * contents.
+   *
+   * @return True if finish control record, false otherwise.
+   */
+  public boolean isFinish() {
+    return m_entry == 0 && m_data.remaining() == 5 && m_data.get(0) == kControlFinish;
+  }
+
+  /**
+   * Returns true if the record is a set metadata control record. Use GetSetMetadataData() to decode
+   * the contents.
+   *
+   * @return True if set metadata control record, false otherwise.
+   */
+  public boolean isSetMetadata() {
+    return m_entry == 0 && m_data.remaining() >= 9 && m_data.get(0) == kControlSetMetadata;
+  }
+
+  /**
+   * Data contained in a start control record as created by DataLog.start() when writing the log.
+   * This can be read by calling getStartData().
+   */
+  public static class StartRecordData {
+    StartRecordData(int entry, String name, String type, String metadata) {
+      this.entry = entry;
+      this.name = name;
+      this.type = type;
+      this.metadata = metadata;
+    }
+
+    /** Entry ID; this will be used for this entry in future records. */
+    @SuppressWarnings("MemberName")
+    public final int entry;
+
+    /** Entry name. */
+    @SuppressWarnings("MemberName")
+    public final String name;
+
+    /** Type of the stored data for this entry, as a string, e.g. "double". */
+    @SuppressWarnings("MemberName")
+    public final String type;
+
+    /** Initial metadata. */
+    @SuppressWarnings("MemberName")
+    public final String metadata;
+  }
+
+  /**
+   * Decodes a start control record.
+   *
+   * @return start record decoded data
+   * @throws InputMismatchException on error
+   */
+  public StartRecordData getStartData() {
+    if (!isStart()) {
+      throw new InputMismatchException("not a start record");
+    }
+    ByteBuffer buf = getRawBuffer();
+    buf.position(1); // skip over control type
+    int entry = buf.getInt();
+    String name = readInnerString(buf);
+    String type = readInnerString(buf);
+    String metadata = readInnerString(buf);
+    return new StartRecordData(entry, name, type, metadata);
+  }
+
+  /**
+   * Data contained in a set metadata control record as created by DataLog.setMetadata(). This can
+   * be read by calling getSetMetadataData().
+   */
+  public static class MetadataRecordData {
+    MetadataRecordData(int entry, String metadata) {
+      this.entry = entry;
+      this.metadata = metadata;
+    }
+
+    /** Entry ID. */
+    @SuppressWarnings("MemberName")
+    public final int entry;
+
+    /** New metadata for the entry. */
+    @SuppressWarnings("MemberName")
+    public final String metadata;
+  }
+
+  /**
+   * Decodes a finish control record.
+   *
+   * @return finish record entry ID
+   * @throws InputMismatchException on error
+   */
+  public int getFinishEntry() {
+    if (!isFinish()) {
+      throw new InputMismatchException("not a finish record");
+    }
+    return m_data.getInt(1);
+  }
+
+  /**
+   * Decodes a set metadata control record.
+   *
+   * @return set metadata record decoded data
+   * @throws InputMismatchException on error
+   */
+  public MetadataRecordData getSetMetadataData() {
+    if (!isSetMetadata()) {
+      throw new InputMismatchException("not a set metadata record");
+    }
+    ByteBuffer buf = getRawBuffer();
+    buf.position(1); // skip over control type
+    int entry = buf.getInt();
+    String metadata = readInnerString(buf);
+    return new MetadataRecordData(entry, metadata);
+  }
+
+  /**
+   * Decodes a data record as a boolean. Note if the data type (as indicated in the corresponding
+   * start control record for this entry) is not "boolean", invalid results may be returned.
+   *
+   * @return boolean value
+   * @throws InputMismatchException on error
+   */
+  public boolean getBoolean() {
+    try {
+      return m_data.get(0) != 0;
+    } catch (IndexOutOfBoundsException ex) {
+      throw new InputMismatchException();
+    }
+  }
+
+  /**
+   * Decodes a data record as an integer. Note if the data type (as indicated in the corresponding
+   * start control record for this entry) is not "int64", invalid results may be returned.
+   *
+   * @return integer value
+   * @throws InputMismatchException on error
+   */
+  public long getInteger() {
+    try {
+      return m_data.getLong(0);
+    } catch (BufferUnderflowException | IndexOutOfBoundsException ex) {
+      throw new InputMismatchException();
+    }
+  }
+
+  /**
+   * Decodes a data record as a float. Note if the data type (as indicated in the corresponding
+   * start control record for this entry) is not "float", invalid results may be returned.
+   *
+   * @return float value
+   * @throws InputMismatchException on error
+   */
+  public float getFloat() {
+    try {
+      return m_data.getFloat(0);
+    } catch (BufferUnderflowException | IndexOutOfBoundsException ex) {
+      throw new InputMismatchException();
+    }
+  }
+
+  /**
+   * Decodes a data record as a double. Note if the data type (as indicated in the corresponding
+   * start control record for this entry) is not "double", invalid results may be returned.
+   *
+   * @return double value
+   * @throws InputMismatchException on error
+   */
+  public double getDouble() {
+    try {
+      return m_data.getDouble(0);
+    } catch (BufferUnderflowException | IndexOutOfBoundsException ex) {
+      throw new InputMismatchException();
+    }
+  }
+
+  /**
+   * Decodes a data record as a string. Note if the data type (as indicated in the corresponding
+   * start control record for this entry) is not "string", invalid results may be returned.
+   *
+   * @return string value
+   */
+  public String getString() {
+    return new String(getRaw(), StandardCharsets.UTF_8);
+  }
+
+  /**
+   * Decodes a data record as a boolean array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "boolean[]", invalid results may be
+   * returned.
+   *
+   * @return boolean array
+   */
+  public boolean[] getBooleanArray() {
+    boolean[] arr = new boolean[m_data.remaining()];
+    for (int i = 0; i < m_data.remaining(); i++) {
+      arr[i] = m_data.get(i) != 0;
+    }
+    return arr;
+  }
+
+  /**
+   * Decodes a data record as an integer array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "int64[]", invalid results may be
+   * returned.
+   *
+   * @return integer array
+   * @throws InputMismatchException on error
+   */
+  public long[] getIntegerArray() {
+    LongBuffer buf = getIntegerBuffer();
+    long[] arr = new long[buf.remaining()];
+    buf.get(arr);
+    return arr;
+  }
+
+  /**
+   * Decodes a data record as an integer array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "int64[]", invalid results may be
+   * returned.
+   *
+   * @return integer buffer
+   * @throws InputMismatchException on error
+   */
+  public LongBuffer getIntegerBuffer() {
+    if ((m_data.limit() % 8) != 0) {
+      throw new InputMismatchException("data size is not a multiple of 8");
+    }
+    return m_data.asLongBuffer();
+  }
+
+  /**
+   * Decodes a data record as a float array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "float[]", invalid results may be
+   * returned.
+   *
+   * @return float array
+   * @throws InputMismatchException on error
+   */
+  public float[] getFloatArray() {
+    FloatBuffer buf = getFloatBuffer();
+    float[] arr = new float[buf.remaining()];
+    buf.get(arr);
+    return arr;
+  }
+
+  /**
+   * Decodes a data record as a float array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "float[]", invalid results may be
+   * returned.
+   *
+   * @return float buffer
+   * @throws InputMismatchException on error
+   */
+  public FloatBuffer getFloatBuffer() {
+    if ((m_data.limit() % 4) != 0) {
+      throw new InputMismatchException("data size is not a multiple of 4");
+    }
+    return m_data.asFloatBuffer();
+  }
+
+  /**
+   * Decodes a data record as a double array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "double[]", invalid results may be
+   * returned.
+   *
+   * @return double array
+   * @throws InputMismatchException on error
+   */
+  public double[] getDoubleArray() {
+    DoubleBuffer buf = getDoubleBuffer();
+    double[] arr = new double[buf.remaining()];
+    buf.get(arr);
+    return arr;
+  }
+
+  /**
+   * Decodes a data record as a double array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "double[]", invalid results may be
+   * returned.
+   *
+   * @return double buffer
+   * @throws InputMismatchException on error
+   */
+  public DoubleBuffer getDoubleBuffer() {
+    if ((m_data.limit() % 8) != 0) {
+      throw new InputMismatchException("data size is not a multiple of 8");
+    }
+    return m_data.asDoubleBuffer();
+  }
+
+  /**
+   * Decodes a data record as a string array. Note if the data type (as indicated in the
+   * corresponding start control record for this entry) is not "string[]", invalid results may be
+   * returned.
+   *
+   * @return string array
+   * @throws InputMismatchException on error
+   */
+  public String[] getStringArray() {
+    ByteBuffer buf = getRawBuffer();
+    try {
+      int size = buf.getInt();
+      // sanity check size
+      if (size > (buf.remaining() / 4)) {
+        throw new InputMismatchException("invalid size");
+      }
+      String[] arr = new String[size];
+      for (int i = 0; i < size; i++) {
+        arr[i] = readInnerString(buf);
+      }
+      return arr;
+    } catch (BufferUnderflowException | IndexOutOfBoundsException ex) {
+      throw new InputMismatchException();
+    }
+  }
+
+  private String readInnerString(ByteBuffer buf) {
+    int size = buf.getInt();
+    if (size > buf.remaining()) {
+      throw new InputMismatchException("invalid string size");
+    }
+    byte[] arr = new byte[size];
+    buf.get(arr);
+    return new String(arr, StandardCharsets.UTF_8);
+  }
+
+  private final int m_entry;
+  private final long m_timestamp;
+  private final ByteBuffer m_data;
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DoubleArrayLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log array of double values. */
+public class DoubleArrayLogEntry extends DataLogEntry {
+  public static final String kDataType = "double[]";
+
+  public DoubleArrayLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public DoubleArrayLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public DoubleArrayLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public DoubleArrayLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(double[] value, long timestamp) {
+    m_log.appendDoubleArray(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(double[] value) {
+    m_log.appendDoubleArray(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/DoubleLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log double values. */
+public class DoubleLogEntry extends DataLogEntry {
+  public static final String kDataType = "double";
+
+  public DoubleLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public DoubleLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public DoubleLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public DoubleLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(double value, long timestamp) {
+    m_log.appendDouble(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(double value) {
+    m_log.appendDouble(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/FloatArrayLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log array of float values. */
+public class FloatArrayLogEntry extends DataLogEntry {
+  public static final String kDataType = "float[]";
+
+  public FloatArrayLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public FloatArrayLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public FloatArrayLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public FloatArrayLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(float[] value, long timestamp) {
+    m_log.appendFloatArray(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(float[] value) {
+    m_log.appendFloatArray(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/FloatLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log float values. */
+public class FloatLogEntry extends DataLogEntry {
+  public static final String kDataType = "float";
+
+  public FloatLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public FloatLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public FloatLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public FloatLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(float value, long timestamp) {
+    m_log.appendFloat(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(float value) {
+    m_log.appendFloat(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/IntegerArrayLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log array of integer values. */
+public class IntegerArrayLogEntry extends DataLogEntry {
+  public static final String kDataType = "int64[]";
+
+  public IntegerArrayLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public IntegerArrayLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public IntegerArrayLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public IntegerArrayLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(long[] value, long timestamp) {
+    m_log.appendIntegerArray(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(long[] value) {
+    m_log.appendIntegerArray(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/IntegerLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log integer values. */
+public class IntegerLogEntry extends DataLogEntry {
+  public static final String kDataType = "int64";
+
+  public IntegerLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public IntegerLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public IntegerLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public IntegerLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(long value, long timestamp) {
+    m_log.appendInteger(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(long value) {
+    m_log.appendInteger(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/RawLogEntry.java

@@ -0,0 +1,53 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log raw byte array values. */
+public class RawLogEntry extends DataLogEntry {
+  public static final String kDataType = "raw";
+
+  public RawLogEntry(DataLog log, String name, String metadata, String type, long timestamp) {
+    super(log, name, type, metadata, timestamp);
+  }
+
+  public RawLogEntry(DataLog log, String name, String metadata, String type) {
+    this(log, name, metadata, type, 0);
+  }
+
+  public RawLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    this(log, name, metadata, kDataType, timestamp);
+  }
+
+  public RawLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public RawLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public RawLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(byte[] value, long timestamp) {
+    m_log.appendRaw(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(byte[] value) {
+    m_log.appendRaw(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/StringArrayLogEntry.java

@@ -0,0 +1,45 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log array of string values. */
+public class StringArrayLogEntry extends DataLogEntry {
+  public static final String kDataType = "string[]";
+
+  public StringArrayLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    super(log, name, kDataType, metadata, timestamp);
+  }
+
+  public StringArrayLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public StringArrayLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public StringArrayLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(String[] value, long timestamp) {
+    m_log.appendStringArray(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(String[] value) {
+    m_log.appendStringArray(m_entry, value, 0);
+  }
+}

wpiutil/src/main/java/edu/wpi/first/util/datalog/StringLogEntry.java

@@ -0,0 +1,53 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package edu.wpi.first.util.datalog;
+
+/** Log string values. */
+public class StringLogEntry extends DataLogEntry {
+  public static final String kDataType = "string";
+
+  public StringLogEntry(DataLog log, String name, String metadata, String type, long timestamp) {
+    super(log, name, type, metadata, timestamp);
+  }
+
+  public StringLogEntry(DataLog log, String name, String metadata, String type) {
+    this(log, name, metadata, type, 0);
+  }
+
+  public StringLogEntry(DataLog log, String name, String metadata, long timestamp) {
+    this(log, name, metadata, kDataType, timestamp);
+  }
+
+  public StringLogEntry(DataLog log, String name, String metadata) {
+    this(log, name, metadata, 0);
+  }
+
+  public StringLogEntry(DataLog log, String name, long timestamp) {
+    this(log, name, "", timestamp);
+  }
+
+  public StringLogEntry(DataLog log, String name) {
+    this(log, name, 0);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  public void append(String value, long timestamp) {
+    m_log.appendString(m_entry, value, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   */
+  public void append(String value) {
+    m_log.appendString(m_entry, value, 0);
+  }
+}

wpiutil/src/main/native/cpp/DataLog.cpp

@@ -0,0 +1,821 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+#include "wpi/DataLog.h"
+
+#include "wpi/Synchronization.h"
+
+#ifndef _WIN32
+#include <unistd.h>
+#endif
+
+#ifdef _WIN32
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+
+#include <windows.h>  // NOLINT(build/include_order)
+
+#endif
+
+#include <atomic>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <random>
+#include <vector>
+
+#include "fmt/format.h"
+#include "wpi/Endian.h"
+#include "wpi/Logger.h"
+#include "wpi/MathExtras.h"
+#include "wpi/fs.h"
+#include "wpi/timestamp.h"
+
+using namespace wpi::log;
+
+static constexpr size_t kBlockSize = 16 * 1024;
+static constexpr size_t kRecordMaxHeaderSize = 17;
+
+template <typename T>
+static unsigned int WriteVarInt(uint8_t* buf, T val) {
+  unsigned int len = 0;
+  do {
+    *buf++ = static_cast<unsigned int>(val) & 0xff;
+    ++len;
+    val >>= 8;
+  } while (val != 0);
+  return len;
+}
+
+// min size: 4, max size: 17
+static unsigned int WriteRecordHeader(uint8_t* buf, uint32_t entry,
+                                      uint64_t timestamp,
+                                      uint32_t payloadSize) {
+  uint8_t* origbuf = buf++;
+
+  unsigned int entryLen = WriteVarInt(buf, entry);
+  buf += entryLen;
+  unsigned int payloadLen = WriteVarInt(buf, payloadSize);
+  buf += payloadLen;
+  unsigned int timestampLen =
+      WriteVarInt(buf, timestamp == 0 ? wpi::Now() : timestamp);
+  buf += timestampLen;
+  *origbuf =
+      ((timestampLen - 1) << 4) | ((payloadLen - 1) << 2) | (entryLen - 1);
+  return buf - origbuf;
+}
+
+class DataLog::Buffer {
+ public:
+  explicit Buffer(size_t alloc = kBlockSize)
+      : m_buf{new uint8_t[alloc]}, m_maxLen{alloc} {}
+  ~Buffer() { delete[] m_buf; }
+
+  Buffer(const Buffer&) = delete;
+  Buffer& operator=(const Buffer&) = delete;
+
+  Buffer(Buffer&& oth)
+      : m_buf{oth.m_buf}, m_len{oth.m_len}, m_maxLen{oth.m_maxLen} {
+    oth.m_buf = nullptr;
+    oth.m_len = 0;
+    oth.m_maxLen = 0;
+  }
+
+  Buffer& operator=(Buffer&& oth) {
+    if (m_buf) {
+      delete[] m_buf;
+    }
+    m_buf = oth.m_buf;
+    m_len = oth.m_len;
+    m_maxLen = oth.m_maxLen;
+    oth.m_buf = nullptr;
+    oth.m_len = 0;
+    oth.m_maxLen = 0;
+    return *this;
+  }
+
+  uint8_t* Reserve(size_t size) {
+    assert(size <= GetRemaining());
+    uint8_t* rv = m_buf + m_len;
+    m_len += size;
+    return rv;
+  }
+
+  void Unreserve(size_t size) { m_len -= size; }
+
+  void Clear() { m_len = 0; }
+
+  size_t GetRemaining() const { return m_maxLen - m_len; }
+
+  wpi::span<uint8_t> GetData() { return {m_buf, m_len}; }
+  wpi::span<const uint8_t> GetData() const { return {m_buf, m_len}; }
+
+ private:
+  uint8_t* m_buf;
+  size_t m_len = 0;
+  size_t m_maxLen;
+};
+
+static void DefaultLog(unsigned int level, const char* file, unsigned int line,
+                       const char* msg) {
+  if (level > wpi::WPI_LOG_INFO) {
+    fmt::print(stderr, "DataLog: {}\n", msg);
+  } else if (level == wpi::WPI_LOG_INFO) {
+    fmt::print("DataLog: {}\n", msg);
+  }
+}
+
+static wpi::Logger defaultMessageLog{DefaultLog};
+
+DataLog::DataLog(std::string_view dir, std::string_view filename, double period,
+                 std::string_view extraHeader)
+    : DataLog{defaultMessageLog, dir, filename, period, extraHeader} {}
+
+DataLog::DataLog(wpi::Logger& msglog, std::string_view dir,
+                 std::string_view filename, double period,
+                 std::string_view extraHeader)
+    : m_msglog{msglog},
+      m_period{period},
+      m_extraHeader{extraHeader},
+      m_newFilename{filename},
+      m_thread{[this, dir = std::string{dir}] { WriterThreadMain(dir); }} {}
+
+DataLog::DataLog(std::function<void(wpi::span<const uint8_t> data)> write,
+                 double period, std::string_view extraHeader)
+    : DataLog{defaultMessageLog, std::move(write), period, extraHeader} {}
+
+DataLog::DataLog(wpi::Logger& msglog,
+                 std::function<void(wpi::span<const uint8_t> data)> write,
+                 double period, std::string_view extraHeader)
+    : m_msglog{msglog},
+      m_period{period},
+      m_extraHeader{extraHeader},
+      m_thread{[this, write = std::move(write)] {
+        WriterThreadMain(std::move(write));
+      }} {}
+
+DataLog::~DataLog() {
+  {
+    std::scoped_lock lock{m_mutex};
+    m_active = false;
+    m_doFlush = true;
+  }
+  m_cond.notify_all();
+  m_thread.join();
+}
+
+void DataLog::SetFilename(std::string_view filename) {
+  {
+    std::scoped_lock lock{m_mutex};
+    m_newFilename = filename;
+  }
+  m_cond.notify_all();
+}
+
+void DataLog::Flush() {
+  {
+    std::scoped_lock lock{m_mutex};
+    m_doFlush = true;
+  }
+  m_cond.notify_all();
+}
+
+void DataLog::Pause() {
+  std::scoped_lock lock{m_mutex};
+  m_paused = true;
+}
+
+void DataLog::Resume() {
+  std::scoped_lock lock{m_mutex};
+  m_paused = false;
+}
+
+static void WriteToFile(fs::file_t f, wpi::span<const uint8_t> data,
+                        std::string_view filename, wpi::Logger& msglog) {
+  do {
+#ifdef _WIN32
+    DWORD ret;
+    if (!WriteFile(f, data.data(), data.size(), &ret, nullptr)) {
+      WPI_ERROR(msglog, "Error writing to log file '{}': {}", filename,
+                GetLastError());
+      break;
+    }
+#else
+    ssize_t ret = ::write(f, data.data(), data.size());
+    if (ret < 0) {
+      // If it's a recoverable error, swallow it and retry the write
+      if (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK) {
+        continue;
+      }
+
+      // Otherwise it's a non-recoverable error; quit trying
+      WPI_ERROR(msglog, "Error writing to log file '{}': {}", filename,
+                std::strerror(errno));
+      break;
+    }
+#endif
+
+    // The write may have written some or all of the data
+    data = data.subspan(ret);
+  } while (data.size() > 0);
+}
+
+static std::string MakeRandomFilename() {
+  // build random filename
+  static std::random_device dev;
+  static std::mt19937 rng(dev());
+  std::uniform_int_distribution<int> dist(0, 15);
+  const char* v = "0123456789abcdef";
+  std::string filename = "wpilog_";
+  for (int i = 0; i < 16; i++) {
+    filename += v[dist(rng)];
+  }
+  filename += ".wpilog";
+  return filename;
+}
+
+void DataLog::WriterThreadMain(std::string_view dir) {
+  std::chrono::duration<double> periodTime{m_period};
+
+  std::error_code ec;
+  fs::path dirPath{dir};
+  std::string filename;
+
+  {
+    std::scoped_lock lock{m_mutex};
+    filename = std::move(m_newFilename);
+    m_newFilename.clear();
+  }
+
+  if (filename.empty()) {
+    filename = MakeRandomFilename();
+  }
+
+  // try preferred filename, or randomize it a few times, before giving up
+  fs::file_t f;
+  for (int i = 0; i < 5; ++i) {
+    // open file for append
+#ifdef _WIN32
+    // WIN32 doesn't allow combination of CreateNew and Append
+    f = fs::OpenFileForWrite(dirPath / filename, ec, fs::CD_CreateNew,
+                             fs::OF_None);
+#else
+    f = fs::OpenFileForWrite(dirPath / filename, ec, fs::CD_CreateNew,
+                             fs::OF_Append);
+#endif
+    if (ec) {
+      WPI_ERROR(m_msglog, "Could not open log file '{}': {}",
+                (dirPath / filename).string(), ec.message());
+      // try again with random filename
+      filename = MakeRandomFilename();
+    } else {
+      break;
+    }
+  }
+
+  if (f == fs::kInvalidFile) {
+    WPI_ERROR(m_msglog, "{}", "Could not open log file, no log being saved");
+  } else {
+    WPI_INFO(m_msglog, "Logging to '{}'", (dirPath / filename).string());
+  }
+
+  // write header (version 1.0)
+  if (f != fs::kInvalidFile) {
+    const uint8_t header[] = {'W', 'P', 'I', 'L', 'O', 'G', 0, 1};
+    WriteToFile(f, header, filename, m_msglog);
+    uint8_t extraLen[4];
+    support::endian::write32le(extraLen, m_extraHeader.size());
+    WriteToFile(f, extraLen, filename, m_msglog);
+    if (m_extraHeader.size() > 0) {
+      WriteToFile(f,
+                  {reinterpret_cast<const uint8_t*>(m_extraHeader.data()),
+                   m_extraHeader.size()},
+                  filename, m_msglog);
+    }
+  }
+
+  std::vector<Buffer> toWrite;
+
+  std::unique_lock lock{m_mutex};
+  while (m_active) {
+    bool doFlush = false;
+    auto timeoutTime = std::chrono::steady_clock::now() + periodTime;
+    if (m_cond.wait_until(lock, timeoutTime) == std::cv_status::timeout) {
+      doFlush = true;
+    }
+
+    if (!m_newFilename.empty()) {
+      auto newFilename = std::move(m_newFilename);
+      m_newFilename.clear();
+      lock.unlock();
+      // rename
+      if (filename != newFilename) {
+        fs::rename(dirPath / filename, dirPath / newFilename, ec);
+      }
+      if (ec) {
+        WPI_ERROR(m_msglog, "Could not rename log file from '{}' to '{}': {}",
+                  filename, newFilename, ec.message());
+      } else {
+        WPI_INFO(m_msglog, "Renamed log file from '{}' to '{}'", filename,
+                 newFilename);
+      }
+      filename = std::move(newFilename);
+      lock.lock();
+    }
+
+    if (doFlush || m_doFlush) {
+      // flush to file
+      m_doFlush = false;
+      if (m_outgoing.empty()) {
+        continue;
+      }
+      // swap outgoing with empty vector
+      toWrite.swap(m_outgoing);
+
+      if (f != fs::kInvalidFile) {
+        lock.unlock();
+        // write buffers to file
+        for (auto&& buf : toWrite) {
+          WriteToFile(f, buf.GetData(), filename, m_msglog);
+        }
+
+        // sync to storage
+#if defined(__linux__)
+        ::fdatasync(f);
+#elif defined(__APPLE__)
+        ::fsync(f);
+#endif
+        lock.lock();
+      }
+
+      // release buffers back to free list
+      for (auto&& buf : toWrite) {
+        buf.Clear();
+        m_free.emplace_back(std::move(buf));
+      }
+      toWrite.resize(0);
+    }
+  }
+
+  if (f != fs::kInvalidFile) {
+    fs::CloseFile(f);
+  }
+}
+
+void DataLog::WriterThreadMain(
+    std::function<void(wpi::span<const uint8_t> data)> write) {
+  std::chrono::duration<double> periodTime{m_period};
+
+  // write header (version 1.0)
+  {
+    const uint8_t header[] = {'W', 'P', 'I', 'L', 'O', 'G', 0, 1};
+    write(header);
+    uint8_t extraLen[4];
+    support::endian::write32le(extraLen, m_extraHeader.size());
+    write(extraLen);
+    if (m_extraHeader.size() > 0) {
+      write({reinterpret_cast<const uint8_t*>(m_extraHeader.data()),
+             m_extraHeader.size()});
+    }
+  }
+
+  std::vector<Buffer> toWrite;
+
+  std::unique_lock lock{m_mutex};
+  while (m_active) {
+    bool doFlush = false;
+    auto timeoutTime = std::chrono::steady_clock::now() + periodTime;
+    if (m_cond.wait_until(lock, timeoutTime) == std::cv_status::timeout) {
+      doFlush = true;
+    }
+
+    if (doFlush || m_doFlush) {
+      // flush to file
+      m_doFlush = false;
+      if (m_outgoing.empty()) {
+        continue;
+      }
+      // swap outgoing with empty vector
+      toWrite.swap(m_outgoing);
+
+      lock.unlock();
+      // write buffers
+      for (auto&& buf : toWrite) {
+        if (!buf.GetData().empty()) {
+          write(buf.GetData());
+        }
+      }
+      lock.lock();
+
+      // release buffers back to free list
+      for (auto&& buf : toWrite) {
+        buf.Clear();
+        m_free.emplace_back(std::move(buf));
+      }
+      toWrite.resize(0);
+    }
+  }
+
+  write({});  // indicate EOF
+}
+
+// Control records use the following format:
+// 1-byte type
+// 4-byte entry
+// rest of data (depending on type)
+
+int DataLog::Start(std::string_view name, std::string_view type,
+                   std::string_view metadata, int64_t timestamp) {
+  std::scoped_lock lock{m_mutex};
+  auto& entryInfo = m_entries[name];
+  if (entryInfo.id == 0) {
+    entryInfo.id = ++m_lastId;
+  }
+  auto& savedCount = m_entryCounts[entryInfo.id];
+  ++savedCount;
+  if (savedCount > 1) {
+    if (entryInfo.type != type) {
+      WPI_ERROR(m_msglog,
+                "type mismatch for '{}': was '{}', requested '{}'; ignoring",
+                name, entryInfo.type, type);
+      return 0;
+    }
+    return entryInfo.id;
+  }
+  entryInfo.type = type;
+  size_t strsize = name.size() + type.size() + metadata.size();
+  uint8_t* buf = StartRecord(0, timestamp, 5 + 12 + strsize, 5);
+  *buf++ = impl::kControlStart;
+  wpi::support::endian::write32le(buf, entryInfo.id);
+  AppendStringImpl(name);
+  AppendStringImpl(type);
+  AppendStringImpl(metadata);
+
+  return entryInfo.id;
+}
+
+void DataLog::Finish(int entry, int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  auto& savedCount = m_entryCounts[entry];
+  if (savedCount == 0) {
+    return;
+  }
+  --savedCount;
+  if (savedCount != 0) {
+    return;
+  }
+  m_entryCounts.erase(entry);
+  uint8_t* buf = StartRecord(0, timestamp, 5, 5);
+  *buf++ = impl::kControlFinish;
+  wpi::support::endian::write32le(buf, entry);
+}
+
+void DataLog::SetMetadata(int entry, std::string_view metadata,
+                          int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  uint8_t* buf = StartRecord(entry, timestamp, 5 + 4 + metadata.size(), 5);
+  *buf++ = impl::kControlSetMetadata;
+  wpi::support::endian::write32le(buf, entry);
+  AppendStringImpl(metadata);
+}
+
+uint8_t* DataLog::Reserve(size_t size) {
+  assert(size <= kBlockSize);
+  if (m_outgoing.empty() || size > m_outgoing.back().GetRemaining()) {
+    if (m_free.empty()) {
+      m_outgoing.emplace_back();
+    } else {
+      m_outgoing.emplace_back(std::move(m_free.back()));
+      m_free.pop_back();
+    }
+  }
+  return m_outgoing.back().Reserve(size);
+}
+
+uint8_t* DataLog::StartRecord(uint32_t entry, uint64_t timestamp,
+                              uint32_t payloadSize, size_t reserveSize) {
+  uint8_t* buf = Reserve(kRecordMaxHeaderSize + reserveSize);
+  auto headerLen = WriteRecordHeader(buf, entry, timestamp, payloadSize);
+  m_outgoing.back().Unreserve(kRecordMaxHeaderSize - headerLen);
+  buf += headerLen;
+  return buf;
+}
+
+void DataLog::AppendImpl(wpi::span<const uint8_t> data) {
+  while (data.size() > kBlockSize) {
+    uint8_t* buf = Reserve(kBlockSize);
+    std::memcpy(buf, data.data(), kBlockSize);
+    data = data.subspan(kBlockSize);
+  }
+  uint8_t* buf = Reserve(data.size());
+  std::memcpy(buf, data.data(), data.size());
+}
+
+void DataLog::AppendStringImpl(std::string_view str) {
+  uint8_t* buf = Reserve(4);
+  wpi::support::endian::write32le(buf, str.size());
+  AppendImpl({reinterpret_cast<const uint8_t*>(str.data()), str.size()});
+}
+
+void DataLog::AppendRaw(int entry, wpi::span<const uint8_t> data,
+                        int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  StartRecord(entry, timestamp, data.size(), 0);
+  AppendImpl(data);
+}
+
+void DataLog::AppendRaw2(int entry,
+                         wpi::span<const wpi::span<const uint8_t>> data,
+                         int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  size_t size = 0;
+  for (auto&& chunk : data) {
+    size += chunk.size();
+  }
+  StartRecord(entry, timestamp, size, 0);
+  for (auto chunk : data) {
+    AppendImpl(chunk);
+  }
+}
+
+void DataLog::AppendBoolean(int entry, bool value, int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  uint8_t* buf = StartRecord(entry, timestamp, 1, 1);
+  buf[0] = value ? 1 : 0;
+}
+
+void DataLog::AppendInteger(int entry, int64_t value, int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  uint8_t* buf = StartRecord(entry, timestamp, 8, 8);
+  wpi::support::endian::write64le(buf, value);
+}
+
+void DataLog::AppendFloat(int entry, float value, int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  uint8_t* buf = StartRecord(entry, timestamp, 4, 4);
+  if constexpr (wpi::support::endian::system_endianness() ==
+                wpi::support::little) {
+    std::memcpy(buf, &value, 4);
+  } else {
+    wpi::support::endian::write32le(buf, wpi::FloatToBits(value));
+  }
+}
+
+void DataLog::AppendDouble(int entry, double value, int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  uint8_t* buf = StartRecord(entry, timestamp, 8, 8);
+  if constexpr (wpi::support::endian::system_endianness() ==
+                wpi::support::little) {
+    std::memcpy(buf, &value, 8);
+  } else {
+    wpi::support::endian::write64le(buf, wpi::DoubleToBits(value));
+  }
+}
+
+void DataLog::AppendString(int entry, std::string_view value,
+                           int64_t timestamp) {
+  AppendRaw(entry,
+            {reinterpret_cast<const uint8_t*>(value.data()), value.size()},
+            timestamp);
+}
+
+void DataLog::AppendBooleanArray(int entry, wpi::span<const bool> arr,
+                                 int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  StartRecord(entry, timestamp, arr.size(), 0);
+  uint8_t* buf;
+  while (arr.size() > kBlockSize) {
+    buf = Reserve(kBlockSize);
+    for (auto val : arr.subspan(0, kBlockSize)) {
+      *buf++ = val ? 1 : 0;
+    }
+    arr = arr.subspan(kBlockSize);
+  }
+  buf = Reserve(arr.size());
+  for (auto val : arr) {
+    *buf++ = val ? 1 : 0;
+  }
+}
+
+void DataLog::AppendBooleanArray(int entry, wpi::span<const int> arr,
+                                 int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  StartRecord(entry, timestamp, arr.size(), 0);
+  uint8_t* buf;
+  while (arr.size() > kBlockSize) {
+    buf = Reserve(kBlockSize);
+    for (auto val : arr.subspan(0, kBlockSize)) {
+      *buf++ = val & 1;
+    }
+    arr = arr.subspan(kBlockSize);
+  }
+  buf = Reserve(arr.size());
+  for (auto val : arr) {
+    *buf++ = val & 1;
+  }
+}
+
+void DataLog::AppendBooleanArray(int entry, wpi::span<const uint8_t> arr,
+                                 int64_t timestamp) {
+  AppendRaw(entry, arr, timestamp);
+}
+
+void DataLog::AppendIntegerArray(int entry, wpi::span<const int64_t> arr,
+                                 int64_t timestamp) {
+  if constexpr (wpi::support::endian::system_endianness() ==
+                wpi::support::little) {
+    AppendRaw(entry,
+              {reinterpret_cast<const uint8_t*>(arr.data()), arr.size() * 8},
+              timestamp);
+  } else {
+    if (entry <= 0) {
+      return;
+    }
+    std::scoped_lock lock{m_mutex};
+    if (m_paused) {
+      return;
+    }
+    StartRecord(entry, timestamp, arr.size() * 8, 0);
+    uint8_t* buf;
+    while ((arr.size() * 8) > kBlockSize) {
+      buf = Reserve(kBlockSize);
+      for (auto val : arr.subspan(0, kBlockSize / 8)) {
+        wpi::support::endian::write64le(buf, val);
+        buf += 8;
+      }
+      arr = arr.subspan(kBlockSize / 8);
+    }
+    buf = Reserve(arr.size() * 8);
+    for (auto val : arr) {
+      wpi::support::endian::write64le(buf, val);
+      buf += 8;
+    }
+  }
+}
+
+void DataLog::AppendFloatArray(int entry, wpi::span<const float> arr,
+                               int64_t timestamp) {
+  if constexpr (wpi::support::endian::system_endianness() ==
+                wpi::support::little) {
+    AppendRaw(entry,
+              {reinterpret_cast<const uint8_t*>(arr.data()), arr.size() * 4},
+              timestamp);
+  } else {
+    if (entry <= 0) {
+      return;
+    }
+    std::scoped_lock lock{m_mutex};
+    if (m_paused) {
+      return;
+    }
+    StartRecord(entry, timestamp, arr.size() * 4, 0);
+    uint8_t* buf;
+    while ((arr.size() * 4) > kBlockSize) {
+      buf = Reserve(kBlockSize);
+      for (auto val : arr.subspan(0, kBlockSize / 4)) {
+        wpi::support::endian::write32le(buf, wpi::FloatToBits(val));
+        buf += 4;
+      }
+      arr = arr.subspan(kBlockSize / 4);
+    }
+    buf = Reserve(arr.size() * 4);
+    for (auto val : arr) {
+      wpi::support::endian::write32le(buf, wpi::FloatToBits(val));
+      buf += 4;
+    }
+  }
+}
+
+void DataLog::AppendDoubleArray(int entry, wpi::span<const double> arr,
+                                int64_t timestamp) {
+  if constexpr (wpi::support::endian::system_endianness() ==
+                wpi::support::little) {
+    AppendRaw(entry,
+              {reinterpret_cast<const uint8_t*>(arr.data()), arr.size() * 8},
+              timestamp);
+  } else {
+    if (entry <= 0) {
+      return;
+    }
+    std::scoped_lock lock{m_mutex};
+    if (m_paused) {
+      return;
+    }
+    StartRecord(entry, timestamp, arr.size() * 8, 0);
+    uint8_t* buf;
+    while ((arr.size() * 8) > kBlockSize) {
+      buf = Reserve(kBlockSize);
+      for (auto val : arr.subspan(0, kBlockSize / 8)) {
+        wpi::support::endian::write64le(buf, wpi::DoubleToBits(val));
+        buf += 8;
+      }
+      arr = arr.subspan(kBlockSize / 8);
+    }
+    buf = Reserve(arr.size() * 8);
+    for (auto val : arr) {
+      wpi::support::endian::write64le(buf, wpi::DoubleToBits(val));
+      buf += 8;
+    }
+  }
+}
+
+void DataLog::AppendStringArray(int entry, wpi::span<const std::string> arr,
+                                int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  // storage: 4-byte array length, each string prefixed by 4-byte length
+  // calculate total size
+  size_t size = 4;
+  for (auto&& str : arr) {
+    size += 4 + str.size();
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  uint8_t* buf = StartRecord(entry, timestamp, size, 4);
+  wpi::support::endian::write32le(buf, arr.size());
+  for (auto&& str : arr) {
+    AppendStringImpl(str);
+  }
+}
+
+void DataLog::AppendStringArray(int entry,
+                                wpi::span<const std::string_view> arr,
+                                int64_t timestamp) {
+  if (entry <= 0) {
+    return;
+  }
+  // storage: 4-byte array length, each string prefixed by 4-byte length
+  // calculate total size
+  size_t size = 4;
+  for (auto&& str : arr) {
+    size += 4 + str.size();
+  }
+  std::scoped_lock lock{m_mutex};
+  if (m_paused) {
+    return;
+  }
+  uint8_t* buf = StartRecord(entry, timestamp, size, 4);
+  wpi::support::endian::write32le(buf, arr.size());
+  for (auto sv : arr) {
+    AppendStringImpl(sv);
+  }
+}

wpiutil/src/main/native/cpp/DataLogReader.cpp

@@ -0,0 +1,307 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+#include "wpi/DataLogReader.h"
+
+#include "wpi/DataLog.h"
+#include "wpi/Endian.h"
+#include "wpi/MathExtras.h"
+
+using namespace wpi::log;
+
+static bool ReadString(wpi::span<const uint8_t>* buf, std::string_view* str) {
+  if (buf->size() < 4) {
+    *str = {};
+    return false;
+  }
+  uint32_t len = wpi::support::endian::read32le(buf->data());
+  if (len > (buf->size() - 4)) {
+    *str = {};
+    return false;
+  }
+  *str = {reinterpret_cast<const char*>(buf->data() + 4), len};
+  *buf = buf->subspan(len + 4);
+  return true;
+}
+
+bool DataLogRecord::IsStart() const {
+  return m_entry == 0 && m_data.size() >= 17 &&
+         m_data[0] == impl::kControlStart;
+}
+
+bool DataLogRecord::IsFinish() const {
+  return m_entry == 0 && m_data.size() == 5 &&
+         m_data[0] == impl::kControlFinish;
+}
+
+bool DataLogRecord::IsSetMetadata() const {
+  return m_entry == 0 && m_data.size() >= 9 &&
+         m_data[0] == impl::kControlSetMetadata;
+}
+
+bool DataLogRecord::GetStartData(StartRecordData* out) const {
+  if (!IsStart()) {
+    return false;
+  }
+  out->entry = wpi::support::endian::read32le(&m_data[1]);
+  auto buf = m_data.subspan(5);
+  if (!ReadString(&buf, &out->name)) {
+    return false;
+  }
+  if (!ReadString(&buf, &out->type)) {
+    return false;
+  }
+  if (!ReadString(&buf, &out->metadata)) {
+    return false;
+  }
+  return true;
+}
+
+bool DataLogRecord::GetFinishEntry(int* out) const {
+  if (!IsFinish()) {
+    return false;
+  }
+  *out = wpi::support::endian::read32le(&m_data[1]);
+  return true;
+}
+
+bool DataLogRecord::GetSetMetadataData(MetadataRecordData* out) const {
+  if (!IsSetMetadata()) {
+    return false;
+  }
+  out->entry = wpi::support::endian::read32le(&m_data[1]);
+  auto buf = m_data.subspan(5);
+  return ReadString(&buf, &out->metadata);
+}
+
+bool DataLogRecord::GetBoolean(bool* value) const {
+  if (m_data.size() != 1) {
+    return false;
+  }
+  *value = m_data[0] != 0;
+  return true;
+}
+
+bool DataLogRecord::GetInteger(int64_t* value) const {
+  if (m_data.size() != 8) {
+    return false;
+  }
+  *value = wpi::support::endian::read64le(m_data.data());
+  return true;
+}
+
+bool DataLogRecord::GetFloat(float* value) const {
+  if (m_data.size() != 4) {
+    return false;
+  }
+  *value = wpi::BitsToFloat(wpi::support::endian::read32le(m_data.data()));
+  return true;
+}
+
+bool DataLogRecord::GetDouble(double* value) const {
+  if (m_data.size() != 8) {
+    return false;
+  }
+  *value = wpi::BitsToDouble(wpi::support::endian::read64le(m_data.data()));
+  return true;
+}
+
+bool DataLogRecord::GetString(std::string_view* value) const {
+  *value = {reinterpret_cast<const char*>(m_data.data()), m_data.size()};
+  return true;
+}
+
+bool DataLogRecord::GetBooleanArray(std::vector<int>* arr) const {
+  arr->clear();
+  arr->reserve(m_data.size());
+  for (auto v : m_data) {
+    arr->push_back(v);
+  }
+  return true;
+}
+
+bool DataLogRecord::GetIntegerArray(std::vector<int64_t>* arr) const {
+  arr->clear();
+  if ((m_data.size() % 8) != 0) {
+    return false;
+  }
+  arr->reserve(m_data.size() / 8);
+  for (size_t pos = 0; pos < m_data.size(); pos += 8) {
+    arr->push_back(wpi::support::endian::read64le(&m_data[pos]));
+  }
+  return true;
+}
+
+bool DataLogRecord::GetFloatArray(std::vector<float>* arr) const {
+  arr->clear();
+  if ((m_data.size() % 4) != 0) {
+    return false;
+  }
+  arr->reserve(m_data.size() / 4);
+  for (size_t pos = 0; pos < m_data.size(); pos += 4) {
+    arr->push_back(
+        wpi::BitsToFloat(wpi::support::endian::read32le(&m_data[pos])));
+  }
+  return true;
+}
+
+bool DataLogRecord::GetDoubleArray(std::vector<double>* arr) const {
+  arr->clear();
+  if ((m_data.size() % 8) != 0) {
+    return false;
+  }
+  arr->reserve(m_data.size() / 8);
+  for (size_t pos = 0; pos < m_data.size(); pos += 8) {
+    arr->push_back(
+        wpi::BitsToDouble(wpi::support::endian::read64le(&m_data[pos])));
+  }
+  return true;
+}
+
+bool DataLogRecord::GetStringArray(std::vector<std::string_view>* arr) const {
+  arr->clear();
+  if (m_data.size() < 4) {
+    return false;
+  }
+  uint32_t size = wpi::support::endian::read32le(m_data.data());
+  // sanity check size
+  if (size > ((m_data.size() - 4) / 4)) {
+    return false;
+  }
+  auto buf = m_data.subspan(4);
+  arr->reserve(size);
+  for (uint32_t i = 0; i < size; ++i) {
+    std::string_view str;
+    if (!ReadString(&buf, &str)) {
+      arr->clear();
+      return false;
+    }
+    arr->push_back(str);
+  }
+  // any left over?  treat as corrupt
+  if (!buf.empty()) {
+    arr->clear();
+    return false;
+  }
+  return true;
+}
+
+DataLogReader::DataLogReader(std::unique_ptr<MemoryBuffer> buffer)
+    : m_buf{std::move(buffer)} {}
+
+bool DataLogReader::IsValid() const {
+  if (!m_buf) {
+    return false;
+  }
+  auto buf = m_buf->GetBuffer();
+  return buf.size() >= 12 &&
+         std::string_view{reinterpret_cast<const char*>(buf.data()), 6} ==
+             "WPILOG" &&
+         wpi::support::endian::read16le(&buf[6]) >= 0x0100;
+}
+
+uint16_t DataLogReader::GetVersion() const {
+  if (!m_buf) {
+    return 0;
+  }
+  auto buf = m_buf->GetBuffer();
+  if (buf.size() < 12) {
+    return 0;
+  }
+  return wpi::support::endian::read16le(&buf[6]);
+}
+
+std::string_view DataLogReader::GetExtraHeader() const {
+  if (!m_buf) {
+    return {};
+  }
+  auto buf = m_buf->GetBuffer();
+  if (buf.size() < 8) {
+    return {};
+  }
+  std::string_view rv;
+  buf = buf.subspan(8);
+  ReadString(&buf, &rv);
+  return rv;
+}
+
+DataLogReader::iterator DataLogReader::begin() const {
+  if (!m_buf) {
+    return end();
+  }
+  auto buf = m_buf->GetBuffer();
+  if (buf.size() < 12) {
+    return end();
+  }
+  uint32_t size = wpi::support::endian::read32le(&buf[8]);
+  if (buf.size() < (12 + size)) {
+    return end();
+  }
+  return DataLogIterator{this, 12 + size};
+}
+
+static uint64_t ReadVarInt(wpi::span<const uint8_t> buf) {
+  uint64_t val = 0;
+  int shift = 0;
+  for (auto v : buf) {
+    val |= static_cast<uint64_t>(v) << shift;
+    shift += 8;
+  }
+  return val;
+}
+
+bool DataLogReader::GetRecord(size_t* pos, DataLogRecord* out) const {
+  if (!m_buf) {
+    return false;
+  }
+  auto buf = m_buf->GetBuffer();
+  if (*pos >= buf.size()) {
+    return false;
+  }
+  buf = buf.subspan(*pos);
+  if (buf.size() < 4) {  // minimum header length
+    return false;
+  }
+  unsigned int entryLen = (buf[0] & 0x3) + 1;
+  unsigned int sizeLen = ((buf[0] >> 2) & 0x3) + 1;
+  unsigned int timestampLen = ((buf[0] >> 4) & 0x7) + 1;
+  unsigned int headerLen = 1 + entryLen + sizeLen + timestampLen;
+  if (buf.size() < headerLen) {
+    return false;
+  }
+  int entry = ReadVarInt(buf.subspan(1, entryLen));
+  uint32_t size = ReadVarInt(buf.subspan(1 + entryLen, sizeLen));
+  if (size > (buf.size() - headerLen)) {
+    return false;
+  }
+  int64_t timestamp =
+      ReadVarInt(buf.subspan(1 + entryLen + sizeLen, timestampLen));
+  *out = DataLogRecord{entry, timestamp, buf.subspan(headerLen, size)};
+  *pos += headerLen + size;
+  return true;
+}
+
+bool DataLogReader::GetNextRecord(size_t* pos) const {
+  if (!m_buf) {
+    return false;
+  }
+  auto buf = m_buf->GetBuffer();
+  if (buf.size() < (*pos + 4)) {  // minimum header length
+    return false;
+  }
+  unsigned int entryLen = (buf[*pos] & 0x3) + 1;
+  unsigned int sizeLen = ((buf[*pos] >> 2) & 0x3) + 1;
+  unsigned int timestampLen = ((buf[*pos] >> 4) & 0x7) + 1;
+  unsigned int headerLen = 1 + entryLen + sizeLen + timestampLen;
+  if (buf.size() < (*pos + headerLen)) {
+    return false;
+  }
+  uint32_t size = ReadVarInt(buf.subspan(*pos + 1 + entryLen, sizeLen));
+  // check this way to avoid overflow
+  if (size >= (buf.size() - *pos - headerLen)) {
+    return false;
+  }
+  *pos += headerLen + size;
+  return true;
+}

wpiutil/src/main/native/cpp/fs.cpp

@@ -82,17 +82,6 @@ namespace fs {
 const file_t kInvalidFile = INVALID_HANDLE_VALUE;
 
 static DWORD nativeDisposition(CreationDisposition Disp, OpenFlags Flags) {
-  // This is a compatibility hack.  Really we should respect the creation
-  // disposition, but a lot of old code relied on the implicit assumption that
-  // OF_Append implied it would open an existing file.  Since the disposition is
-  // now explicit and defaults to CD_CreateAlways, this assumption would cause
-  // any usage of OF_Append to append to a new file, even if the file already
-  // existed.  A better solution might have two new creation dispositions:
-  // CD_AppendAlways and CD_AppendNew.  This would also address the problem of
-  // OF_Append being used on a read-only descriptor, which doesn't make sense.
-  if (Flags & OF_Append)
-    return OPEN_ALWAYS;
-
   switch (Disp) {
     case CD_CreateAlways:
       return CREATE_ALWAYS;
@@ -251,12 +240,6 @@ static int nativeOpenFlags(CreationDisposition Disp, OpenFlags Flags,
     Result |= O_RDWR;
   }
 
-  // This is for compatibility with old code that assumed F_Append implied
-  // would open an existing file.  See Windows/Path.inc for a longer comment.
-  if (Flags & F_Append) {
-    Disp = CD_OpenAlways;
-  }
-
   if (Disp == CD_CreateNew) {
     Result |= O_CREAT;  // Create if it doesn't exist.
     Result |= O_EXCL;   // Fail if it does.

wpiutil/src/main/native/cpp/jni/DataLogJNI.cpp

@@ -0,0 +1,357 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+#include <jni.h>
+
+#include "edu_wpi_first_util_datalog_DataLogJNI.h"
+#include "wpi/DataLog.h"
+#include "wpi/jni_util.h"
+
+using namespace wpi::java;
+using namespace wpi::log;
+
+extern "C" {
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    create
+ * Signature: (Ljava/lang/String;Ljava/lang/String;DLjava/lang/String;)J
+ */
+JNIEXPORT jlong JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_create
+  (JNIEnv* env, jclass, jstring dir, jstring filename, jdouble period,
+   jstring extraHeader)
+{
+  return reinterpret_cast<jlong>(new DataLog{JStringRef{env, dir},
+                                             JStringRef{env, filename}, period,
+                                             JStringRef{env, extraHeader}});
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    setFilename
+ * Signature: (JLjava/lang/String;)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_setFilename
+  (JNIEnv* env, jclass, jlong impl, jstring filename)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->SetFilename(JStringRef{env, filename});
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    flush
+ * Signature: (J)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_flush
+  (JNIEnv*, jclass, jlong impl)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->Flush();
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    pause
+ * Signature: (J)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_pause
+  (JNIEnv*, jclass, jlong impl)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->Pause();
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    resume
+ * Signature: (J)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_resume
+  (JNIEnv*, jclass, jlong impl)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->Resume();
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    start
+ * Signature: (JLjava/lang/String;Ljava/lang/String;Ljava/lang/String;J)I
+ */
+JNIEXPORT jint JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_start
+  (JNIEnv* env, jclass, jlong impl, jstring name, jstring type,
+   jstring metadata, jlong timestamp)
+{
+  if (impl == 0) {
+    return 0;
+  }
+  return reinterpret_cast<DataLog*>(impl)->Start(
+      JStringRef{env, name}, JStringRef{env, type}, JStringRef{env, metadata},
+      timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    finish
+ * Signature: (JIJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_finish
+  (JNIEnv*, jclass, jlong impl, jint entry, jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->Finish(entry, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    setMetadata
+ * Signature: (JILjava/lang/String;J)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_setMetadata
+  (JNIEnv* env, jclass, jlong impl, jint entry, jstring metadata,
+   jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->SetMetadata(
+      entry, JStringRef{env, metadata}, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    close
+ * Signature: (J)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_close
+  (JNIEnv*, jclass, jlong impl)
+{
+  delete reinterpret_cast<DataLog*>(impl);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendRaw
+ * Signature: (JI[BJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendRaw
+  (JNIEnv* env, jclass, jlong impl, jint entry, jbyteArray value,
+   jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  JByteArrayRef cvalue{env, value};
+  reinterpret_cast<DataLog*>(impl)->AppendRaw(
+      entry,
+      {reinterpret_cast<const uint8_t*>(cvalue.array().data()), cvalue.size()},
+      timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendBoolean
+ * Signature: (JIZJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendBoolean
+  (JNIEnv*, jclass, jlong impl, jint entry, jboolean value, jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendBoolean(entry, value, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendInteger
+ * Signature: (JIJJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendInteger
+  (JNIEnv*, jclass, jlong impl, jint entry, jlong value, jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendInteger(entry, value, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendFloat
+ * Signature: (JIFJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendFloat
+  (JNIEnv*, jclass, jlong impl, jint entry, jfloat value, jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendFloat(entry, value, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendDouble
+ * Signature: (JIDJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendDouble
+  (JNIEnv*, jclass, jlong impl, jint entry, jdouble value, jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendDouble(entry, value, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendString
+ * Signature: (JILjava/lang/String;J)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendString
+  (JNIEnv* env, jclass, jlong impl, jint entry, jstring value, jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendString(entry, JStringRef{env, value},
+                                                 timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendBooleanArray
+ * Signature: (JI[ZJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendBooleanArray
+  (JNIEnv* env, jclass, jlong impl, jint entry, jbooleanArray value,
+   jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendBooleanArray(
+      entry, JBooleanArrayRef{env, value}, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendIntegerArray
+ * Signature: (JI[JJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendIntegerArray
+  (JNIEnv* env, jclass, jlong impl, jint entry, jlongArray value,
+   jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  JLongArrayRef jarr{env, value};
+  if constexpr (sizeof(jlong) == sizeof(int64_t)) {
+    reinterpret_cast<DataLog*>(impl)->AppendIntegerArray(
+        entry,
+        {reinterpret_cast<const int64_t*>(jarr.array().data()),
+         jarr.array().size()},
+        timestamp);
+  } else {
+    wpi::SmallVector<int64_t, 16> arr;
+    arr.reserve(jarr.size());
+    for (auto v : jarr.array()) {
+      arr.push_back(v);
+    }
+    reinterpret_cast<DataLog*>(impl)->AppendIntegerArray(entry, arr, timestamp);
+  }
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendFloatArray
+ * Signature: (JI[FJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendFloatArray
+  (JNIEnv* env, jclass, jlong impl, jint entry, jfloatArray value,
+   jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendFloatArray(
+      entry, JFloatArrayRef{env, value}, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendDoubleArray
+ * Signature: (JI[DJ)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendDoubleArray
+  (JNIEnv* env, jclass, jlong impl, jint entry, jdoubleArray value,
+   jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendDoubleArray(
+      entry, JDoubleArrayRef{env, value}, timestamp);
+}
+
+/*
+ * Class:     edu_wpi_first_util_datalog_DataLogJNI
+ * Method:    appendStringArray
+ * Signature: (JI[Ljava/lang/Object;J)V
+ */
+JNIEXPORT void JNICALL
+Java_edu_wpi_first_util_datalog_DataLogJNI_appendStringArray
+  (JNIEnv* env, jclass, jlong impl, jint entry, jobjectArray value,
+   jlong timestamp)
+{
+  if (impl == 0) {
+    return;
+  }
+  size_t len = env->GetArrayLength(value);
+  std::vector<std::string> arr;
+  arr.reserve(len);
+  for (size_t i = 0; i < len; ++i) {
+    JLocal<jstring> elem{
+        env, static_cast<jstring>(env->GetObjectArrayElement(value, i))};
+    if (!elem) {
+      return;
+    }
+    arr.emplace_back(JStringRef{env, elem}.str());
+  }
+  reinterpret_cast<DataLog*>(impl)->AppendStringArray(entry, arr, timestamp);
+}
+
+}  // extern "C"

wpiutil/src/main/native/include/wpi/DataLog.h

@@ -0,0 +1,680 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+#pragma once
+
+#include <stdint.h>
+
+#include <functional>
+#include <initializer_list>
+#include <memory>
+#include <string>
+#include <string_view>
+#include <thread>
+#include <vector>
+
+#include "wpi/DenseMap.h"
+#include "wpi/StringMap.h"
+#include "wpi/condition_variable.h"
+#include "wpi/mutex.h"
+#include "wpi/span.h"
+
+namespace wpi {
+class Logger;
+}  // namespace wpi
+
+namespace wpi::log {
+
+namespace impl {
+
+enum ControlRecordType {
+  kControlStart = 0,
+  kControlFinish,
+  kControlSetMetadata
+};
+
+}  // namespace impl
+
+/**
+ * A data log. The log file is created immediately upon construction with a
+ * temporary filename.  The file may be renamed at any time using the
+ * SetFilename() function.
+ *
+ * The lifetime of the data log object must be longer than any data log entry
+ * objects that refer to it.
+ *
+ * The data log is periodically flushed to disk.  It can also be explicitly
+ * flushed to disk by using the Flush() function.
+ */
+class DataLog final {
+ public:
+  /**
+   * Construct a new Data Log.  The log will be initially created with a
+   * temporary filename.
+   *
+   * @param dir directory to store the log
+   * @param filename filename to use; if none provided, a random filename is
+   *                 generated of the form "wpilog_{}.wpilog"
+   * @param period time between automatic flushes to disk, in seconds;
+   *               this is a time/storage tradeoff
+   * @param extraHeader extra header data
+   */
+  explicit DataLog(std::string_view dir = "", std::string_view filename = "",
+                   double period = 0.25, std::string_view extraHeader = "");
+
+  /**
+   * Construct a new Data Log.  The log will be initially created with a
+   * temporary filename.
+   *
+   * @param msglog message logger (will be called from separate thread)
+   * @param dir directory to store the log
+   * @param filename filename to use; if none provided, a random filename is
+   *                 generated of the form "wpilog_{}.wpilog"
+   * @param period time between automatic flushes to disk, in seconds;
+   *               this is a time/storage tradeoff
+   * @param extraHeader extra header data
+   */
+  explicit DataLog(wpi::Logger& msglog, std::string_view dir = "",
+                   std::string_view filename = "", double period = 0.25,
+                   std::string_view extraHeader = "");
+
+  /**
+   * Construct a new Data Log that passes its output to the provided function
+   * rather than a file.  The write function will be called on a separate
+   * background thread and may block.  The write function is called with an
+   * empty data array when the thread is terminating.
+   *
+   * @param write write function
+   * @param period time between automatic calls to write, in seconds;
+   *               this is a time/storage tradeoff
+   * @param extraHeader extra header data
+   */
+  explicit DataLog(std::function<void(wpi::span<const uint8_t> data)> write,
+                   double period = 0.25, std::string_view extraHeader = "");
+
+  /**
+   * Construct a new Data Log that passes its output to the provided function
+   * rather than a file.  The write function will be called on a separate
+   * background thread and may block.  The write function is called with an
+   * empty data array when the thread is terminating.
+   *
+   * @param msglog message logger (will be called from separate thread)
+   * @param write write function
+   * @param period time between automatic calls to write, in seconds;
+   *               this is a time/storage tradeoff
+   * @param extraHeader extra header data
+   */
+  explicit DataLog(wpi::Logger& msglog,
+                   std::function<void(wpi::span<const uint8_t> data)> write,
+                   double period = 0.25, std::string_view extraHeader = "");
+
+  ~DataLog();
+  DataLog(const DataLog&) = delete;
+  DataLog& operator=(const DataLog&) = delete;
+  DataLog(DataLog&&) = delete;
+  DataLog& operator=(const DataLog&&) = delete;
+
+  /**
+   * Change log filename.
+   *
+   * @param filename filename
+   */
+  void SetFilename(std::string_view filename);
+
+  /**
+   * Explicitly flushes the log data to disk.
+   */
+  void Flush();
+
+  /**
+   * Pauses appending of data records to the log.  While paused, no data records
+   * are saved (e.g. AppendX is a no-op).  Has no effect on entry starts /
+   * finishes / metadata changes.
+   */
+  void Pause();
+
+  /**
+   * Resumes appending of data records to the log.
+   */
+  void Resume();
+
+  /**
+   * Start an entry.  Duplicate names are allowed (with the same type), and
+   * result in the same index being returned (Start/Finish are reference
+   * counted).  A duplicate name with a different type will result in an error
+   * message being printed to the console and 0 being returned (which will be
+   * ignored by the Append functions).
+   *
+   * @param name Name
+   * @param type Data type
+   * @param metadata Initial metadata (e.g. data properties)
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   *
+   * @return Entry index
+   */
+  int Start(std::string_view name, std::string_view type,
+            std::string_view metadata = {}, int64_t timestamp = 0);
+
+  /**
+   * Finish an entry.
+   *
+   * @param entry Entry index
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Finish(int entry, int64_t timestamp = 0);
+
+  /**
+   * Updates the metadata for an entry.
+   *
+   * @param entry Entry index
+   * @param metadata New metadata for the entry
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void SetMetadata(int entry, std::string_view metadata, int64_t timestamp = 0);
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param entry Entry index, as returned by Start()
+   * @param data Data to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void AppendRaw(int entry, wpi::span<const uint8_t> data, int64_t timestamp);
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param entry Entry index, as returned by Start()
+   * @param data Data to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void AppendRaw2(int entry, wpi::span<const wpi::span<const uint8_t>> data,
+                  int64_t timestamp);
+
+  void AppendBoolean(int entry, bool value, int64_t timestamp);
+  void AppendInteger(int entry, int64_t value, int64_t timestamp);
+  void AppendFloat(int entry, float value, int64_t timestamp);
+  void AppendDouble(int entry, double value, int64_t timestamp);
+  void AppendString(int entry, std::string_view value, int64_t timestamp);
+  void AppendBooleanArray(int entry, wpi::span<const bool> arr,
+                          int64_t timestamp);
+  void AppendBooleanArray(int entry, wpi::span<const int> arr,
+                          int64_t timestamp);
+  void AppendBooleanArray(int entry, wpi::span<const uint8_t> arr,
+                          int64_t timestamp);
+  void AppendIntegerArray(int entry, wpi::span<const int64_t> arr,
+                          int64_t timestamp);
+  void AppendFloatArray(int entry, wpi::span<const float> arr,
+                        int64_t timestamp);
+  void AppendDoubleArray(int entry, wpi::span<const double> arr,
+                         int64_t timestamp);
+  void AppendStringArray(int entry, wpi::span<const std::string> arr,
+                         int64_t timestamp);
+  void AppendStringArray(int entry, wpi::span<const std::string_view> arr,
+                         int64_t timestamp);
+
+ private:
+  void WriterThreadMain(std::string_view dir);
+  void WriterThreadMain(
+      std::function<void(wpi::span<const uint8_t> data)> write);
+
+  // must be called with m_mutex held
+  uint8_t* StartRecord(uint32_t entry, uint64_t timestamp, uint32_t payloadSize,
+                       size_t reserveSize);
+  uint8_t* Reserve(size_t size);
+  void AppendImpl(wpi::span<const uint8_t> data);
+  void AppendStringImpl(std::string_view str);
+
+  wpi::Logger& m_msglog;
+  mutable wpi::mutex m_mutex;
+  wpi::condition_variable m_cond;
+  bool m_active{true};
+  bool m_doFlush{false};
+  bool m_paused{false};
+  double m_period;
+  std::string m_extraHeader;
+  std::string m_newFilename;
+  class Buffer;
+  std::vector<Buffer> m_free;
+  std::vector<Buffer> m_outgoing;
+  struct EntryInfo {
+    std::string type;
+    int id{0};
+  };
+  wpi::StringMap<EntryInfo> m_entries;
+  wpi::DenseMap<int, unsigned int> m_entryCounts;
+  int m_lastId = 0;
+  std::thread m_thread;
+};
+
+/**
+ * Log entry base class.
+ */
+class DataLogEntry {
+ protected:
+  DataLogEntry() = default;
+  DataLogEntry(DataLog& log, std::string_view name, std::string_view type,
+               std::string_view metadata = {}, int64_t timestamp = 0)
+      : m_log{&log}, m_entry{log.Start(name, type, metadata, timestamp)} {}
+
+ public:
+  DataLogEntry(const DataLogEntry&) = delete;
+  DataLogEntry& operator=(const DataLogEntry&) = delete;
+
+  DataLogEntry(DataLogEntry&& rhs) : m_log{rhs.m_log}, m_entry{rhs.m_entry} {
+    rhs.m_log = nullptr;
+  }
+  DataLogEntry& operator=(DataLogEntry&& rhs) {
+    if (m_log) {
+      m_log->Finish(m_entry);
+    }
+    m_log = rhs.m_log;
+    rhs.m_log = nullptr;
+    m_entry = rhs.m_entry;
+    return *this;
+  }
+
+  explicit operator bool() const { return m_log != nullptr; }
+
+  /**
+   * Updates the metadata for the entry.
+   *
+   * @param metadata New metadata for the entry
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void SetMetadata(std::string_view metadata, int64_t timestamp = 0) {
+    m_log->SetMetadata(m_entry, metadata, timestamp);
+  }
+
+  /**
+   * Finishes the entry.
+   *
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Finish(int64_t timestamp = 0) { m_log->Finish(m_entry, timestamp); }
+
+ protected:
+  DataLog* m_log = nullptr;
+  int m_entry = 0;
+};
+
+/**
+ * Log arbitrary byte data.
+ */
+class RawLogEntry : public DataLogEntry {
+ public:
+  static constexpr std::string_view kDataType = "raw";
+
+  RawLogEntry() = default;
+  RawLogEntry(DataLog& log, std::string_view name, int64_t timestamp = 0)
+      : RawLogEntry{log, name, {}, kDataType, timestamp} {}
+  RawLogEntry(DataLog& log, std::string_view name, std::string_view metadata,
+              int64_t timestamp = 0)
+      : RawLogEntry{log, name, metadata, kDataType, timestamp} {}
+  RawLogEntry(DataLog& log, std::string_view name, std::string_view metadata,
+              std::string_view type, int64_t timestamp = 0)
+      : DataLogEntry{log, name, type, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param data Data to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const uint8_t> data, int64_t timestamp = 0) {
+    m_log->AppendRaw(m_entry, data, timestamp);
+  }
+};
+
+/**
+ * Log boolean values.
+ */
+class BooleanLogEntry : public DataLogEntry {
+ public:
+  static constexpr std::string_view kDataType = "boolean";
+
+  BooleanLogEntry() = default;
+  BooleanLogEntry(DataLog& log, std::string_view name, int64_t timestamp = 0)
+      : BooleanLogEntry{log, name, {}, timestamp} {}
+  BooleanLogEntry(DataLog& log, std::string_view name,
+                  std::string_view metadata, int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(bool value, int64_t timestamp = 0) {
+    m_log->AppendBoolean(m_entry, value, timestamp);
+  }
+};
+
+/**
+ * Log integer values.
+ */
+class IntegerLogEntry : public DataLogEntry {
+ public:
+  static constexpr std::string_view kDataType = "int64";
+
+  IntegerLogEntry() = default;
+  IntegerLogEntry(DataLog& log, std::string_view name, int64_t timestamp = 0)
+      : IntegerLogEntry{log, name, {}, timestamp} {}
+  IntegerLogEntry(DataLog& log, std::string_view name,
+                  std::string_view metadata, int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(int64_t value, int64_t timestamp = 0) {
+    m_log->AppendInteger(m_entry, value, timestamp);
+  }
+};
+
+/**
+ * Log float values.
+ */
+class FloatLogEntry : public DataLogEntry {
+ public:
+  static constexpr std::string_view kDataType = "float";
+
+  FloatLogEntry() = default;
+  FloatLogEntry(DataLog& log, std::string_view name, int64_t timestamp = 0)
+      : FloatLogEntry{log, name, {}, timestamp} {}
+  FloatLogEntry(DataLog& log, std::string_view name, std::string_view metadata,
+                int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(float value, int64_t timestamp = 0) {
+    m_log->AppendFloat(m_entry, value, timestamp);
+  }
+};
+
+/**
+ * Log double values.
+ */
+class DoubleLogEntry : public DataLogEntry {
+ public:
+  static constexpr std::string_view kDataType = "double";
+
+  DoubleLogEntry() = default;
+  DoubleLogEntry(DataLog& log, std::string_view name, int64_t timestamp = 0)
+      : DoubleLogEntry{log, name, {}, timestamp} {}
+  DoubleLogEntry(DataLog& log, std::string_view name, std::string_view metadata,
+                 int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(double value, int64_t timestamp = 0) {
+    m_log->AppendDouble(m_entry, value, timestamp);
+  }
+};
+
+/**
+ * Log string values.
+ */
+class StringLogEntry : public DataLogEntry {
+ public:
+  static constexpr const char* kDataType = "string";
+
+  StringLogEntry() = default;
+  StringLogEntry(DataLog& log, std::string_view name, int64_t timestamp = 0)
+      : StringLogEntry{log, name, {}, kDataType, timestamp} {}
+  StringLogEntry(DataLog& log, std::string_view name, std::string_view metadata,
+                 int64_t timestamp = 0)
+      : StringLogEntry{log, name, metadata, kDataType, timestamp} {}
+  StringLogEntry(DataLog& log, std::string_view name, std::string_view metadata,
+                 std::string_view type, int64_t timestamp = 0)
+      : DataLogEntry{log, name, type, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param value Value to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(std::string_view value, int64_t timestamp = 0) {
+    m_log->AppendString(m_entry, value, timestamp);
+  }
+};
+
+/**
+ * Log array of boolean values.
+ */
+class BooleanArrayLogEntry : public DataLogEntry {
+ public:
+  static constexpr const char* kDataType = "boolean[]";
+
+  BooleanArrayLogEntry() = default;
+  BooleanArrayLogEntry(DataLog& log, std::string_view name,
+                       int64_t timestamp = 0)
+      : BooleanArrayLogEntry{log, name, {}, timestamp} {}
+  BooleanArrayLogEntry(DataLog& log, std::string_view name,
+                       std::string_view metadata, int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.  For find functions to work, timestamp
+   * must be monotonically increasing.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const bool> arr, int64_t timestamp = 0) {
+    m_log->AppendBooleanArray(m_entry, arr, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(std::initializer_list<bool> arr, int64_t timestamp = 0) {
+    Append(wpi::span{arr.begin(), arr.end()}, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const int> arr, int64_t timestamp = 0) {
+    m_log->AppendBooleanArray(m_entry, arr, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(std::initializer_list<int> arr, int64_t timestamp = 0) {
+    Append(wpi::span{arr.begin(), arr.end()}, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const uint8_t> arr, int64_t timestamp = 0) {
+    m_log->AppendBooleanArray(m_entry, arr, timestamp);
+  }
+};
+
+/**
+ * Log array of integer values.
+ */
+class IntegerArrayLogEntry : public DataLogEntry {
+ public:
+  static constexpr const char* kDataType = "int64[]";
+
+  IntegerArrayLogEntry() = default;
+  IntegerArrayLogEntry(DataLog& log, std::string_view name,
+                       int64_t timestamp = 0)
+      : IntegerArrayLogEntry{log, name, {}, timestamp} {}
+  IntegerArrayLogEntry(DataLog& log, std::string_view name,
+                       std::string_view metadata, int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const int64_t> arr, int64_t timestamp = 0) {
+    m_log->AppendIntegerArray(m_entry, arr, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(std::initializer_list<int64_t> arr, int64_t timestamp = 0) {
+    Append({arr.begin(), arr.end()}, timestamp);
+  }
+};
+
+/**
+ * Log array of float values.
+ */
+class FloatArrayLogEntry : public DataLogEntry {
+ public:
+  static constexpr const char* kDataType = "float[]";
+
+  FloatArrayLogEntry() = default;
+  FloatArrayLogEntry(DataLog& log, std::string_view name, int64_t timestamp = 0)
+      : FloatArrayLogEntry{log, name, {}, timestamp} {}
+  FloatArrayLogEntry(DataLog& log, std::string_view name,
+                     std::string_view metadata, int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const float> arr, int64_t timestamp = 0) {
+    m_log->AppendFloatArray(m_entry, arr, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(std::initializer_list<float> arr, int64_t timestamp = 0) {
+    Append({arr.begin(), arr.end()}, timestamp);
+  }
+};
+
+/**
+ * Log array of double values.
+ */
+class DoubleArrayLogEntry : public DataLogEntry {
+ public:
+  static constexpr const char* kDataType = "double[]";
+
+  DoubleArrayLogEntry() = default;
+  DoubleArrayLogEntry(DataLog& log, std::string_view name,
+                      int64_t timestamp = 0)
+      : DoubleArrayLogEntry{log, name, {}, timestamp} {}
+  DoubleArrayLogEntry(DataLog& log, std::string_view name,
+                      std::string_view metadata, int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const double> arr, int64_t timestamp = 0) {
+    m_log->AppendDoubleArray(m_entry, arr, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(std::initializer_list<double> arr, int64_t timestamp = 0) {
+    Append({arr.begin(), arr.end()}, timestamp);
+  }
+};
+
+/**
+ * Log array of string values.
+ */
+class StringArrayLogEntry : public DataLogEntry {
+ public:
+  static constexpr const char* kDataType = "string[]";
+
+  StringArrayLogEntry() = default;
+  StringArrayLogEntry(DataLog& log, std::string_view name,
+                      int64_t timestamp = 0)
+      : StringArrayLogEntry{log, name, {}, timestamp} {}
+  StringArrayLogEntry(DataLog& log, std::string_view name,
+                      std::string_view metadata, int64_t timestamp = 0)
+      : DataLogEntry{log, name, kDataType, metadata, timestamp} {}
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const std::string> arr, int64_t timestamp = 0) {
+    m_log->AppendStringArray(m_entry, arr, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(wpi::span<const std::string_view> arr, int64_t timestamp = 0) {
+    m_log->AppendStringArray(m_entry, arr, timestamp);
+  }
+
+  /**
+   * Appends a record to the log.
+   *
+   * @param arr Values to record
+   * @param timestamp Time stamp (may be 0 to indicate now)
+   */
+  void Append(std::initializer_list<std::string_view> arr,
+              int64_t timestamp = 0) {
+    Append(wpi::span<const std::string_view>{arr.begin(), arr.end()},
+           timestamp);
+  }
+};
+
+}  // namespace wpi::log

wpiutil/src/main/native/include/wpi/DataLogReader.h

@@ -0,0 +1,369 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+#pragma once
+
+#include <stdint.h>
+
+#include <iterator>
+#include <memory>
+#include <utility>
+#include <vector>
+
+#include "wpi/MemoryBuffer.h"
+#include "wpi/span.h"
+
+namespace wpi::log {
+
+/**
+ * Data contained in a start control record as created by DataLog::Start() when
+ * writing the log. This can be read by calling DataLogRecord::GetStartData().
+ */
+struct StartRecordData {
+  /** Entry ID; this will be used for this entry in future records. */
+  int entry;
+
+  /** Entry name. */
+  std::string_view name;
+
+  /** Type of the stored data for this entry, as a string, e.g. "double". */
+  std::string_view type;
+
+  /** Initial metadata. */
+  std::string_view metadata;
+};
+
+/**
+ * Data contained in a set metadata control record as created by
+ * DataLog::SetMetadata(). This can be read by calling
+ * DataLogRecord::GetSetMetadataData().
+ */
+struct MetadataRecordData {
+  /** Entry ID. */
+  int entry;
+
+  /** New metadata for the entry. */
+  std::string_view metadata;
+};
+
+/**
+ * A record in the data log. May represent either a control record (entry == 0)
+ * or a data record. Used only for reading (e.g. with DataLogReader).
+ */
+class DataLogRecord {
+ public:
+  DataLogRecord() = default;
+  DataLogRecord(int entry, int64_t timestamp, wpi::span<const uint8_t> data)
+      : m_timestamp{timestamp}, m_data{data}, m_entry{entry} {}
+
+  /**
+   * Gets the entry ID.
+   *
+   * @return entry ID
+   */
+  int GetEntry() const { return m_entry; }
+
+  /**
+   * Gets the record timestamp.
+   *
+   * @return Timestamp, in integer microseconds
+   */
+  int64_t GetTimestamp() const { return m_timestamp; }
+
+  /**
+   * Gets the size of the raw data.
+   *
+   * @return size
+   */
+  size_t GetSize() const { return m_data.size(); }
+
+  /**
+   * Gets the raw data. Use the GetX functions to decode based on the data type
+   * in the entry's start record.
+   */
+  wpi::span<const uint8_t> GetRaw() const { return m_data; }
+
+  /**
+   * Returns true if the record is a control record.
+   *
+   * @return True if control record, false if normal data record.
+   */
+  bool IsControl() const { return m_entry == 0; }
+
+  /**
+   * Returns true if the record is a start control record. Use GetStartData()
+   * to decode the contents.
+   *
+   * @return True if start control record, false otherwise.
+   */
+  bool IsStart() const;
+
+  /**
+   * Returns true if the record is a finish control record. Use GetFinishEntry()
+   * to decode the contents.
+   *
+   * @return True if finish control record, false otherwise.
+   */
+  bool IsFinish() const;
+
+  /**
+   * Returns true if the record is a set metadata control record. Use
+   * GetSetMetadataData() to decode the contents.
+   *
+   * @return True if set metadata control record, false otherwise.
+   */
+  bool IsSetMetadata() const;
+
+  /**
+   * Decodes a start control record.
+   *
+   * @param[out] out start record decoded data (if successful)
+   * @return True on success, false on error
+   */
+  bool GetStartData(StartRecordData* out) const;
+
+  /**
+   * Decodes a finish control record.
+   *
+   * @param[out] out finish record entry ID (if successful)
+   * @return True on success, false on error
+   */
+  bool GetFinishEntry(int* out) const;
+
+  /**
+   * Decodes a set metadata control record.
+   *
+   * @param[out] out set metadata record decoded data (if successful)
+   * @return True on success, false on error
+   */
+  bool GetSetMetadataData(MetadataRecordData* out) const;
+
+  /**
+   * Decodes a data record as a boolean. Note if the data type (as indicated in
+   * the corresponding start control record for this entry) is not "boolean",
+   * invalid results may be returned.
+   *
+   * @param[out] value boolean value (if successful)
+   * @return True on success, false on error
+   */
+  bool GetBoolean(bool* value) const;
+
+  /**
+   * Decodes a data record as an integer. Note if the data type (as indicated in
+   * the corresponding start control record for this entry) is not "int64",
+   * invalid results may be returned.
+   *
+   * @param[out] value integer value (if successful)
+   * @return True on success, false on error
+   */
+  bool GetInteger(int64_t* value) const;
+
+  /**
+   * Decodes a data record as a float. Note if the data type (as indicated in
+   * the corresponding start control record for this entry) is not "float",
+   * invalid results may be returned.
+   *
+   * @param[out] value float value (if successful)
+   * @return True on success, false on error
+   */
+  bool GetFloat(float* value) const;
+
+  /**
+   * Decodes a data record as a double. Note if the data type (as indicated in
+   * the corresponding start control record for this entry) is not "double",
+   * invalid results may be returned.
+   *
+   * @param[out] value double value (if successful)
+   * @return True on success, false on error
+   */
+  bool GetDouble(double* value) const;
+
+  /**
+   * Decodes a data record as a string. Note if the data type (as indicated in
+   * the corresponding start control record for this entry) is not "string",
+   * invalid results may be returned.
+   *
+   * @param[out] value string value
+   * @return True (never fails)
+   */
+  bool GetString(std::string_view* value) const;
+
+  /**
+   * Decodes a data record as a boolean array. Note if the data type (as
+   * indicated in the corresponding start control record for this entry) is not
+   * "boolean[]", invalid results may be returned.
+   *
+   * @param[out] arr boolean array
+   * @return True (never fails)
+   */
+  bool GetBooleanArray(std::vector<int>* arr) const;
+
+  /**
+   * Decodes a data record as an integer array. Note if the data type (as
+   * indicated in the corresponding start control record for this entry) is not
+   * "int64[]", invalid results may be returned.
+   *
+   * @param[out] arr integer array (if successful)
+   * @return True on success, false on error
+   */
+  bool GetIntegerArray(std::vector<int64_t>* arr) const;
+
+  /**
+   * Decodes a data record as a float array. Note if the data type (as
+   * indicated in the corresponding start control record for this entry) is not
+   * "float[]", invalid results may be returned.
+   *
+   * @param[out] arr float array (if successful)
+   * @return True on success, false on error
+   */
+  bool GetFloatArray(std::vector<float>* arr) const;
+
+  /**
+   * Decodes a data record as a double array. Note if the data type (as
+   * indicated in the corresponding start control record for this entry) is not
+   * "double[]", invalid results may be returned.
+   *
+   * @param[out] arr double array (if successful)
+   * @return True on success, false on error
+   */
+  bool GetDoubleArray(std::vector<double>* arr) const;
+
+  /**
+   * Decodes a data record as a string array. Note if the data type (as
+   * indicated in the corresponding start control record for this entry) is not
+   * "string[]", invalid results may be returned.
+   *
+   * @param[out] arr string array (if successful)
+   * @return True on success, false on error
+   */
+  bool GetStringArray(std::vector<std::string_view>* arr) const;
+
+ private:
+  int64_t m_timestamp{0};
+  wpi::span<const uint8_t> m_data;
+  int m_entry{-1};
+};
+
+class DataLogReader;
+
+/** DataLogReader iterator. */
+class DataLogIterator {
+ public:
+  using iterator_category = std::forward_iterator_tag;
+  using value_type = DataLogRecord;
+  using pointer = const value_type*;
+  using reference = const value_type&;
+
+  DataLogIterator(const DataLogReader* reader, size_t pos)
+      : m_reader{reader}, m_pos{pos} {}
+
+  bool operator==(const DataLogIterator& oth) const {
+    return m_reader == oth.m_reader && m_pos == oth.m_pos;
+  }
+  bool operator!=(const DataLogIterator& oth) const {
+    return !this->operator==(oth);
+  }
+
+  bool operator<(const DataLogIterator& oth) const { return m_pos < oth.m_pos; }
+  bool operator>(const DataLogIterator& oth) const {
+    return !this->operator<(oth) && !this->operator==(oth);
+  }
+  bool operator<=(const DataLogIterator& oth) const {
+    return !this->operator>(oth);
+  }
+  bool operator>=(const DataLogIterator& oth) const {
+    return !this->operator<(oth);
+  }
+
+  DataLogIterator& operator++();
+
+  DataLogIterator operator++(int) {
+    DataLogIterator tmp = *this;
+    ++*this;
+    return tmp;
+  }
+
+  reference operator*() const;
+
+  pointer operator->() const { return &this->operator*(); }
+
+ private:
+  const DataLogReader* m_reader;
+  size_t m_pos;
+  mutable bool m_valid = false;
+  mutable DataLogRecord m_value;
+};
+
+/** Data log reader (reads logs written by the DataLog class). */
+class DataLogReader {
+  friend class DataLogIterator;
+
+ public:
+  using iterator = DataLogIterator;
+
+  /** Constructs from a memory buffer. */
+  explicit DataLogReader(std::unique_ptr<MemoryBuffer> buffer);
+
+  /** Returns true if the data log is valid (e.g. has a valid header). */
+  explicit operator bool() const { return IsValid(); }
+
+  /** Returns true if the data log is valid (e.g. has a valid header). */
+  bool IsValid() const;
+
+  /**
+   * Gets the data log version. Returns 0 if data log is invalid.
+   *
+   * @return Version number; most significant byte is major, least significant
+   *         is minor (so version 1.0 will be 0x0100)
+   */
+  uint16_t GetVersion() const;
+
+  /**
+   * Gets the extra header data.
+   *
+   * @return Extra header data
+   */
+  std::string_view GetExtraHeader() const;
+
+  /**
+   * Gets the buffer identifier, typically the filename.
+   *
+   * @return Identifier string
+   */
+  std::string_view GetBufferIdentifier() const {
+    return m_buf ? m_buf->GetBufferIdentifier() : "Invalid";
+  }
+
+  /** Returns iterator to first record. */
+  iterator begin() const;
+
+  /** Returns end iterator. */
+  iterator end() const { return DataLogIterator{this, SIZE_MAX}; }
+
+ private:
+  std::unique_ptr<MemoryBuffer> m_buf;
+
+  bool GetRecord(size_t* pos, DataLogRecord* out) const;
+  bool GetNextRecord(size_t* pos) const;
+};
+
+inline DataLogIterator& DataLogIterator::operator++() {
+  if (!m_reader->GetNextRecord(&m_pos)) {
+    m_pos = SIZE_MAX;
+  }
+  m_valid = false;
+  return *this;
+}
+
+inline DataLogIterator::reference DataLogIterator::operator*() const {
+  if (!m_valid) {
+    size_t pos = m_pos;
+    if (m_reader->GetRecord(&pos, &m_value)) {
+      m_valid = true;
+    }
+  }
+  return m_value;
+}
+
+}  // namespace wpi::log

wpiutil/src/printlog/java/printlog/PrintLog.java

@@ -0,0 +1,150 @@
+// Copyright (c) FIRST and other WPILib contributors.
+// Open Source Software; you can modify and/or share it under the terms of
+// the WPILib BSD license file in the root directory of this project.
+
+package printlog;
+
+import edu.wpi.first.util.datalog.DataLogReader;
+import edu.wpi.first.util.datalog.DataLogRecord;
+import java.io.IOException;
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.time.format.DateTimeFormatter;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.InputMismatchException;
+import java.util.Map;
+
+public final class PrintLog {
+  private static final DateTimeFormatter m_timeFormatter =
+      DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+
+  /** Main function. */
+  public static void main(String[] args) {
+    if (args.length != 1) {
+      System.err.println("Usage: printlog <file>");
+      System.exit(1);
+      return;
+    }
+    DataLogReader reader;
+    try {
+      reader = new DataLogReader(args[0]);
+    } catch (IOException ex) {
+      System.err.println("could not open file: " + ex.getMessage());
+      System.exit(1);
+      return;
+    }
+    if (!reader.isValid()) {
+      System.err.println("not a log file");
+      System.exit(1);
+      return;
+    }
+
+    Map<Integer, DataLogRecord.StartRecordData> entries = new HashMap<>();
+    for (DataLogRecord record : reader) {
+      if (record.isStart()) {
+        try {
+          DataLogRecord.StartRecordData data = record.getStartData();
+          System.out.println(
+              "Start("
+                  + data.entry
+                  + ", name='"
+                  + data.name
+                  + "', type='"
+                  + data.type
+                  + "', metadata='"
+                  + data.metadata
+                  + "') ["
+                  + (record.getTimestamp() / 1000000.0)
+                  + "]");
+          if (entries.containsKey(data.entry)) {
+            System.out.println("...DUPLICATE entry ID, overriding");
+          }
+          entries.put(data.entry, data);
+        } catch (InputMismatchException ex) {
+          System.out.println("Start(INVALID)");
+        }
+      } else if (record.isFinish()) {
+        try {
+          int entry = record.getFinishEntry();
+          System.out.println("Finish(" + entry + ") [" + (record.getTimestamp() / 1000000.0) + "]");
+          if (!entries.containsKey(entry)) {
+            System.out.println("...ID not found");
+          } else {
+            entries.remove(entry);
+          }
+        } catch (InputMismatchException ex) {
+          System.out.println("Finish(INVALID)");
+        }
+      } else if (record.isSetMetadata()) {
+        try {
+          DataLogRecord.MetadataRecordData data = record.getSetMetadataData();
+          System.out.println(
+              "SetMetadata("
+                  + data.entry
+                  + ", '"
+                  + data.metadata
+                  + "') ["
+                  + (record.getTimestamp() / 1000000.0)
+                  + "]");
+          if (!entries.containsKey(data.entry)) {
+            System.out.println("...ID not found");
+          }
+        } catch (InputMismatchException ex) {
+          System.out.println("SetMetadata(INVALID)");
+        }
+      } else if (record.isControl()) {
+        System.out.println("Unrecognized control record");
+      } else {
+        System.out.print("Data(" + record.getEntry() + ", size=" + record.getSize() + ") ");
+        DataLogRecord.StartRecordData entry = entries.get(record.getEntry());
+        if (entry == null) {
+          System.out.println("<ID not found>");
+          continue;
+        }
+        System.out.println(
+            "<name='"
+                + entry.name
+                + "', type='"
+                + entry.type
+                + "'> ["
+                + (record.getTimestamp() / 1000000.0)
+                + "]");
+
+        try {
+          // handle systemTime specially
+          if ("systemTime".equals(entry.name) && "int64".equals(entry.type)) {
+            long val = record.getInteger();
+            System.out.println(
+                "  "
+                    + m_timeFormatter.format(
+                        LocalDateTime.ofEpochSecond(val / 1000000, 0, ZoneOffset.UTC))
+                    + "."
+                    + String.format("%06d", val % 1000000));
+            continue;
+          }
+
+          if ("double".equals(entry.type)) {
+            System.out.println("  " + record.getDouble());
+          } else if ("int64".equals(entry.type)) {
+            System.out.println("  " + record.getInteger());
+          } else if ("string".equals(entry.type) || "json".equals(entry.type)) {
+            System.out.println("  '" + record.getString() + "'");
+          } else if ("boolean".equals(entry.type)) {
+            System.out.println("  " + record.getBoolean());
+          } else if ("double[]".equals(entry.type)) {
+            System.out.println("  " + Arrays.asList(record.getDoubleArray()));
+          } else if ("int64[]".equals(entry.type)) {
+            System.out.println("  " + Arrays.asList(record.getIntegerArray()));
+          } else if ("string[]".equals(entry.type)) {
+            System.out.println("  " + Arrays.asList(record.getStringArray()));
+          }
+        } catch (InputMismatchException ex) {
+          System.out.println("  invalid");
+        }
+      }
+    }
+  }
+
+  private PrintLog() {}
+}