libprotobuf is a very annoying dependency to deal with, and with the switch to nanopb for generated C++ code, libprotobuf is only used for dynamic decode in the GUI apps. libprotobuf has been swapped out with upb, a much smaller C-based library that supports reflection and can therefore do dynamic decode. This means we can remove the libprotobuf dependency and stop dealing with build issues because of it.
Currently the major DataLog backend API (reading and writing) is split between wpiutil and glass. In the interest of allowing code that wants to use these APIs to not need to link to glass and declutter wpiutil, all of those APIs are moved to a new library named "datalog".
Signed-off-by: Jade Turner <spacey-sooty@proton.me>
Co-authored-by: Jade Turner <spacey-sooty@proton.me>
Co-authored-by: Gold856 <117957790+Gold856@users.noreply.github.com>
Due to how submessages are encoded (with a length prefix), nanopb currently does the encoding twice. It encodes once to get the length to write, then writes the length, then reencodes the entire message a 2nd time.
This results in a requirement that each encode always encodes the same. Generally, this is fine, but it'd be nice to not make this a requirement.
The double encode also requires going through the entire set of fields again, which has the possibility to be slow.
Instead of doing this, write to a temporary SmallVector. Then we can just encode the length of that buffer, and do a memcpy into primary stream. Theoretically in most cases, this should be much faster.
The Google C++ protobuf implementation has issues with dynamic linkage across DLL boundaries because it uses global variables. It also has a compile-time dependency because the protoc version must exactly match the libprotobuf version. Using nanopb with a customized generator fixes both of these issues.
Co-authored-by: Gold856 <117957790+Gold856@users.noreply.github.com>
This refactors Alert in both c++ and java to fix the issues with the current c++ implementation and improve performance.
Currently, constructing an Alert adds it to a list of Alerts with the same group and type. Activating an alert sets a flag on the alert. When the SendableAlerts is polled (GetStrings), the entire list is iterated over, filtered, and the filtered list is sorted by timestamp. This leads to a worst case O(m + nlog(n)) time complexity for GetStrings, where m and n are the count of total constructed alerts active alerts respectively. It also allocates intermediate data structures to hold the active alert strings for sorting.
This changes the implementation to improve the performance of GetStrings, by shifting the sorting overhead to Alert.Set
Constructing the Alert only initializes the alert's initial state, and initializes the SendableAlerts for the group if it is not already initialized.
Activating or deactivating an alert sets an internal flag for state tracking, and inserts or removes a structure containing the timestamp and text into a self-sorting data structure (std::set, TreeSet) containing other active alerts with the same group and type. (worst case O(log(n))
Now, SendableAlerts.GetStrings only has to iterate over the structure and copy the strings to the returned array. (amortized O(n))
This also fixes the c++ implementation by removing the need for SendableAlerts to directly access the Alert.
This also adds a helper method to SendableRegistry to force initialization of the instance to prevent static initialization ordering issues.
After a struct-type field descriptor had offsets calculated more than once, IsBitField would return true, causing the second call to CalculateOffsets to calculate incorrect offsets.
As string_view operations on std::map<std::string> won't be integrated
until C++26, placeholder implementations are used which are less efficient
in a couple of situations (e.g. insert with hint).
This makes it easier to define schemas when the type name is non-trivial (e.g., templated structs).
This is breaking for a) custom struct implementations and b) anything calling `wpi::Struct<T>::GetTypeString(info...)` in C++ directly. In both cases, it's a simple translation: For A, rename `GetTypeString()` to `GetTypeName()` and remove the struct: at the beginning, and for B, use `wpi::GetStructTypeString<T>(info...)` instead.
Update() checks/updates the last value and appends only if changed.
GetLastValue() gets the last value.
Also add OutputStream support to Java DataLogWriter.
Reverts #6609 since that fix didn't Just Work(tm) on Windows. (edit: or Ubuntu. Seems to have broken everything except macOS.) This PR configures CMake to try and find protobuf-config.cmake first, which allows protobuf to pull in abseil for us. If protobuf-config.cmake is not available (coprocessors which don't have a new enough protobuf installed are a common case), it will fallback to CMake's built-in FindProtobuf module, which is what we were using before.
Add wpi::CreateMessage, a wrapper with an ifdef to switch between Arena::CreateMessage and Arena::Create, since the former is deprecated in newer versions of protobuf. This allows forward compatibility with newer versions of protobuf.
Currently in the entire C API of WPILib we have ~8 different ways of handling strings. The C API actually isn't built for pure C callers (We don't actually have any of those). Instead, they're built for interop between languages like LabVIEW and C# which can talk to C API's directly.
For output parameters, the choice was fairly obvious. An output struct containing a const string pointer and a length makes the most sense. Its easy to use these from most other languages, and doesn't require special null termination handling. Freeing these is also easy, as if you ever receive one of these string structures, theres just a single function call to free it.
Input parameters are a bit more complex. To be used from pure C, and from LabVIEW, a null terminated string is the best in most cases. However, null terminated strings in general have a lot of downsides. Additionally, from LabVIEW there are other considerations around encoding that having a wrapper struct helps make a bit easier. From a language like C#, a wrapper struct is by far the easiest, as custom marshalling can make it trivial to marshal both UTF8 and UTF16 strings down.
The final consideration is its nice to have an identical concept for both input and output. It makes the rules fairly easy to understand.
WPILib will not have any APIs that manipulate a string allocated externally. This means WPI_String can be const, as across the boundary it is always const.
If a WPILib API takes a const WPI_String*, WPILib will not manipulate or attempt to free that string, and that string is treated as an input. It is up to the caller to handle that memory, WPILib will never hold onto that memory longer than the call.
If a WPILib API takes a WPI_String*, that string is an output. WPILib will allocate that API with WPI_AllocateString(), fill in the string, and return to the caller. When the caller is done with the string, they must free it with WPI_FreeString().
If an output struct contains a WPI_String member, that member is considered read only, and should not be explicitly freed. The caller should call the free function for that struct.
If an array of WPI_Strings are returned, each individual string is considered read only, and should not be explicitly freed. The free function for that array should be called by the caller.
If an input struct containing a WPI_String, or an input array of WPI_Strings is passed to WPILib, the individual strings will not be manipulated or freed by WPILib, and the caller owns and should free that memory.
Callbacks also follow these rules. The most common is a callback either getting passed a const WPI_String* or a struct containing a WPI_String. In both of these cases, the callback target should consider these strings read only, and not attempt to free them or manipulate them.
