The initial build file generation for robotpy projects was relatively
naive and purpose built to get `allwpilib` compiling, without supporting
all the available features.
This modifies the generation scripts to be able to support multiple
embedded libraries, which will be necessary for #8858, since `mrclib.so`
will need to be bundled along with the hal libraries. In addition some
cleanup was done to get the wheels looking more like what is in pypi.
There were complaints about no patch files being created from CI when
the examples pre-gen fails for people who don't build with bazel. This
adds a new action step to run just the non-robotpy pregen.
I also added an argument to the diff tests to make it a unified diff,
which might make it easier to hand fix.
This primarily fixes up the bazel publishing to match the gradle
publishing again, as some new libraries were added but not hooked up to
the maven publishing.
During the process, I noticed that the 3rd party libraries (googletest,
catch2, and imgui_suite) were still getting published on the old
`edu.wpi` namespace. I tried to clean up all the other references to
that that I could. Note: opencv and libssh are handled outside
`allwpilib` so they need to be updated separately.
Jackson is a very heavy library; it supports loads of features that we
don't need, and historically has caused issues due to long class loading
times (a little over 2 seconds to load AprilTagFieldLayout). This often
manifests as a help request in the form of "my robot disables when I do
X, but doesn't disable when doing X in subsequent attempts until code
restart." While SC has brought down Jackson loading times significantly,
with AprilTagFieldLayout loads taking only 330 milliseconds, that's
still a rather long delay, and while libraries should handle any JSON
loading ahead of time to prevent delays in auto/teleop, it would still
be good to make the worst case better to reduce user frustration.
Benchmarks indicate using [Avaje
Jsonb](https://github.com/avaje/avaje-jsonb) to load AprilTagFieldLayout
only takes ~70 ms, a fair chunk of which isn't actually in Avaje Jsonb
(~4 ms is spent on using getResourceAsStream to retrieve the JSON file,
~8 ms is spent on just loading the AprilTag class and its dependencies).
Note that all times listed are end-to-end, meaning nothing else was done
except for the operation being benchmarked, and doing arithmetic on them
can be flawed due to some classes being loaded twice, i.e.,
getResourceAsStream and `new AprilTag()` likely load some of the same
JDK classes and so subtracting both from the Avaje Jsonb load time is
likely slightly incorrect because class loading is being double counted.
For our purposes, it's likely accurate enough and is mostly just for
contextualization.
Benchmarks were run on a Raspberry Pi CM5 with 2 GB of RAM. Source code
for the
[results](https://github.com/user-attachments/files/26471452/benchmark.txt)
can be found in the "Fastjson2" commit
(2456d15ca8ebd17635e607cd40bf8816e77869a1).
Avaje Jsonb uses code generation via annotation processors to generate
the classes needed to do JSON serde and uses service providers to find
them, which will require downstream changes in robot projects, as the
different service providers in each library must be merged together for
Avaje Jsonb to function. We will use the Gradle shadow plugin, as its
already used by the installer and therefore adds zero additional
dependencies.
Robotpy dropped support for 3.10 recently, so this updates to the next
lowest version. This will be helpful when the examples get merged into
`mostrobotpy`
There's not really a specific reason why I just jumped to 3.11 instead
of a newer version. If anybody suggests otherwise I will gladly bump it
higher.
* [bazel] Package headers in ntcoreffi correctly
The original package includes headers from ntcore and wpiutil, so
include those too.
* Merge in new 2027
libprotobuf is a very annoying dependency to deal with, and with the switch to nanopb for generated C++ code, libprotobuf is only used for dynamic decode in the GUI apps. libprotobuf has been swapped out with upb, a much smaller C-based library that supports reflection and can therefore do dynamic decode. This means we can remove the libprotobuf dependency and stop dealing with build issues because of it.
This follows the gradle build accurately. Gradle copies debug symbols
into a second file (libfoo.so.debug) and links it back into the .so
file. Disable this behavior when gradle doesn't do it today.
Also, name everything correctly. When building debug builds, most
libraries get a 'd' at the end of them. Do that here too.
Currently the major DataLog backend API (reading and writing) is split between wpiutil and glass. In the interest of allowing code that wants to use these APIs to not need to link to glass and declutter wpiutil, all of those APIs are moved to a new library named "datalog".
Signed-off-by: Jade Turner <spacey-sooty@proton.me>
Co-authored-by: Jade Turner <spacey-sooty@proton.me>
Co-authored-by: Gold856 <117957790+Gold856@users.noreply.github.com>
The Google C++ protobuf implementation has issues with dynamic linkage across DLL boundaries because it uses global variables. It also has a compile-time dependency because the protoc version must exactly match the libprotobuf version. Using nanopb with a customized generator fixes both of these issues.
Co-authored-by: Gold856 <117957790+Gold856@users.noreply.github.com>
