Commit Graph

148 Commits

Author SHA1 Message Date
Peter Johnson
8efa586ace [ntcore] Don't check type string on publishing an entry (#5816)
Only check the raw type value.
2023-10-24 00:12:02 -07:00
Peter Johnson
196d963dc4 [ntcore] Fix off-by-one error in stream write (#5810) 2023-10-23 21:31:36 -07:00
Peter Johnson
1fa7445667 [ntcore] Check for valid client in incoming text and binary (#5799) 2023-10-20 23:25:05 -07:00
Peter Johnson
cf54d9ccb7 [wpiutil, ntcore] Add structured data support (#5391)
This adds support for two serialization formats for complex data types:

- Protobuf for complex objects with variable length internals that need forward and backward wire compatibility (lower speed, more flexible)
- Raw struct (ByteBuffer-style) for fixed-length objects (higher speed, less flexible)

Deserialization can be done either by creating a new object (for immutable objects) or overwriting the contents of an existing object (for mutable objects).

Implementing classes should provide inner classes that implement the Protobuf or Struct interface (in Java) or specialize the wpi::Protobuf or wpi::Struct struct (in C++). It is possible for classes to implement both. If the class itself does not implement serialization, it's possible for third parties/users to provide an implementation instead.

Uses the Google protobuf implementation for C++ and the QuickBuffers alternative protobuf implementation for Java.
2023-10-19 21:41:47 -07:00
Peter Johnson
7c6fe56cf2 [ntcore] Fix crash on disconnect (#5788)
The object was being destroyed due to the error handling path.
Instead, defer to the next loop cycle using a one-shot timer.

Properly handle error return values from Send functions.

Fix UB in accessing one past the end of a vector.
2023-10-19 14:36:22 -07:00
PJ Reiniger
210255bfff [wpiutil] Update json to 3.11.2 (#5680) 2023-10-14 21:53:56 -07:00
Peter Johnson
1d23513945 [ntcore] Fix string array value comparison (#5745)
It did not correctly check size prior to checking size=0.
2023-10-10 00:27:56 -07:00
Peter Johnson
420f2f7c80 [ntcore] Add RTT-only subprotocol (#5731)
This is useful for aliveness checking by clients that can't send
WebSocket PING messages.
2023-10-08 00:35:10 -07:00
Peter Johnson
2b63e35ded [ntcore] Fix moving outgoing queue to new period (#5735)
std::remove_if() is destructive--it can assume the removed elements are
not used, but NetworkOutgoingQueue needs them to stay intact to be moved
to a different queue. Use std::stable_partition() instead.
2023-10-08 00:34:36 -07:00
Peter Johnson
be939cb636 [ntcore] Fix notification of SetDefaultEntryValue (#5733) 2023-10-07 20:42:55 -07:00
Peter Johnson
8b7c6852cf [ntcore] Networking improvements (#5659)
- Utilize TrySend to properly backpressure network traffic
- Split updates into reasonable sized frames using WS fragmentation
- Use WS pings for network aliveness (requires 4.1 protocol revision)
- Measure RTT only at start of connection, rather than periodically
  (this avoids them being affected by other network traffic)
- Refactor network queue
- Refactor network ping, ping from server as well
- Improve meta topic performance
- Implement unified approach for network value updates (currently client and server use very different approaches) that respects requested subscriber update frequency

This adds a new protocol version (4.1) due to WS bugs in prior versions.
2023-10-04 22:02:42 -07:00
Peter Johnson
aecbcb08fc [ntcore] Correctly start DataLog for existing publishers (#5703) 2023-10-01 06:43:18 -07:00
PJ Reiniger
dd5612fbee [json] Add forward definition header (#5676) 2023-09-22 13:01:27 -07:00
Tyler Veness
5ab54ff760 Replace wpi::raw_istream with wpi::MemoryBuffer (#5675)
Instances of wpi::raw_istream were left that are reading incrementally
from file descriptors like USB devices.
2023-09-21 23:20:09 -07:00
Tyler Veness
1b6ec5a95d [wpiutil] Upgrade to LLVM 17.0.1 (#5482) 2023-09-21 19:54:33 -07:00
Peter Johnson
e93c233d60 [ntcore] Compute Value memory size when creating value (#5657)
This avoids a later switch-based computation in NetworkLoopQueue.
2023-09-17 20:01:57 -07:00
Tyler Veness
4f2114d6f5 Fix warnings from GCC 13 release build (#5637) 2023-09-15 10:37:52 -07:00
Tyler Veness
3a33ce918b [ntcore] Add missing StringMap include (#5620) 2023-09-08 20:13:57 -07:00
Tyler Veness
b7657a8e28 [wpimath] Split WPIMathJNI into logical chunks (#5552)
This makes things easier to find, and speeds up compilation.
2023-08-29 09:00:19 -07:00
Tyler Veness
165ebe4c79 Upgrade to fmt 10.1.0 (#5326) 2023-08-28 15:15:14 -07:00
Joseph Eng
2e4ad35e36 [wpiutil] jni_util: Add JSpan and CriticalJSpan (#5554)
These replace JArrayRef et al and support statically sized arrays similar to std::span.
2023-08-24 00:02:56 -07:00
Peter Johnson
89e738262c [ntcore] Limit buffer pool size to 64KB per connection (#5485)
Previously this was unlimited, which could result in holding on to a
large amount of memory if the connection got backlogged or a burst of
data transmission occurred.
2023-08-20 13:53:52 -07:00
Peter Johnson
c01814b80e [wpiutil] Add C API for DataLog (#5509) 2023-08-06 20:18:50 -07:00
Peter Johnson
b46a872494 [ntcore] Remove pImpl from implementation (#5480)
Also change Timestamped into a template.
2023-08-03 23:43:55 -07:00
Peter Johnson
7496e0d208 [ntcore] Value: More efficiently store arrays (#5484) 2023-08-01 22:26:36 -07:00
Carl Hauser
c3fab7f1f2 [ntcore] Don't update timestamp when value is unchanged (#5356)
This fixes an issue with commands run/cancel.
2023-07-23 17:36:26 -07:00
Gold856
1f6428ab63 [ntcore] Fix undefined comparison behavior when array is empty (#5448)
If both arrays are empty, it returns true, avoiding UB with memcmp potentially getting a nullptr.
2023-07-17 17:16:54 -07:00
Peter Johnson
f031513470 [ntcore] NetworkTable::GetSubTables(): Remove duplicates (#5076)
In Java, a set is used. Use a two-stage approach in C++ to achieve the
same result.
2023-07-09 21:31:58 -07:00
Tyler Veness
0eccc3f247 [ntcore] Fix clang-tidy warnings (#5405) 2023-06-20 08:21:44 -07:00
Tyler Veness
652d1c44e3 [wpiutil] Upgrade to macOS 12 to remove concept shims (#5379)
The macOS deployment target has been upgraded from 10.15 to 11. Also, a
deprecation warning for sprintf() in libuv was suppressed.
2023-06-08 19:59:54 -07:00
Tyler Veness
91cbcea841 Replace SFINAE with concepts (#5361)
Concepts are cleaner to use and result in much better error messages for incorrect template use.
2023-06-07 09:50:09 -07:00
Tyler Veness
5fac18ff4a Update formatting to clang-format 16 (#5370) 2023-05-31 22:10:53 -07:00
Peter Johnson
2108a61362 [ntcore] NT4 client: close timed-out connections (#5175)
Pings are sent every 3 seconds.  If we don't see a response to the last
ping before it's time to send the next one, close the connection.
2023-03-25 16:27:40 -07:00
Peter Johnson
0a66479693 [ntcore] Optimize scan of outgoing messages (#5227)
The algorithm being used for scanning outgoing messages was O(n^2)
because it did a full linear search and then appended. This scan is
performed for each client. If there is a burst of outgoing changes, the
outgoing queue can get quite deep all at once and this scan can be very
slow. Replacing with a map fixes this.
2023-03-25 15:20:22 -07:00
Peter Johnson
53b5fd2ace [ntcore] Use int64 for datalog type string (#5186)
There's a spec difference between NT4 and datalog for integers; NT4 uses
"int", datalog uses "int64". We were using "int" for datalog as well.

Also add backwards compatibility support to datalogtool for treating
"int" as "int64".
2023-03-13 21:28:07 -07:00
sciencewhiz
4af84a1c12 Fix Typos (NFC) (#5137) 2023-02-26 15:06:37 -08:00
Peter Johnson
ecb072724d [ntcore] Client::Disconnect(): actually close connection (#5113) 2023-02-17 23:56:49 -08:00
Peter Johnson
ba37986561 [ntcore] NetworkClient::Disconnect: Add null check (#5112) 2023-02-17 16:48:34 -08:00
Peter Johnson
8068369542 [wpinet] uv: Stop creating handles when closing loop (#5102)
This prevents EventLoopRunner::Stop() from hanging in the case when
new handles are created after the async walk closes all the handles.
2023-02-16 22:49:14 -08:00
Peter Johnson
805c837a42 [ntcore] Fix use-after-free in server (#5101)
The client name deduplication didn't properly deduplicate.  Instead,
always append the client index to guarantee a unique name.
2023-02-16 22:45:50 -08:00
Peter Johnson
843574a810 [ntcore] Use wpi::Now instead of loop time for transmit time
As the send function is called after local processing, there can be a
substantial delay between the loop time and the actual send.
2023-02-13 23:00:03 -08:00
Peter Johnson
b30664d630 [ntcore] Reduce initial connection overhead
Mixing the announce and value messages causes significant downstream
inefficiency in both time and space.
2023-02-13 23:00:03 -08:00
Peter Johnson
e57ded8c39 [ntcore] Improve disconnect error reporting (#5085)
Also fix memory leak in WebSocketConnection destructor.
2023-02-11 22:56:29 -08:00
Peter Johnson
19267bef0c [ntcore] Output warning on property set on unpublished topic (#5059)
Previously this was a debug-level message. This can primarily impact
users who call SetPersistent() on an entry before calling SetDefault().
2023-02-05 11:57:29 -08:00
Peter Johnson
84cbd48d84 [ntcore] Handle excludeSelf on SetDefault (#5058) 2023-02-05 11:57:09 -08:00
Peter Johnson
49459d3e45 [ntcore] Change wire timeout to fixed 1 second (#5048)
Previously the timeout was 10 times the update rate, so with low update
rates it could be as small as 50 ms, causing spurious disconnects when
large or many topics were published.
2023-02-03 22:05:41 -08:00
Peter Johnson
b7535252c2 [ntcore] Don't leak buffers in rare WS shutdown case (#5046)
If the request called the callback after the WebSocket had been
destroyed, the buffers were leaked.
2023-02-03 21:56:35 -08:00
Peter Johnson
b61ac6db33 [ntcore] Add client disconnect function (#5022)
As setServer doesn't disconnect, it's useful to have a function that
disconnects without needing to completely stop the client.
2023-02-03 15:28:00 -08:00
Peter Johnson
1a47cc2e86 [ntcore] Use full handle when subscribing (#5013)
Just using the index is insufficient because of Subscriber overlap with
MultiSubscriber.
2023-01-27 09:14:38 -08:00
Peter Johnson
8785bba080 [ntcore] Special-case default timestamps (#5003)
Previously, a setDefault() on the server could override a client doing a
real set() if the time offset between client and server was negative,
resulting in a negative timestamp from the client.  This is a not
uncommon situation with robot code, as the robot code always starts at
time 0, so any clients that set values earlier (in real time) would have
negative timestamps.

Also improve special casing of 0 in the transmit side to make sure a
normal timestamp will never get sent as 0.
2023-01-25 11:36:13 -08:00