Team2890/allwpilib
4
0
Fork 0
mirror of https://github.com/wpilibsuite/allwpilib synced 2026-07-03 03:01:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Don't read protobuf static data across shared library lines directly (#6979)

Browse Source 
Reading exported data from shared objects on windows is broken. It requires __declspec(dllimport). However, this is problematic, as we use the same static libraries both from a shared and static context. So we can't just blindly apply dllimport.

The linker should have caught this, as data members are exported in a different way. However, due to a bug in native-utils, data member symbols were exposed directly. However, interacting with those data member was completely broken.

The only way we can really solve this is to just not use static data members. We're pretty good about this in WPILib itself. However, protobuf is absolutely terrible at this. There are a ton of inline functions that access global data. For the protobuf library itself, we can solve this easily enough.

However, for the generated protobuf code, this is much more problematic. The member needed to bypass the global data is private. This means using just the stock protobuf code, this problem is not solvable. But, protobuf generated code has insertion points. Those insertion points let us add our own code into the generated code via a protoc plugin. And it just so happens that an insertion point exists to add extra public methodsto the generated protobuf header. There is also an insertion point to let us add to the cpp file.

The methods we need are the getters, for unpacking protobufs. For any protobuf that has a message as a member, we generate a new wpi_x() getter (the existing one is just x(), where x is the field name). We then implement this in the cpp file. A trick we can use is in the cpp file, we can safely call the x() function, as the cpp file is in the same library as the global. Thus we can call that inline method, and not actually need to directly access any internal private state of the protobuf object.

TL;DR, all protobuf classes that have messages as fields now have a wpi_x() accessor that must be used instead of x() if you want the code to work on windows. After wpilibsuite/native-utils#212, the bad code will fail to link, rather then just fail at runtime.
This commit is contained in:
Thad House
2024-08-21 07:53:20 -07:00
committed by GitHub
parent eef516fcc9
commit a9ac5b8e24
40 changed files with 443 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
upstream_utils/protobuf_patches/0001-Fix-sign-compare-warnings.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Sat, 10 Jun 2023 14:13:07 -0700
Subject: [PATCH 01/12] Fix sign-compare warnings
Subject: [PATCH 01/13] Fix sign-compare warnings
---
src/google/protobuf/compiler/importer.cc | 2 +-

2
upstream_utils/protobuf_patches/0002-Remove-redundant-move.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Sat, 10 Jun 2023 14:41:39 -0700
Subject: [PATCH 02/12] Remove redundant move
Subject: [PATCH 02/13] Remove redundant move
---
src/google/protobuf/extension_set.h | 2 +-

2
upstream_utils/protobuf_patches/0003-Fix-maybe-uninitialized-warnings.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Sat, 10 Jun 2023 15:00:20 -0700
Subject: [PATCH 03/12] Fix maybe-uninitialized warnings
Subject: [PATCH 03/13] Fix maybe-uninitialized warnings
---
src/google/protobuf/arena.cc | 6 +++---

2
upstream_utils/protobuf_patches/0004-Fix-coded_stream-WriteRaw.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Sat, 10 Jun 2023 15:03:38 -0700
Subject: [PATCH 04/12] Fix coded_stream WriteRaw
Subject: [PATCH 04/13] Fix coded_stream WriteRaw
---
src/google/protobuf/implicit_weak_message.h | 2 +-

2
upstream_utils/protobuf_patches/0005-Suppress-enum-enum-conversion-warning.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Sat, 10 Jun 2023 15:13:45 -0700
Subject: [PATCH 05/12] Suppress enum-enum conversion warning
Subject: [PATCH 05/13] Suppress enum-enum conversion warning
---
src/google/protobuf/generated_message_tctable_impl.h | 9 +++++++++

2
upstream_utils/protobuf_patches/0006-Fix-noreturn-function-returning.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Sat, 10 Jun 2023 15:16:46 -0700
Subject: [PATCH 06/12] Fix noreturn function returning
Subject: [PATCH 06/13] Fix noreturn function returning
---
src/google/protobuf/generated_message_tctable_impl.h | 3 +++

2
upstream_utils/protobuf_patches/0007-Work-around-GCC-12-restrict-warning-compiler-bug.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Sat, 10 Jun 2023 15:59:45 -0700
Subject: [PATCH 07/12] Work around GCC 12 restrict warning compiler bug
Subject: [PATCH 07/13] Work around GCC 12 restrict warning compiler bug
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105329
---

2
upstream_utils/protobuf_patches/0008-Disable-MSVC-switch-warning.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Tue, 13 Jun 2023 23:56:15 -0700
Subject: [PATCH 08/12] Disable MSVC switch warning
Subject: [PATCH 08/13] Disable MSVC switch warning
---
src/google/protobuf/generated_message_reflection.cc | 4 ++++

2
upstream_utils/protobuf_patches/0009-Disable-unused-function-warning.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Tue, 13 Jun 2023 23:58:50 -0700
Subject: [PATCH 09/12] Disable unused function warning
Subject: [PATCH 09/13] Disable unused function warning
---
src/google/protobuf/generated_message_tctable_lite.cc | 4 ++++

2
upstream_utils/protobuf_patches/0010-Disable-pedantic-warning.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Wed, 14 Jun 2023 00:02:26 -0700
Subject: [PATCH 10/12] Disable pedantic warning
Subject: [PATCH 10/13] Disable pedantic warning
---
src/google/protobuf/descriptor.h | 8 ++++++++

2
upstream_utils/protobuf_patches/0011-Avoid-use-of-sprintf.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Peter Johnson <johnson.peter@gmail.com>
Date: Mon, 9 Oct 2023 19:28:08 -0700
Subject: [PATCH 11/12] Avoid use of sprintf
Subject: [PATCH 11/13] Avoid use of sprintf
---
src/google/protobuf/stubs/strutil.cc | 14 +++++++++++---

2
upstream_utils/protobuf_patches/0012-Suppress-stringop-overflow-warning-false-positives.patch
View File

@@ -1,7 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Tyler Veness <calcmogul@gmail.com>
Date: Fri, 10 Nov 2023 14:17:53 -0800
Subject: [PATCH 12/12] Suppress stringop-overflow warning false positives
Subject: [PATCH 12/13] Suppress stringop-overflow warning false positives
---
src/google/protobuf/io/coded_stream.h | 7 +++++++

167
upstream_utils/protobuf_patches/0013-Switch-descriptor-to-not-use-globals-from-header-inl.patch Normal file
View File

@@ -0,0 +1,167 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Thad House <thadhouse1@gmail.com>
Date: Sun, 18 Aug 2024 22:43:37 -0700
Subject: [PATCH 13/13] Switch descriptor to not use globals from header inline
functions
---
src/google/protobuf/descriptor.cc | 66 +++++++++++++++++++------------
src/google/protobuf/descriptor.h | 20 +++++-----
2 files changed, 51 insertions(+), 35 deletions(-)
diff --git a/src/google/protobuf/descriptor.cc b/src/google/protobuf/descriptor.cc
index 5f3427dc72497b618c4dea3ec6985eeb39139349..5be05da4fb1c21fb1068ae7341cabef9dcf598a5 100644
--- a/src/google/protobuf/descriptor.cc
+++ b/src/google/protobuf/descriptor.cc
@@ -791,31 +791,31 @@ class Symbol {
const internal::SymbolBase* ptr_;
};
-const FieldDescriptor::CppType
- FieldDescriptor::kTypeToCppTypeMap[MAX_TYPE + 1] = {
- static_cast<CppType>(0), // 0 is reserved for errors
-
- CPPTYPE_DOUBLE, // TYPE_DOUBLE
- CPPTYPE_FLOAT, // TYPE_FLOAT
- CPPTYPE_INT64, // TYPE_INT64
- CPPTYPE_UINT64, // TYPE_UINT64
- CPPTYPE_INT32, // TYPE_INT32
- CPPTYPE_UINT64, // TYPE_FIXED64
- CPPTYPE_UINT32, // TYPE_FIXED32
- CPPTYPE_BOOL, // TYPE_BOOL
- CPPTYPE_STRING, // TYPE_STRING
- CPPTYPE_MESSAGE, // TYPE_GROUP
- CPPTYPE_MESSAGE, // TYPE_MESSAGE
- CPPTYPE_STRING, // TYPE_BYTES
- CPPTYPE_UINT32, // TYPE_UINT32
- CPPTYPE_ENUM, // TYPE_ENUM
- CPPTYPE_INT32, // TYPE_SFIXED32
- CPPTYPE_INT64, // TYPE_SFIXED64
- CPPTYPE_INT32, // TYPE_SINT32
- CPPTYPE_INT64, // TYPE_SINT64
+static const FieldDescriptor::CppType
+ kTypeToCppTypeMap[FieldDescriptor::MAX_TYPE + 1] = {
+ static_cast<FieldDescriptor::CppType>(0), // 0 is reserved for errors
+
+ FieldDescriptor::CPPTYPE_DOUBLE, // TYPE_DOUBLE
+ FieldDescriptor::CPPTYPE_FLOAT, // TYPE_FLOAT
+ FieldDescriptor::CPPTYPE_INT64, // TYPE_INT64
+ FieldDescriptor::CPPTYPE_UINT64, // TYPE_UINT64
+ FieldDescriptor::CPPTYPE_INT32, // TYPE_INT32
+ FieldDescriptor::CPPTYPE_UINT64, // TYPE_FIXED64
+ FieldDescriptor::CPPTYPE_UINT32, // TYPE_FIXED32
+ FieldDescriptor::CPPTYPE_BOOL, // TYPE_BOOL
+ FieldDescriptor::CPPTYPE_STRING, // TYPE_STRING
+ FieldDescriptor::CPPTYPE_MESSAGE, // TYPE_GROUP
+ FieldDescriptor::CPPTYPE_MESSAGE, // TYPE_MESSAGE
+ FieldDescriptor::CPPTYPE_STRING, // TYPE_BYTES
+ FieldDescriptor::CPPTYPE_UINT32, // TYPE_UINT32
+ FieldDescriptor::CPPTYPE_ENUM, // TYPE_ENUM
+ FieldDescriptor::CPPTYPE_INT32, // TYPE_SFIXED32
+ FieldDescriptor::CPPTYPE_INT64, // TYPE_SFIXED64
+ FieldDescriptor::CPPTYPE_INT32, // TYPE_SINT32
+ FieldDescriptor::CPPTYPE_INT64, // TYPE_SINT64
};
-const char* const FieldDescriptor::kTypeToName[MAX_TYPE + 1] = {
+static const char* const kTypeToName[FieldDescriptor::MAX_TYPE + 1] = {
"ERROR", // 0 is reserved for errors
"double", // TYPE_DOUBLE
@@ -838,7 +838,7 @@ const char* const FieldDescriptor::kTypeToName[MAX_TYPE + 1] = {
"sint64", // TYPE_SINT64
};
-const char* const FieldDescriptor::kCppTypeToName[MAX_CPPTYPE + 1] = {
+static const char* const kCppTypeToName[FieldDescriptor::MAX_CPPTYPE + 1] = {
"ERROR", // 0 is reserved for errors
"int32", // CPPTYPE_INT32
@@ -853,7 +853,7 @@ const char* const FieldDescriptor::kCppTypeToName[MAX_CPPTYPE + 1] = {
"message", // CPPTYPE_MESSAGE
};
-const char* const FieldDescriptor::kLabelToName[MAX_LABEL + 1] = {
+static const char* const kLabelToName[FieldDescriptor::MAX_LABEL + 1] = {
"ERROR", // 0 is reserved for errors
"optional", // LABEL_OPTIONAL
@@ -861,6 +861,22 @@ const char* const FieldDescriptor::kLabelToName[MAX_LABEL + 1] = {
"repeated", // LABEL_REPEATED
};
+const FieldDescriptor::CppType *FieldDescriptor::GetTypeToCppTypeMap() {
+ return kTypeToCppTypeMap;
+}
+
+const char* const *FieldDescriptor::GetTypeToName() {
+ return kTypeToName;
+}
+
+const char* const *FieldDescriptor::GetCppTypeToName() {
+ return kCppTypeToName;
+}
+
+const char* const *FieldDescriptor::GetLabelToName() {
+ return kLabelToName;
+}
+
const char* FileDescriptor::SyntaxName(FileDescriptor::Syntax syntax) {
switch (syntax) {
case SYNTAX_PROTO2:
diff --git a/src/google/protobuf/descriptor.h b/src/google/protobuf/descriptor.h
index bee3e32b9f1d5ba47b83d1e388716a3c3b6e82c6..1cb2421b5362a757abe3735e15321e630b1cab3e 100644
--- a/src/google/protobuf/descriptor.h
+++ b/src/google/protobuf/descriptor.h
@@ -976,13 +976,13 @@ class PROTOBUF_EXPORT FieldDescriptor : private internal::SymbolBase {
mutable std::atomic<const Message*> default_generated_instance_;
};
- static const CppType kTypeToCppTypeMap[MAX_TYPE + 1];
+ static const CppType *GetTypeToCppTypeMap();
- static const char* const kTypeToName[MAX_TYPE + 1];
+ static const char* const *GetTypeToName();
- static const char* const kCppTypeToName[MAX_CPPTYPE + 1];
+ static const char* const *GetCppTypeToName();
- static const char* const kLabelToName[MAX_LABEL + 1];
+ static const char* const *GetLabelToName();
// Must be constructed using DescriptorPool.
FieldDescriptor() {}
@@ -2392,27 +2392,27 @@ inline int MethodDescriptor::index() const {
}
inline const char* FieldDescriptor::type_name() const {
- return kTypeToName[type()];
+ return GetTypeToName()[type()];
}
inline FieldDescriptor::CppType FieldDescriptor::cpp_type() const {
- return kTypeToCppTypeMap[type()];
+ return GetTypeToCppTypeMap()[type()];
}
inline const char* FieldDescriptor::cpp_type_name() const {
- return kCppTypeToName[kTypeToCppTypeMap[type()]];
+ return GetCppTypeToName()[GetTypeToCppTypeMap()[type()]];
}
inline FieldDescriptor::CppType FieldDescriptor::TypeToCppType(Type type) {
- return kTypeToCppTypeMap[type];
+ return GetTypeToCppTypeMap()[type];
}
inline const char* FieldDescriptor::TypeName(Type type) {
- return kTypeToName[type];
+ return GetTypeToName()[type];
}
inline const char* FieldDescriptor::CppTypeName(CppType cpp_type) {
- return kCppTypeToName[cpp_type];
+ return GetCppTypeToName()[cpp_type];
}
inline bool FieldDescriptor::IsTypePackable(Type field_type) {