[upstream_utils] Document how to update thirdparty libraries (#4253)

Also, add a CI job to ensure the sources in the repo are consistent with
the update scripts.

.github/workflows/upstream-utils.yml

@@ -0,0 +1,53 @@
+name: Upstream utils
+
+on:
+  pull_request:
+  push:
+    branches-ignore:
+      - main
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  update:
+    name: "Update"
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Fetch all history and metadata
+        run: |
+          git fetch --prune --unshallow
+          git checkout -b pr
+          git branch -f main origin/main
+      - name: Set up Python 3.9
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.9
+      - name: Configure committer identity
+        run: |
+          git config --global user.email "you@example.com"
+          git config --global user.name "Your Name"
+      - name: Run update_drake.py
+        run: |
+          cd upstream_utils
+          ./update_drake.py
+      - name: Run update_eigen.py
+        run: |
+          cd upstream_utils
+          ./update_eigen.py
+      - name: Run update_libuv.py
+        run: |
+          cd upstream_utils
+          ./update_libuv.py
+      - name: Run update_llvm.py
+        run: |
+          cd upstream_utils
+          ./update_llvm.py
+      - name: Run update_stack_walker.py
+        run: |
+          cd upstream_utils
+          ./update_stack_walker.py
+      - name: Check output
+        run: git --no-pager diff --exit-code HEAD

upstream_utils/README.md

@@ -0,0 +1,66 @@
+# Upstream utils
+
+## Layout
+
+Each thirdparty library has a Python script for updating it. They generally:
+
+1. Check out a thirdparty Git repository to a specific commit or tag
+2. Apply patch files to the thirdparty repo to fix things specific to our build
+3. Copy a subset of the thirdparty files into our repo
+4. Comment out any header includes that were invalidated, if needed
+
+`upstream_utils.py` contains utilities common to these update scripts.
+
+Patches are generated in the thirdparty repo with `git format-patch` so they can
+be applied as individual commits and easily rebased onto newer versions. Each
+library has its own patch directory (e.g., `lib_patches`).
+
+## Updating thirdparty library version
+
+The example below will update a hypothetical library called `lib` to the tag
+`2.0`.
+
+Start in the `upstream_utils` folder. Restore the original repo.
+```bash
+./update_lib.py
+```
+
+Navigate to the repo.
+```bash
+cd /tmp/lib
+```
+
+Fetch the desired version using one of the following methods.
+```bash
+# Fetch a full branch or tag
+git fetch origin 2.0
+
+# Fetch just a tag (useful for expensive-to-clone repos)
+git fetch --depth 1 origin tag 2.0
+```
+
+Rebase any patches onto the new version.
+```bash
+git rebase 2.0
+```
+
+Generate patch files for the new version.
+```bash
+git format-patch 2.0..HEAD
+```
+
+Move the patch files to `upstream_utils`.
+```
+mv *.patch allwpilib/upstream_utils/lib_patches
+```
+
+Navigate back to `upstream_utils`
+```bash
+cd allwpilib/upstream_utils
+```
+
+Modify the version number in the call to `setup_upstream_repo()` in
+`update_lib.py`, then  rerun `update_lib.py` to reimport the thirdparty files.
+```bash
+./update_lib.py
+```

upstream_utils/upstream_utils.py

@@ -72,6 +72,7 @@ def setup_upstream_repo(url, treeish, shallow=True):
     Keyword arguments:
     url -- The URL of the git repo
     treeish -- The tree-ish to check out (branch or tag)
+    shallow -- Whether to do a shallow clone
 
     Returns:
     root -- root directory of destination Git repository